Dengcheng Zhu <dzhu@wavecomp.com> <dengcheng.zhu@imgtec.com>
Dengcheng Zhu <dzhu@wavecomp.com> <dengcheng.zhu@mips.com>
<dev.kurt@vandijck-laurijssen.be> <kurt.van.dijck@eia.be>
-Dmitry Eremin-Solenikov <dbaryshkov@gmail.com>
+Dmitry Baryshkov <dbaryshkov@gmail.com>
+Dmitry Baryshkov <dbaryshkov@gmail.com> <[dbaryshkov@gmail.com]>
+Dmitry Baryshkov <dbaryshkov@gmail.com> <dmitry_baryshkov@mentor.com>
+Dmitry Baryshkov <dbaryshkov@gmail.com> <dmitry_eremin@mentor.com>
Dmitry Safonov <0x7f454c46@gmail.com> <dima@arista.com>
Dmitry Safonov <0x7f454c46@gmail.com> <d.safonov@partner.samsung.com>
Dmitry Safonov <0x7f454c46@gmail.com> <dsafonov@virtuozzo.com>
Sarangdhar Joshi <spjoshi@codeaurora.org>
Sascha Hauer <s.hauer@pengutronix.de>
S.Çağlar Onur <caglar@pardus.org.tr>
+Sean Christopherson <seanjc@google.com> <sean.j.christopherson@intel.com>
Sean Nyekjaer <sean@geanix.com> <sean.nyekjaer@prevas.dk>
Sebastian Reichel <sre@kernel.org> <sebastian.reichel@collabora.co.uk>
Sebastian Reichel <sre@kernel.org> <sre@debian.org>
E: andersen@codepoet.org
W: https://www.codepoet.org/
P: 1024D/30D39057 1BC4 2742 E885 E4DE 9301 0C82 5F9B 643E 30D3 9057
-D: Maintainer of ide-cd and Uniform CD-ROM driver,
+D: Maintainer of ide-cd and Uniform CD-ROM driver,
D: ATAPI CD-Changer support, Major 2.1.x CD-ROM update.
S: 352 North 525 East
S: Springville, Utah 84663
E: pbd@op.net
D: Driver for WaveFront soundcards (Turtle Beach Maui, Tropez, Tropez+)
D: Various bugfixes and changes to sound drivers
-S: USA
+S: USA
N: Carlos Henrique Bauer
E: chbauer@acm.org
D: AX25-HOWTO, HAM-HOWTO, IPX-HOWTO, NET-2-HOWTO
D: ax25-utils maintainer.
+N: Kamil Debski
+E: kamil@wypas.org
+D: Samsung S5P 2D graphics acceleration and Multi Format Codec drivers
+D: Samsung USB2 phy drivers
+D: PWM fan driver
+
N: Helge Deller
E: deller@gmx.de
W: http://www.parisc-linux.org/
E: dfrasnel@alphalinux.org
W: http://www.alphalinux.org/
P: 1024/3EF87611 B9 F1 44 50 D3 E8 C2 80 DA E5 55 AA 56 7C 42 DA
-D: DEC Alpha hacker
+D: DEC Alpha hacker
D: Miscellaneous bug squisher
N: Jim Freeman
S: New South Wales, 2121
S: Australia
-N: Carlos E. Gorges
+N: Carlos E. Gorges
E: carlos@techlinux.com.br
D: fix smp support on cmpci driver
P: 2048G/EA3C4B19 FF31 33A6 0362 4915 B7EB E541 17D0 0379 EA3C 4B19
E: wgreathouse@myfavoritei.com
D: Current Belkin USB Serial Adapter F5U103 hacker
D: Kernel hacker, embedded systems
-S: 7802 Fitzwater Road
+S: 7802 Fitzwater Road
S: Brecksville, OH 44141-1334
S: USA
E: grant@torque.net
W: http://www.torque.net/linux-pp.html
D: original author of ppa driver for parallel port ZIP drive
-D: original architect of the parallel-port sharing scheme
+D: original architect of the parallel-port sharing scheme
D: PARIDE subsystem: drivers for parallel port IDE & ATAPI devices
S: 44 St. Joseph Street, Suite 506
S: Toronto, Ontario, M4Y 2W4
E: benh@kernel.crashing.org
D: Various parts of PPC/PPC64 & PowerMac
S: 312/107 Canberra Avenue
-S: Griffith, ACT 2603
+S: Griffith, ACT 2603
S: Australia
N: Andreas Herrmann
N: Bernhard Kaindl
E: bkaindl@netway.at
E: edv@bartelt.via.at
-D: Author of a menu based configuration tool, kmenu, which
+D: Author of a menu based configuration tool, kmenu, which
D: is the predecessor of 'make menuconfig' and 'make xconfig'.
D: digiboard driver update(modularisation work and 2.1.x upd)
S: Tallak 95
D: IP transparent proxy support
S: X/OS Experts in Open Systems BV
S: Kruislaan 419
-S: 1098 VA Amsterdam
+S: 1098 VA Amsterdam
S: The Netherlands
N: Goran Koruga
N: Andrzej M. Krzysztofowicz
E: ankry@mif.pg.gda.pl
-D: Some 8-bit XT disk driver and devfs hacking
+D: Some 8-bit XT disk driver and devfs hacking
D: Aladdin 1533/1543(C) chipset IDE
D: PIIX chipset IDE
S: ul. Matemblewska 1B/10
D: Logical Volume Manager
S: Bartningstr. 12
S: 64289 Darmstadt
-S: Germany
+S: Germany
N: Mark W. McClelland
E: mmcclell@bigfoot.com
D: RCU and variants
D: rcutorture module
-N: Mike McLagan
-E: mike.mclagan@linux.org
-W: http://www.invlogic.com/~mmclagan
-D: DLCI/FRAD drivers for Sangoma SDLAs
-S: Innovative Logic Corp
-S: Post Office Box 1068
-S: Laurel, Maryland 20732
-S: USA
-
N: Bradley McLean
E: brad@bradpc.gaylord.com
D: Device driver hacker
P: 1024/04B6E8F5 6C 77 33 CA CC D6 22 03 AB AB 15 A3 AE AD 39 7D
D: Kernel hacker. PostgreSQL hacker. Software watchdog daemon.
D: Maintainer of several Debian packages
-S: Th.-Heuss-Str. 61
+S: Th.-Heuss-Str. 61
S: D-41812 Erkelenz
S: Germany
W: http://www.i-Connect.Net/~mike/
D: Developer and maintainer of the EATA-DMA SCSI driver
D: Co-developer EATA-PIO SCSI driver
-D: /proc/scsi and assorted other snippets
+D: /proc/scsi and assorted other snippets
S: Zum Schiersteiner Grund 2
S: 55127 Mainz
S: Germany
N: Venkatesh Pallipadi (Venki)
D: x86/HPET
+N: Kyungmin Park
+E: kyungmin.park@samsung.com
+D: Samsung S5Pv210 and Exynos4210 mobile platforms
+
N: David Parsons
E: orc@pell.chi.il.us
D: improved memory detection code.
S: Chandler, Arizona 85249
S: USA
-N: Frederic Potter
+N: Frederic Potter
E: fpotter@cirpack.com
D: Some PCI kernel support
S: 76131 Karlsruhe
S: Germany
-N: James Simmons
+N: James Simmons
E: jsimmons@infradead.org
-E: jsimmons@users.sf.net
+E: jsimmons@users.sf.net
D: Frame buffer device maintainer
D: input layer development
D: tty/console layer
-D: various mipsel devices
-S: 115 Carmel Avenue
+D: various mipsel devices
+S: 115 Carmel Avenue
S: El Cerrito CA 94530
-S: USA
+S: USA
N: Jaspreet Singh
E: jaspreet@sangoma.com
W: www.sangoma.com
-D: WANPIPE drivers & API Support for Sangoma S508/FT1 cards
+D: WANPIPE drivers & API Support for Sangoma S508/FT1 cards
S: Sangoma Technologies Inc.,
S: 1001 Denison Street
S: Suite 101
E: csmall@triode.apana.org.au
E: vk2xlz@gonzo.vk2xlz.ampr.org (packet radio)
D: Gracilis PackeTwin device driver
-D: RSPF daemon
+D: RSPF daemon
S: 10 Stockalls Place
S: Minto, NSW, 2566
S: Australia
E: tsusheng@scf.usc.edu
D: IGMP(Internet Group Management Protocol) version 2
S: 2F 14 ALY 31 LN 166 SEC 1 SHIH-PEI RD
-S: Taipei
+S: Taipei
S: Taiwan 112
S: Republic of China
S: 24335 Delta Drive
D: patches for ghostscript, worked on color 'ls', etc.
S: 301 15th Street S.
S: Moorhead, Minnesota 56560
-S: USA
+S: USA
N: Jos Vos
E: jos@xos.nl
D: Various IP firewall updates, ipfwadm
S: X/OS Experts in Open Systems BV
S: Kruislaan 419
-S: 1098 VA Amsterdam
+S: 1098 VA Amsterdam
S: The Netherlands
N: Jeroen Vreeken
N: Victor Yodaiken
E: yodaiken@fsmlabs.com
D: RTLinux (RealTime Linux)
-S: POB 1822
+S: POB 1822
S: Socorro NM, 87801
S: USA
S: France
# Don't add your name here, unless you really _are_ after Marc
-# alphabetically. Leonard used to be very proud of being the
+# alphabetically. Leonard used to be very proud of being the
# last entry, and he'll get positively pissed if he can't even
# be second-to-last. (and this file really _is_ supposed to be
# in alphabetic order)
When counting down the counter start from preset value
and fire event when reach 0.
-What: /sys/bus/iio/devices/iio:deviceX/in_count_quadrature_mode_available
-KernelVersion: 4.12
-Contact: benjamin.gaignard@st.com
-Description:
- Reading returns the list possible quadrature modes.
-
-What: /sys/bus/iio/devices/iio:deviceX/in_count0_quadrature_mode
-KernelVersion: 4.12
-Contact: benjamin.gaignard@st.com
-Description:
- Configure the device counter quadrature modes:
-
- channel_A:
- Encoder A input servers as the count input and B as
- the UP/DOWN direction control input.
-
- channel_B:
- Encoder B input serves as the count input and A as
- the UP/DOWN direction control input.
-
- quadrature:
- Encoder A and B inputs are mixed to get direction
- and count with a scale of 0.25.
-
What: /sys/bus/iio/devices/iio:deviceX/in_count_enable_mode_available
KernelVersion: 4.12
Contact: benjamin.gaignard@st.com
==============================
Since the boot configuration file is loaded with initrd, it will be added
-to the end of the initrd (initramfs) image file with size, checksum and
-12-byte magic word as below.
+to the end of the initrd (initramfs) image file with padding, size,
+checksum and 12-byte magic word as below.
-[initrd][bootconfig][size(u32)][checksum(u32)][#BOOTCONFIG\n]
+[initrd][bootconfig][padding][size(le32)][checksum(le32)][#BOOTCONFIG\n]
+
+The size and checksum fields are unsigned 32bit little endian value.
+
+When the boot configuration is added to the initrd image, the total
+file size is aligned to 4 bytes. To fill the gap, null characters
+(``\0``) will be added. Thus the ``size`` is the length of the bootconfig
+file + padding bytes.
The Linux kernel decodes the last part of the initrd image in memory to
get the boot configuration data.
Because of this "piggyback" method, there is no need to change or
-update the boot loader and the kernel image itself.
+update the boot loader and the kernel image itself as long as the boot
+loader passes the correct initrd file size. If by any chance, the boot
+loader passes a longer size, the kernel feils to find the bootconfig data.
To do this operation, Linux kernel provides "bootconfig" command under
tools/bootconfig, which allows admin to apply or delete the config file
contain 256 key-value pairs. In most cases, the number of config items
will be under 100 entries and smaller than 8KB, so it would be enough.
If the node number exceeds 1024, parser returns an error even if the file
-size is smaller than 32KB.
+size is smaller than 32KB. (Note that this maximum size is not including
+the padding null characters.)
Anyway, since bootconfig command verifies it when appending a boot config
to initrd image, user can notice it before boot.
mds=off [X86]
tsx_async_abort=off [X86]
kvm.nx_huge_pages=off [X86]
+ no_entry_flush [PPC]
+ no_uaccess_flush [PPC]
Exceptions:
This does not have any effect on
noefi Disable EFI runtime services support.
+ no_entry_flush [PPC] Don't flush the L1-D cache when entering the kernel.
+
noexec [IA-64]
noexec [X86]
nospec_store_bypass_disable
[HW] Disable all mitigations for the Speculative Store Bypass vulnerability
+ no_uaccess_flush
+ [PPC] Don't flush the L1-D cache after accessing user data.
+
noxsave [BUGS=X86] Disables x86 extended register state save
and restore using xsave. The kernel will fallback to
enabling legacy floating-point and sse state.
re-run kunit_tool.
5. Try to run ``make ARCH=um defconfig`` before running ``kunit.py run``. This
may help clean up any residual config items which could be causing problems.
-6. Finally, try running KUnit outside UML. KUnit and KUnit tests can run be
+6. Finally, try running KUnit outside UML. KUnit and KUnit tests can be
built into any kernel, or can be built as a module and loaded at runtime.
Doing so should allow you to determine if UML is causing the issue you're
seeing. When tests are built-in, they will execute when the kernel boots, and
.. code-block:: none
- config FOO_KUNIT_TEST
- tristate "KUnit test for foo" if !KUNIT_ALL_TESTS
- depends on KUNIT
- default KUNIT_ALL_TESTS
- help
- This builds unit tests for foo.
+ config FOO_KUNIT_TEST
+ tristate "KUnit test for foo" if !KUNIT_ALL_TESTS
+ depends on KUNIT
+ default KUNIT_ALL_TESTS
+ help
+ This builds unit tests for foo.
- For more information on KUnit and unit tests in general, please refer
- to the KUnit documentation in Documentation/dev-tools/kunit
+ For more information on KUnit and unit tests in general, please refer
+ to the KUnit documentation in Documentation/dev-tools/kunit/.
- If unsure, say N
+ If unsure, say N.
Test File and Module Names
the second parameter, in this case, is what the value is expected to be; the
last value is what the value actually is. If ``add`` passes all of these
expectations, the test case, ``add_test_basic`` will pass; if any one of these
-expectations fail, the test case will fail.
+expectations fails, the test case will fail.
It is important to understand that a test case *fails* when any expectation is
violated; however, the test will continue running, potentially trying other
kunit_test_suite(example_test_suite);
In the above example the test suite, ``example_test_suite``, would run the test
-cases ``example_test_foo``, ``example_test_bar``, and ``example_test_baz``,
+cases ``example_test_foo``, ``example_test_bar``, and ``example_test_baz``;
each would have ``example_test_init`` called immediately before it and would
have ``example_test_exit`` called immediately after it.
``kunit_test_suite(example_test_suite)`` registers the test suite with the
such that the definition of that function can be changed without affecting the
rest of the code base. In the kernel this primarily comes from two constructs,
classes, structs that contain function pointers that are provided by the
-implementer, and architecture specific functions which have definitions selected
+implementer, and architecture-specific functions which have definitions selected
at compile time.
Classes
By default KUnit uses UML as a way to provide dependencies for code under test.
Under most circumstances KUnit's usage of UML should be treated as an
implementation detail of how KUnit works under the hood. Nevertheless, there
-are instances where being able to run architecture specific code or test
+are instances where being able to run architecture-specific code or test
against real hardware is desirable. For these reasons KUnit supports running on
other architectures.
hardware state in between test cases; if this is not possible, you may only be
able to run one test case per invocation.
-.. TODO(brendanhiggins@google.com): Add an actual example of an architecture
+.. TODO(brendanhiggins@google.com): Add an actual example of an architecture-
dependent KUnit test.
KUnit debugfs representation
};
can@53fc8000 {
- compatible = "fsl,imx53-flexcan", "fsl,p1010-flexcan";
+ compatible = "fsl,imx53-flexcan", "fsl,imx25-flexcan";
reg = <0x53fc8000 0x4000>;
interrupts = <82>;
clocks = <&clks IMX5_CLK_CAN1_IPG_GATE>, <&clks IMX5_CLK_CAN1_SERIAL_GATE>;
resets:
maxItems: 1
+ wifi-2.4ghz-coexistence:
+ type: boolean
+ description: >
+ Should the pixel frequencies in the WiFi frequencies range be
+ avoided?
+
required:
- compatible
- reg
- fsl,imx8qm-flexcan
- fsl,imx8mp-flexcan
- fsl,imx6q-flexcan
- - fsl,imx53-flexcan
- - fsl,imx35-flexcan
- fsl,imx28-flexcan
- fsl,imx25-flexcan
- fsl,p1010-flexcan
- fsl,vf610-flexcan
- fsl,ls1021ar2-flexcan
- fsl,lx2160ar1-flexcan
+ - items:
+ - enum:
+ - fsl,imx53-flexcan
+ - fsl,imx35-flexcan
+ - const: fsl,imx25-flexcan
- items:
- enum:
- fsl,imx7d-flexcan
- const: per
clock-frequency:
+ $ref: /schemas/types.yaml#/definitions/uint32
description: |
The oscillator frequency driving the flexcan device, filled in by the
boot loader. This property should only be used the used operating system
req_bit is the bit offset of CAN stop request.
$ref: /schemas/types.yaml#/definitions/phandle-array
items:
- - description: The 'gpr' is the phandle to general purpose register node.
- - description: The 'req_gpr' is the gpr register offset of CAN stop request.
- maximum: 0xff
- - description: The 'req_bit' is the bit offset of CAN stop request.
- maximum: 0x1f
+ items:
+ - description: The 'gpr' is the phandle to general purpose register node.
+ - description: The 'req_gpr' is the gpr register offset of CAN stop request.
+ maximum: 0xff
+ - description: The 'req_bit' is the bit offset of CAN stop request.
+ maximum: 0x1f
fsl,clk-source:
description: |
by default.
0: clock source 0 (oscillator clock)
1: clock source 1 (peripheral clock)
- $ref: /schemas/types.yaml#/definitions/uint32
+ $ref: /schemas/types.yaml#/definitions/uint8
default: 1
minimum: 0
maximum: 1
interrupts = <48 0x2>;
interrupt-parent = <&mpic>;
clock-frequency = <200000000>;
- fsl,clk-source = <0>;
+ fsl,clk-source = /bits/ 8 <0>;
};
- |
#include <dt-bindings/interrupt-controller/irq.h>
spi-max-frequency = <10000000>;
bosch,mram-cfg = <0x0 0 0 32 0 0 1 1>;
interrupt-parent = <&gpio1>;
- interrupts = <14 GPIO_ACTIVE_LOW>;
+ interrupts = <14 IRQ_TYPE_LEVEL_LOW>;
device-state-gpios = <&gpio3 21 GPIO_ACTIVE_HIGH>;
device-wake-gpios = <&gpio1 15 GPIO_ACTIVE_HIGH>;
reset-gpios = <&gpio1 27 GPIO_ACTIVE_HIGH>;
+++ /dev/null
-Microchip KSZ Series Ethernet switches
-==================================
-
-Required properties:
-
-- compatible: For external switch chips, compatible string must be exactly one
- of the following:
- - "microchip,ksz8765"
- - "microchip,ksz8794"
- - "microchip,ksz8795"
- - "microchip,ksz9477"
- - "microchip,ksz9897"
- - "microchip,ksz9896"
- - "microchip,ksz9567"
- - "microchip,ksz8565"
- - "microchip,ksz9893"
- - "microchip,ksz9563"
- - "microchip,ksz8563"
-
-Optional properties:
-
-- reset-gpios : Should be a gpio specifier for a reset line
-- microchip,synclko-125 : Set if the output SYNCLKO frequency should be set to
- 125MHz instead of 25MHz.
-
-See Documentation/devicetree/bindings/net/dsa/dsa.txt for a list of additional
-required and optional properties.
-
-Examples:
-
-Ethernet switch connected via SPI to the host, CPU port wired to eth0:
-
- eth0: ethernet@10001000 {
- fixed-link {
- speed = <1000>;
- full-duplex;
- };
- };
-
- spi1: spi@f8008000 {
- pinctrl-0 = <&pinctrl_spi_ksz>;
- cs-gpios = <&pioC 25 0>;
- id = <1>;
-
- ksz9477: ksz9477@0 {
- compatible = "microchip,ksz9477";
- reg = <0>;
-
- spi-max-frequency = <44000000>;
- spi-cpha;
- spi-cpol;
-
- ports {
- #address-cells = <1>;
- #size-cells = <0>;
- port@0 {
- reg = <0>;
- label = "lan1";
- };
- port@1 {
- reg = <1>;
- label = "lan2";
- };
- port@2 {
- reg = <2>;
- label = "lan3";
- };
- port@3 {
- reg = <3>;
- label = "lan4";
- };
- port@4 {
- reg = <4>;
- label = "lan5";
- };
- port@5 {
- reg = <5>;
- label = "cpu";
- ethernet = <ð0>;
- fixed-link {
- speed = <1000>;
- full-duplex;
- };
- };
- };
- };
- ksz8565: ksz8565@0 {
- compatible = "microchip,ksz8565";
- reg = <0>;
-
- spi-max-frequency = <44000000>;
- spi-cpha;
- spi-cpol;
-
- ports {
- #address-cells = <1>;
- #size-cells = <0>;
- port@0 {
- reg = <0>;
- label = "lan1";
- };
- port@1 {
- reg = <1>;
- label = "lan2";
- };
- port@2 {
- reg = <2>;
- label = "lan3";
- };
- port@3 {
- reg = <3>;
- label = "lan4";
- };
- port@6 {
- reg = <6>;
- label = "cpu";
- ethernet = <ð0>;
- fixed-link {
- speed = <1000>;
- full-duplex;
- };
- };
- };
- };
- };
--- /dev/null
+# SPDX-License-Identifier: GPL-2.0-only
+%YAML 1.2
+---
+$id: http://devicetree.org/schemas/net/dsa/microchip,ksz.yaml#
+$schema: http://devicetree.org/meta-schemas/core.yaml#
+
+title: Microchip KSZ Series Ethernet switches
+
+maintainers:
+ - Marek Vasut <marex@denx.de>
+ - Woojung Huh <Woojung.Huh@microchip.com>
+
+allOf:
+ - $ref: dsa.yaml#
+
+properties:
+ # See Documentation/devicetree/bindings/net/dsa/dsa.yaml for a list of additional
+ # required and optional properties.
+ compatible:
+ enum:
+ - microchip,ksz8765
+ - microchip,ksz8794
+ - microchip,ksz8795
+ - microchip,ksz9477
+ - microchip,ksz9897
+ - microchip,ksz9896
+ - microchip,ksz9567
+ - microchip,ksz8565
+ - microchip,ksz9893
+ - microchip,ksz9563
+ - microchip,ksz8563
+
+ reset-gpios:
+ description:
+ Should be a gpio specifier for a reset line.
+ maxItems: 1
+
+ microchip,synclko-125:
+ $ref: /schemas/types.yaml#/definitions/flag
+ description:
+ Set if the output SYNCLKO frequency should be set to 125MHz instead of 25MHz.
+
+required:
+ - compatible
+ - reg
+
+unevaluatedProperties: false
+
+examples:
+ - |
+ #include <dt-bindings/gpio/gpio.h>
+
+ // Ethernet switch connected via SPI to the host, CPU port wired to eth0:
+ eth0 {
+ fixed-link {
+ speed = <1000>;
+ full-duplex;
+ };
+ };
+
+ spi0 {
+ #address-cells = <1>;
+ #size-cells = <0>;
+
+ pinctrl-0 = <&pinctrl_spi_ksz>;
+ cs-gpios = <&pioC 25 0>;
+ id = <1>;
+
+ ksz9477: switch@0 {
+ compatible = "microchip,ksz9477";
+ reg = <0>;
+ reset-gpios = <&gpio5 0 GPIO_ACTIVE_LOW>;
+
+ spi-max-frequency = <44000000>;
+
+ ethernet-ports {
+ #address-cells = <1>;
+ #size-cells = <0>;
+ port@0 {
+ reg = <0>;
+ label = "lan1";
+ };
+ port@1 {
+ reg = <1>;
+ label = "lan2";
+ };
+ port@2 {
+ reg = <2>;
+ label = "lan3";
+ };
+ port@3 {
+ reg = <3>;
+ label = "lan4";
+ };
+ port@4 {
+ reg = <4>;
+ label = "lan5";
+ };
+ port@5 {
+ reg = <5>;
+ label = "cpu";
+ ethernet = <ð0>;
+ fixed-link {
+ speed = <1000>;
+ full-duplex;
+ };
+ };
+ };
+ };
+
+ ksz8565: switch@1 {
+ compatible = "microchip,ksz8565";
+ reg = <1>;
+
+ spi-max-frequency = <44000000>;
+
+ ethernet-ports {
+ #address-cells = <1>;
+ #size-cells = <0>;
+ port@0 {
+ reg = <0>;
+ label = "lan1";
+ };
+ port@1 {
+ reg = <1>;
+ label = "lan2";
+ };
+ port@2 {
+ reg = <2>;
+ label = "lan3";
+ };
+ port@3 {
+ reg = <3>;
+ label = "lan4";
+ };
+ port@6 {
+ reg = <6>;
+ label = "cpu";
+ ethernet = <ð0>;
+ fixed-link {
+ speed = <1000>;
+ full-duplex;
+ };
+ };
+ };
+ };
+ };
+...
- reg: address on the bus
- interrupts: GPIO interrupt to which the chip is connected
- enable-gpios: Output GPIO pin used for enabling/disabling the chip
-- firmware-gpios: Output GPIO pin used to enter firmware download mode
Optional SoC Specific Properties:
- pinctrl-names: Contains only one value - "default".
- pintctrl-0: Specifies the pin control groups used for this controller.
+- firmware-gpios: Output GPIO pin used to enter firmware download mode
Example (for ARM-based BeagleBone with NPC100 NFC controller on I2C2):
clock-frequency = <100000>;
interrupt-parent = <&gpio1>;
- interrupts = <29 GPIO_ACTIVE_HIGH>;
+ interrupts = <29 IRQ_TYPE_LEVEL_HIGH>;
enable-gpios = <&gpio0 30 GPIO_ACTIVE_HIGH>;
firmware-gpios = <&gpio0 31 GPIO_ACTIVE_HIGH>;
clock-frequency = <400000>;
interrupt-parent = <&gpio1>;
- interrupts = <17 GPIO_ACTIVE_HIGH>;
+ interrupts = <17 IRQ_TYPE_LEVEL_HIGH>;
enable-gpios = <&gpio3 21 GPIO_ACTIVE_HIGH>;
firmware-gpios = <&gpio3 19 GPIO_ACTIVE_HIGH>;
properties:
compatible:
- const: samsung,s3fwrn5-i2c
+ enum:
+ - samsung,s3fwrn5-i2c
+ - samsung,s3fwrn82
en-gpios:
maxItems: 1
required:
- compatible
- en-gpios
- - interrupts
- - reg
- wake-gpios
+allOf:
+ - if:
+ properties:
+ compatible:
+ contains:
+ const: samsung,s3fwrn5-i2c
+ then:
+ required:
+ - interrupts
+ - reg
+
examples:
- |
#include <dt-bindings/gpio/gpio.h>
wake-gpios = <&gpj0 2 GPIO_ACTIVE_HIGH>;
};
};
+ # UART example on Raspberry Pi
+ - |
+ uart0 {
+ status = "okay";
+
+ nfc {
+ compatible = "samsung,s3fwrn82";
+
+ en-gpios = <&gpio 20 GPIO_ACTIVE_HIGH>;
+ wake-gpios = <&gpio 16 GPIO_ACTIVE_HIGH>;
+
+ status = "okay";
+ };
+ };
- reg : The I2C address of the device.
+Optional properties:
+
+- realtek,power-up-delay-ms
+ Set a delay time for flush work to be completed,
+ this value is adjustable depending on platform.
Example:
rt1015: codec@28 {
compatible = "realtek,rt1015";
reg = <0x28>;
+ realtek,power-up-delay-ms = <50>;
};
Because the generator is implemented in a separate file, it can be
reused elsewhere in the media subsystem.
- Currently vidtv supports working with 3 PSI tables: PAT, PMT and
- SDT.
+ Currently vidtv supports working with 5 PSI tables: PAT, PMT,
+ SDT, NIT and EIT.
The specification for PAT and PMT can be found in *ISO 13818-1:
- Systems*, while the specification for the SDT can be found in *ETSI
+ Systems*, while the specification for the SDT, NIT, EIT can be found in *ETSI
EN 300 468: Specification for Service Information (SI) in DVB
systems*.
#. Their programs will be concatenated to populate the PAT
+ #. Their events will be concatenated to populate the EIT
+
#. For each program in the PAT, a PMT section will be created
#. The PMT section for a channel will be assigned its streams.
The first step to check whether the demod loaded successfully is to run::
$ dvb-fe-tool
+ Device Dummy demod for DVB-T/T2/C/S/S2 (/dev/dvb/adapter0/frontend0) capabilities:
+ CAN_FEC_1_2
+ CAN_FEC_2_3
+ CAN_FEC_3_4
+ CAN_FEC_4_5
+ CAN_FEC_5_6
+ CAN_FEC_6_7
+ CAN_FEC_7_8
+ CAN_FEC_8_9
+ CAN_FEC_AUTO
+ CAN_GUARD_INTERVAL_AUTO
+ CAN_HIERARCHY_AUTO
+ CAN_INVERSION_AUTO
+ CAN_QAM_16
+ CAN_QAM_32
+ CAN_QAM_64
+ CAN_QAM_128
+ CAN_QAM_256
+ CAN_QAM_AUTO
+ CAN_QPSK
+ CAN_TRANSMISSION_MODE_AUTO
+ DVB API Version 5.11, Current v5 delivery system: DVBC/ANNEX_A
+ Supported delivery systems:
+ DVBT
+ DVBT2
+ [DVBC/ANNEX_A]
+ DVBS
+ DVBS2
+ Frequency range for the current standard:
+ From: 51.0 MHz
+ To: 2.15 GHz
+ Step: 62.5 kHz
+ Tolerance: 29.5 MHz
+ Symbol rate ranges for the current standard:
+ From: 1.00 MBauds
+ To: 45.0 MBauds
This should return what is currently set up at the demod struct, i.e.::
here's an example::
[Channel]
- FREQUENCY = 330000000
+ FREQUENCY = 474000000
MODULATION = QAM/AUTO
SYMBOL_RATE = 6940000
INNER_FEC = AUTO
Assuming this channel is named 'channel.conf', you can then run::
$ dvbv5-scan channel.conf
+ dvbv5-scan ~/vidtv.conf
+ ERROR command BANDWIDTH_HZ (5) not found during retrieve
+ Cannot calc frequency shift. Either bandwidth/symbol-rate is unavailable (yet).
+ Scanning frequency #1 330000000
+ (0x00) Signal= -68.00dBm
+ Scanning frequency #2 474000000
+ Lock (0x1f) Signal= -34.45dBm C/N= 33.74dB UCB= 0
+ Service Beethoven, provider LinuxTV.org: digital television
For more information on dvb-scan, check its documentation online here:
`dvb-scan Documentation <https://www.linuxtv.org/wiki/index.php/Dvbscan>`_.
dvbv5-zap is a command line tool that can be used to record MPEG-TS to disk. The
typical use is to tune into a channel and put it into record mode. The example
-below - which is taken from the documentation - illustrates that::
+below - which is taken from the documentation - illustrates that\ [1]_::
- $ dvbv5-zap -c dvb_channel.conf "trilhas sonoras" -r
- using demux '/dev/dvb/adapter0/demux0'
+ $ dvbv5-zap -c dvb_channel.conf "beethoven" -o music.ts -P -t 10
+ using demux 'dvb0.demux0'
reading channels from file 'dvb_channel.conf'
- service has pid type 05: 204
- tuning to 573000000 Hz
- audio pid 104
- dvb_set_pesfilter 104
- Lock (0x1f) Quality= Good Signal= 100.00% C/N= -13.80dB UCB= 70 postBER= 3.14x10^-3 PER= 0
- DVR interface '/dev/dvb/adapter0/dvr0' can now be opened
+ tuning to 474000000 Hz
+ pass all PID's to TS
+ dvb_set_pesfilter 8192
+ dvb_dev_set_bufsize: buffer set to 6160384
+ Lock (0x1f) Quality= Good Signal= -34.66dBm C/N= 33.41dB UCB= 0 postBER= 0 preBER= 1.05x10^-3 PER= 0
+ Lock (0x1f) Quality= Good Signal= -34.57dBm C/N= 33.46dB UCB= 0 postBER= 0 preBER= 1.05x10^-3 PER= 0
+ Record to file 'music.ts' started
+ received 24587768 bytes (2401 Kbytes/sec)
+ Lock (0x1f) Quality= Good Signal= -34.42dBm C/N= 33.89dB UCB= 0 postBER= 0 preBER= 2.44x10^-3 PER= 0
+
+.. [1] In this example, it records 10 seconds with all program ID's stored
+ at the music.ts file.
+
-The channel can be watched by playing the contents of the DVR interface, with
-some player that recognizes the MPEG-TS format, such as *mplayer* or *vlc*.
+The channel can be watched by playing the contents of the stream with some
+player that recognizes the MPEG-TS format, such as ``mplayer`` or ``vlc``.
By playing the contents of the stream one can visually inspect the workings of
-vidtv, e.g.::
+vidtv, e.g., to play a recorded TS file with::
+
+ $ mplayer music.ts
+
+or, alternatively, running this command on one terminal::
+
+ $ dvbv5-zap -c dvb_channel.conf "beethoven" -P -r &
+
+And, on a second terminal, playing the contents from DVR interface with::
$ mplayer /dev/dvb/adapter0/dvr0
- Updating the error statistics accordingly (e.g. BER, etc).
- Simulating some noise in the encoded data.
+
+Functions and structs used within vidtv
+---------------------------------------
+
+.. kernel-doc:: drivers/media/test-drivers/vidtv/vidtv_bridge.h
+
+.. kernel-doc:: drivers/media/test-drivers/vidtv/vidtv_channel.h
+
+.. kernel-doc:: drivers/media/test-drivers/vidtv/vidtv_demod.h
+
+.. kernel-doc:: drivers/media/test-drivers/vidtv/vidtv_encoder.h
+
+.. kernel-doc:: drivers/media/test-drivers/vidtv/vidtv_mux.h
+
+.. kernel-doc:: drivers/media/test-drivers/vidtv/vidtv_pes.h
+
+.. kernel-doc:: drivers/media/test-drivers/vidtv/vidtv_psi.h
+
+.. kernel-doc:: drivers/media/test-drivers/vidtv/vidtv_s302m.h
+
+.. kernel-doc:: drivers/media/test-drivers/vidtv/vidtv_ts.h
+
+.. kernel-doc:: drivers/media/test-drivers/vidtv/vidtv_tuner.h
+
+.. kernel-doc:: drivers/media/test-drivers/vidtv/vidtv_common.c
+
+.. kernel-doc:: drivers/media/test-drivers/vidtv/vidtv_tuner.c
They can be enabled individually. The full list of the parameters: ::
make CC=clang LD=ld.lld AR=llvm-ar NM=llvm-nm STRIP=llvm-strip \
- OBJCOPY=llvm-objcopy OBJDUMP=llvm-objdump OBJSIZE=llvm-size \
- READELF=llvm-readelf HOSTCC=clang HOSTCXX=clang++ HOSTAR=llvm-ar \
- HOSTLD=ld.lld
+ OBJCOPY=llvm-objcopy OBJDUMP=llvm-objdump READELF=llvm-readelf \
+ HOSTCC=clang HOSTCXX=clang++ HOSTAR=llvm-ar HOSTLD=ld.lld
Currently, the integrated assembler is disabled by default. You can pass
``LLVM_IAS=1`` to enable it.
on the socket as usual. There are also CAN specific socket options
described below.
-The basic CAN frame structure and the sockaddr structure are defined
-in include/linux/can.h:
+The Classical CAN frame structure (aka CAN 2.0B), the CAN FD frame structure
+and the sockaddr structure are defined in include/linux/can.h:
.. code-block:: C
struct can_frame {
canid_t can_id; /* 32 bit CAN_ID + EFF/RTR/ERR flags */
- __u8 can_dlc; /* frame payload length in byte (0 .. 8) */
+ union {
+ /* CAN frame payload length in byte (0 .. CAN_MAX_DLEN)
+ * was previously named can_dlc so we need to carry that
+ * name for legacy support
+ */
+ __u8 len;
+ __u8 can_dlc; /* deprecated */
+ };
__u8 __pad; /* padding */
__u8 __res0; /* reserved / padding */
- __u8 __res1; /* reserved / padding */
+ __u8 len8_dlc; /* optional DLC for 8 byte payload length (9 .. 15) */
__u8 data[8] __attribute__((aligned(8)));
};
+Remark: The len element contains the payload length in bytes and should be
+used instead of can_dlc. The deprecated can_dlc was misleadingly named as
+it always contained the plain payload length in bytes and not the so called
+'data length code' (DLC).
+
+To pass the raw DLC from/to a Classical CAN network device the len8_dlc
+element can contain values 9 .. 15 when the len element is 8 (the real
+payload length for all DLC values greater or equal to 8).
+
The alignment of the (linear) payload data[] to a 64bit boundary
allows the user to define their own structs and unions to easily access
the CAN payload. There is no given byteorder on the CAN bus by
/* transport protocol class address info (e.g. ISOTP) */
struct { canid_t rx_id, tx_id; } tp;
+ /* J1939 address information */
+ struct {
+ /* 8 byte name when using dynamic addressing */
+ __u64 name;
+
+ /* pgn:
+ * 8 bit: PS in PDU2 case, else 0
+ * 8 bit: PF
+ * 1 bit: DP
+ * 1 bit: reserved
+ */
+ __u32 pgn;
+
+ /* 1 byte address */
+ __u8 addr;
+ } j1939;
+
/* reserved for future CAN protocols address information */
} can_addr;
};
bytes of payload (struct can_frame) like the CAN_RAW socket. Therefore e.g.
the CAN_RAW socket supports a new socket option CAN_RAW_FD_FRAMES that
switches the socket into a mode that allows the handling of CAN FD frames
-and (legacy) CAN frames simultaneously (see :ref:`socketcan-rawfd`).
+and Classical CAN frames simultaneously (see :ref:`socketcan-rawfd`).
The struct canfd_frame is defined in include/linux/can.h:
length and the DLC has a 1:1 mapping in the range of 0 .. 8. To preserve
the easy handling of the length information the canfd_frame.len element
contains a plain length value from 0 .. 64. So both canfd_frame.len and
-can_frame.can_dlc are equal and contain a length information and no DLC.
+can_frame.len are equal and contain a length information and no DLC.
For details about the distinction of CAN and CAN FD capable devices and
the mapping to the bus-relevant data length code (DLC), see :ref:`socketcan-can-fd-driver`.
.. code-block:: C
- #define CAN_MTU (sizeof(struct can_frame)) == 16 => 'legacy' CAN frame
+ #define CAN_MTU (sizeof(struct can_frame)) == 16 => Classical CAN frame
#define CANFD_MTU (sizeof(struct canfd_frame)) == 72 => CAN FD frame
printf("got CAN FD frame with length %d\n", cfd.len);
/* cfd.flags contains valid data */
} else if (nbytes == CAN_MTU) {
- printf("got legacy CAN frame with length %d\n", cfd.len);
+ printf("got Classical CAN frame with length %d\n", cfd.len);
/* cfd.flags is undefined */
} else {
fprintf(stderr, "read: invalid CAN(FD) frame\n");
printf("%02X ", cfd.data[i]);
When reading with size CANFD_MTU only returns CAN_MTU bytes that have
-been received from the socket a legacy CAN frame has been read into the
+been received from the socket a Classical CAN frame has been read into the
provided CAN FD structure. Note that the canfd_frame.flags data field is
not specified in the struct can_frame and therefore it is only valid in
CANFD_MTU sized CAN FD frames.
To build a CAN FD aware application use struct canfd_frame as basic CAN
data structure for CAN_RAW based applications. When the application is
executed on an older Linux kernel and switching the CAN_RAW_FD_FRAMES
-socket option returns an error: No problem. You'll get legacy CAN frames
+socket option returns an error: No problem. You'll get Classical CAN frames
or CAN FD frames and can process them the same way.
When sending to CAN devices make sure that the device is capable to handle
RX_RTR_FRAME:
Send reply for RTR-request (placed in op->frames[0]).
+CAN_FD_FRAME:
+ The CAN frames following the bcm_msg_head are struct canfd_frame's
Broadcast Manager Transmission Timers
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
stats - SocketCAN core statistics (rx/tx frames, match ratios, ...)
reset_stats - manual statistic reset
- version - prints the SocketCAN core version and the ABI version
+ version - prints SocketCAN core and ABI version (removed in Linux 5.10)
Writing Own CAN Protocol Modules
dev->type = ARPHRD_CAN; /* the netdevice hardware type */
dev->flags = IFF_NOARP; /* CAN has no arp */
- dev->mtu = CAN_MTU; /* sizeof(struct can_frame) -> legacy CAN interface */
+ dev->mtu = CAN_MTU; /* sizeof(struct can_frame) -> Classical CAN interface */
or alternative, when the controller supports CAN with flexible data rate:
dev->mtu = CANFD_MTU; /* sizeof(struct canfd_frame) -> CAN FD interface */
[ fd { on | off } ]
[ fd-non-iso { on | off } ]
[ presume-ack { on | off } ]
+ [ cc-len8-dlc { on | off } ]
[ restart-ms TIME-MS ]
[ restart ]
second bit timing has to be specified in order to enable the CAN FD bitrate.
Additionally CAN FD capable CAN controllers support up to 64 bytes of
-payload. The representation of this length in can_frame.can_dlc and
+payload. The representation of this length in can_frame.len and
canfd_frame.len for userspace applications and inside the Linux network
layer is a plain value from 0 .. 64 instead of the CAN 'data length code'.
-The data length code was a 1:1 mapping to the payload length in the legacy
+The data length code was a 1:1 mapping to the payload length in the Classical
CAN frames anyway. The payload length to the bus-relevant DLC mapping is
only performed inside the CAN drivers, preferably with the helper
-functions can_dlc2len() and can_len2dlc().
+functions can_fd_dlc2len() and can_fd_len2dlc().
The CAN netdevice driver capabilities can be distinguished by the network
devices maximum transfer unit (MTU)::
- MTU = 16 (CAN_MTU) => sizeof(struct can_frame) => 'legacy' CAN device
+ MTU = 16 (CAN_MTU) => sizeof(struct can_frame) => Classical CAN device
MTU = 72 (CANFD_MTU) => sizeof(struct canfd_frame) => CAN FD capable device
The CAN device MTU can be retrieved e.g. with a SIOCGIFMTU ioctl() syscall.
-N.B. CAN FD capable devices can also handle and send legacy CAN frames.
+N.B. CAN FD capable devices can also handle and send Classical CAN frames.
When configuring CAN FD capable CAN controllers an additional 'data' bitrate
has to be set. This bitrate for the data phase of the CAN FD frame has to be
* - ``esp_parsing``
- ``drop``
- Traps packets dropped due to an error in the ESP header parsing
+ * - ``blackhole_nexthop``
+ - ``drop``
+ - Traps packets that the device decided to drop in case they hit a
+ blackhole nexthop
Driver-specific Packet Traps
============================
+++ /dev/null
-.. SPDX-License-Identifier: GPL-2.0
-
-================
-Frame Relay (FR)
-================
-
-Frame Relay (FR) support for linux is built into a two tiered system of device
-drivers. The upper layer implements RFC1490 FR specification, and uses the
-Data Link Connection Identifier (DLCI) as its hardware address. Usually these
-are assigned by your network supplier, they give you the number/numbers of
-the Virtual Connections (VC) assigned to you.
-
-Each DLCI is a point-to-point link between your machine and a remote one.
-As such, a separate device is needed to accommodate the routing. Within the
-net-tools archives is 'dlcicfg'. This program will communicate with the
-base "DLCI" device, and create new net devices named 'dlci00', 'dlci01'...
-The configuration script will ask you how many DLCIs you need, as well as
-how many DLCIs you want to assign to each Frame Relay Access Device (FRAD).
-
-The DLCI uses a number of function calls to communicate with the FRAD, all
-of which are stored in the FRAD's private data area. assoc/deassoc,
-activate/deactivate and dlci_config. The DLCI supplies a receive function
-to the FRAD to accept incoming packets.
-
-With this initial offering, only 1 FRAD driver is available. With many thanks
-to Sangoma Technologies, David Mandelstam & Gene Kozin, the S502A, S502E &
-S508 are supported. This driver is currently set up for only FR, but as
-Sangoma makes more firmware modules available, it can be updated to provide
-them as well.
-
-Configuration of the FRAD makes use of another net-tools program, 'fradcfg'.
-This program makes use of a configuration file (which dlcicfg can also read)
-to specify the types of boards to be configured as FRADs, as well as perform
-any board specific configuration. The Sangoma module of fradcfg loads the
-FR firmware into the card, sets the irq/port/memory information, and provides
-an initial configuration.
-
-Additional FRAD device drivers can be added as hardware is available.
-
-At this time, the dlcicfg and fradcfg programs have not been incorporated into
-the net-tools distribution. They can be found at ftp.invlogic.com, in
-/pub/linux. Note that with OS/2 FTPD, you end up in /pub by default, so just
-use 'cd linux'. v0.10 is for use on pre-2.0.3 and earlier, v0.15 is for
-pre-2.0.4 and later.
eql
fib_trie
filter
- framerelay
generic-hdlc
generic_netlink
gen_stats
tcp-thin
team
timestamping
+ tipc
tproxy
tuntap
udplite
PGN
---
+The J1939 protocol uses the 29-bit CAN identifier with the following structure:
+
+ ============ ============== ====================
+ 29 bit CAN-ID
+ --------------------------------------------------
+ Bit positions within the CAN-ID
+ --------------------------------------------------
+ 28 ... 26 25 ... 8 7 ... 0
+ ============ ============== ====================
+ Priority PGN SA (Source Address)
+ ============ ============== ====================
+
The PGN (Parameter Group Number) is a number to identify a packet. The PGN
is composed as follows:
-1 bit : Reserved Bit
-1 bit : Data Page
-8 bits : PF (PDU Format)
-8 bits : PS (PDU Specific)
+
+ ============ ============== ================= =================
+ PGN
+ ------------------------------------------------------------------
+ Bit positions within the CAN-ID
+ ------------------------------------------------------------------
+ 25 24 23 ... 16 15 ... 8
+ ============ ============== ================= =================
+ R (Reserved) DP (Data Page) PF (PDU Format) PS (PDU Specific)
+ ============ ============== ================= =================
In J1939-21 distinction is made between PDU1 format (where PF < 240) and PDU2
format (where PF >= 240). Furthermore, when using the PDU2 format, the PS-field
contains a so-called Group Extension, which is part of the PGN. When using PDU2
format, the Group Extension is set in the PS-field.
+ ============== ========================
+ PDU1 Format (specific) (peer to peer)
+ ----------------------------------------
+ Bit positions within the CAN-ID
+ ----------------------------------------
+ 23 ... 16 15 ... 8
+ ============== ========================
+ 00h ... EFh DA (Destination address)
+ ============== ========================
+
+ ============== ========================
+ PDU2 Format (global) (broadcast)
+ ----------------------------------------
+ Bit positions within the CAN-ID
+ ----------------------------------------
+ 23 ... 16 15 ... 8
+ ============== ========================
+ F0h ... FFh GE (Group Extenstion)
+ ============== ========================
+
On the other hand, when using PDU1 format, the PS-field contains a so-called
Destination Address, which is _not_ part of the PGN. When communicating a PGN
from user space to kernel (or vice versa) and PDU2 format is used, the PS-field
minimum, your changes should survive an ``allyesconfig`` and an
``allmodconfig`` build without new warnings or failures.
+Q: How do I post corresponding changes to user space components?
+----------------------------------------------------------------
+A: User space code exercising kernel features should be posted
+alongside kernel patches. This gives reviewers a chance to see
+how any new interface is used and how well it works.
+
+When user space tools reside in the kernel repo itself all changes
+should generally come as one series. If series becomes too large
+or the user space project is not reviewed on netdev include a link
+to a public repo where user space patches can be seen.
+
+In case user space tooling lives in a separate repository but is
+reviewed on netdev (e.g. patches to `iproute2` tools) kernel and
+user space patches should form separate series (threads) when posted
+to the mailing list, e.g.::
+
+ [PATCH net-next 0/3] net: some feature cover letter
+ └─ [PATCH net-next 1/3] net: some feature prep
+ └─ [PATCH net-next 2/3] net: some feature do it
+ └─ [PATCH net-next 3/3] selftest: net: some feature
+
+ [PATCH iproute2-next] ip: add support for some feature
+
+Posting as one thread is discouraged because it confuses patchwork
+(as of patchwork 2.2.2).
+
Q: Any other tips to help ensure my net/net-next patch gets OK'd?
-----------------------------------------------------------------
A: Attention to detail. Re-read your own work as if you were the
* page_pool_get_dma_dir(): Retrieve the stored DMA direction.
+* page_pool_put_page_bulk(): Tries to refill a number of pages into the
+ ptr_ring cache holding ptr_ring producer lock. If the ptr_ring is full,
+ page_pool_put_page_bulk() will release leftover pages to the page allocator.
+ page_pool_put_page_bulk() is suitable to be run inside the driver NAPI tx
+ completion loop for the XDP_REDIRECT use case.
+ Please note the caller must not use data area after running
+ page_pool_put_page_bulk(), as this function overwrites it.
+
Coding examples
===============
--- /dev/null
+.. SPDX-License-Identifier: GPL-2.0
+
+=================
+Linux Kernel TIPC
+=================
+
+TIPC (Transparent Inter Process Communication) is a protocol that is
+specially designed for intra-cluster communication.
+
+For more information about TIPC, see http://tipc.sourceforge.net.
+
+TIPC Base Types
+---------------
+
+.. kernel-doc:: net/tipc/subscr.h
+ :internal:
+
+.. kernel-doc:: net/tipc/bearer.h
+ :internal:
+
+.. kernel-doc:: net/tipc/name_table.h
+ :internal:
+
+.. kernel-doc:: net/tipc/name_distr.h
+ :internal:
+
+.. kernel-doc:: net/tipc/bcast.c
+ :internal:
+
+TIPC Bearer Interfaces
+----------------------
+
+.. kernel-doc:: net/tipc/bearer.c
+ :internal:
+
+.. kernel-doc:: net/tipc/udp_media.c
+ :internal:
+
+TIPC Crypto Interfaces
+----------------------
+
+.. kernel-doc:: net/tipc/crypto.c
+ :internal:
+
+TIPC Discoverer Interfaces
+--------------------------
+
+.. kernel-doc:: net/tipc/discover.c
+ :internal:
+
+TIPC Link Interfaces
+--------------------
+
+.. kernel-doc:: net/tipc/link.c
+ :internal:
+
+TIPC msg Interfaces
+-------------------
+
+.. kernel-doc:: net/tipc/msg.c
+ :internal:
+
+TIPC Name Interfaces
+--------------------
+
+.. kernel-doc:: net/tipc/name_table.c
+ :internal:
+
+.. kernel-doc:: net/tipc/name_distr.c
+ :internal:
+
+TIPC Node Management Interfaces
+-------------------------------
+
+.. kernel-doc:: net/tipc/node.c
+ :internal:
+
+TIPC Socket Interfaces
+----------------------
+
+.. kernel-doc:: net/tipc/socket.c
+ :internal:
+
+TIPC Network Topology Interfaces
+--------------------------------
+
+.. kernel-doc:: net/tipc/subscr.c
+ :internal:
+
+TIPC Server Interfaces
+----------------------
+
+.. kernel-doc:: net/tipc/topsrv.c
+ :internal:
+
+TIPC Trace Interfaces
+---------------------
+
+.. kernel-doc:: net/tipc/trace.c
+ :internal:
+------------------+
| VMALLOC area | VMALLOC_START 0xc0000000 128MB - 64KB
+------------------+ VMALLOC_END
- | Cache aliasing | TLBTEMP_BASE_1 0xc7ff0000 DCACHE_WAY_SIZE
+ +------------------+
+ | Cache aliasing | TLBTEMP_BASE_1 0xc8000000 DCACHE_WAY_SIZE
| remap area 1 |
+------------------+
| Cache aliasing | TLBTEMP_BASE_2 DCACHE_WAY_SIZE
+------------------+
| VMALLOC area | VMALLOC_START 0xa0000000 128MB - 64KB
+------------------+ VMALLOC_END
- | Cache aliasing | TLBTEMP_BASE_1 0xa7ff0000 DCACHE_WAY_SIZE
+ +------------------+
+ | Cache aliasing | TLBTEMP_BASE_1 0xa8000000 DCACHE_WAY_SIZE
| remap area 1 |
+------------------+
| Cache aliasing | TLBTEMP_BASE_2 DCACHE_WAY_SIZE
+------------------+
| VMALLOC area | VMALLOC_START 0x90000000 128MB - 64KB
+------------------+ VMALLOC_END
- | Cache aliasing | TLBTEMP_BASE_1 0x97ff0000 DCACHE_WAY_SIZE
+ +------------------+
+ | Cache aliasing | TLBTEMP_BASE_1 0x98000000 DCACHE_WAY_SIZE
| remap area 1 |
+------------------+
| Cache aliasing | TLBTEMP_BASE_2 DCACHE_WAY_SIZE
ARM/Allwinner sunXi SoC support
M: Maxime Ripard <mripard@kernel.org>
M: Chen-Yu Tsai <wens@csie.org>
+R: Jernej Skrabec <jernej.skrabec@siol.net>
L: linux-arm-kernel@lists.infradead.org (moderated for non-subscribers)
S: Maintained
T: git git://git.kernel.org/pub/scm/linux/kernel/git/sunxi/linux.git
ARM/CORESIGHT FRAMEWORK AND DRIVERS
M: Mathieu Poirier <mathieu.poirier@linaro.org>
-R: Suzuki K Poulose <suzuki.poulose@arm.com>
+M: Suzuki K Poulose <suzuki.poulose@arm.com>
R: Mike Leach <mike.leach@linaro.org>
+R: Leo Yan <leo.yan@linaro.org>
L: coresight@lists.linaro.org (moderated for non-subscribers)
L: linux-arm-kernel@lists.infradead.org (moderated for non-subscribers)
S: Maintained
+T: git git://git.kernel.org/pub/scm/linux/kernel/git/coresight/linux.git
F: Documentation/ABI/testing/sysfs-bus-coresight-devices-*
F: Documentation/devicetree/bindings/arm/coresight-cpu-debug.txt
F: Documentation/devicetree/bindings/arm/coresight-cti.yaml
ARM/LPC32XX SOC SUPPORT
M: Vladimir Zapolskiy <vz@mleia.com>
-M: Sylvain Lemieux <slemieux.tyco@gmail.com>
L: linux-arm-kernel@lists.infradead.org (moderated for non-subscribers)
S: Maintained
T: git git://github.com/vzapolskiy/linux-lpc32xx.git
F: sound/soc/rockchip/
N: rockchip
-ARM/SAMSUNG EXYNOS ARM ARCHITECTURES
+ARM/SAMSUNG S3C, S5P AND EXYNOS ARM ARCHITECTURES
M: Krzysztof Kozlowski <krzk@kernel.org>
L: linux-arm-kernel@lists.infradead.org (moderated for non-subscribers)
L: linux-samsung-soc@vger.kernel.org
N: s3c64xx
N: s5pv210
-ARM/SAMSUNG MOBILE MACHINE SUPPORT
-M: Kyungmin Park <kyungmin.park@samsung.com>
-L: linux-arm-kernel@lists.infradead.org (moderated for non-subscribers)
-S: Maintained
-F: arch/arm/mach-s5pv210/
-
ARM/SAMSUNG S5P SERIES 2D GRAPHICS ACCELERATION (G2D) SUPPORT
-M: Kyungmin Park <kyungmin.park@samsung.com>
-M: Kamil Debski <kamil@wypas.org>
M: Andrzej Hajda <a.hajda@samsung.com>
L: linux-arm-kernel@lists.infradead.org
L: linux-media@vger.kernel.org
F: drivers/media/platform/s5p-jpeg/
ARM/SAMSUNG S5P SERIES Multi Format Codec (MFC) SUPPORT
-M: Kyungmin Park <kyungmin.park@samsung.com>
-M: Kamil Debski <kamil@wypas.org>
-M: Jeongtae Park <jtp.park@samsung.com>
M: Andrzej Hajda <a.hajda@samsung.com>
L: linux-arm-kernel@lists.infradead.org
L: linux-media@vger.kernel.org
BPF (Safe dynamic programs and tools)
M: Alexei Starovoitov <ast@kernel.org>
M: Daniel Borkmann <daniel@iogearbox.net>
+M: Andrii Nakryiko <andrii@kernel.org>
R: Martin KaFai Lau <kafai@fb.com>
R: Song Liu <songliubraving@fb.com>
R: Yonghong Song <yhs@fb.com>
-R: Andrii Nakryiko <andrii@kernel.org>
R: John Fastabend <john.fastabend@gmail.com>
R: KP Singh <kpsingh@chromium.org>
L: netdev@vger.kernel.org
F: arch/x86/net/
X: arch/x86/net/bpf_jit_comp32.c
+BPF LSM (Security Audit and Enforcement using BPF)
+M: KP Singh <kpsingh@chromium.org>
+R: Florent Revest <revest@chromium.org>
+R: Brendan Jackman <jackmanb@chromium.org>
+L: bpf@vger.kernel.org
+S: Maintained
+F: Documentation/bpf/bpf_lsm.rst
+F: include/linux/bpf_lsm.h
+F: kernel/bpf/bpf_lsm.c
+F: security/bpf/
+
BROADCOM B44 10/100 ETHERNET DRIVER
M: Michael Chan <michael.chan@broadcom.com>
L: netdev@vger.kernel.org
M: Arend van Spriel <arend.vanspriel@broadcom.com>
M: Franky Lin <franky.lin@broadcom.com>
M: Hante Meuleman <hante.meuleman@broadcom.com>
-M: Chi-Hsien Lin <chi-hsien.lin@cypress.com>
-M: Wright Feng <wright.feng@cypress.com>
+M: Chi-hsien Lin <chi-hsien.lin@infineon.com>
+M: Wright Feng <wright.feng@infineon.com>
+M: Chung-hsien Hsu <chung-hsien.hsu@infineon.com>
L: linux-wireless@vger.kernel.org
L: brcm80211-dev-list.pdl@broadcom.com
-L: brcm80211-dev-list@cypress.com
+L: SHA-cyfmac-dev-list@infineon.com
S: Supported
F: drivers/net/wireless/broadcom/brcm80211/
F: drivers/media/dvb-frontends/cxd2820r*
CXGB3 ETHERNET DRIVER (CXGB3)
-M: Vishal Kulkarni <vishal@chelsio.com>
+M: Raju Rangoju <rajur@chelsio.com>
L: netdev@vger.kernel.org
S: Supported
W: http://www.chelsio.com
F: drivers/net/ethernet/chelsio/inline_crypto/
CXGB4 ETHERNET DRIVER (CXGB4)
-M: Vishal Kulkarni <vishal@chelsio.com>
+M: Raju Rangoju <rajur@chelsio.com>
L: netdev@vger.kernel.org
S: Supported
W: http://www.chelsio.com
F: include/uapi/rdma/cxgb4-abi.h
CXGB4VF ETHERNET DRIVER (CXGB4VF)
-M: Vishal Kulkarni <vishal@gmail.com>
+M: Raju Rangoju <rajur@chelsio.com>
L: netdev@vger.kernel.org
S: Supported
W: http://www.chelsio.com
W: http://floatingpoint.sourceforge.net/emulator/index.html
F: arch/x86/math-emu/
-FRAME RELAY DLCI/FRAD (Sangoma drivers too)
-L: netdev@vger.kernel.org
-S: Orphan
-F: drivers/net/wan/dlci.c
-F: drivers/net/wan/sdla.c
-
FRAMEBUFFER LAYER
M: Bartlomiej Zolnierkiewicz <b.zolnierkie@samsung.com>
L: dri-devel@lists.freedesktop.org
F: drivers/net/wireless/intel/iwlegacy/
INTEL WIRELESS WIFI LINK (iwlwifi)
-M: Johannes Berg <johannes.berg@intel.com>
-M: Emmanuel Grumbach <emmanuel.grumbach@intel.com>
M: Luca Coelho <luciano.coelho@intel.com>
-M: Intel Linux Wireless <linuxwifi@intel.com>
L: linux-wireless@vger.kernel.org
S: Supported
W: https://wireless.wiki.kernel.org/en/users/drivers/iwlwifi
IOMMU DRIVERS
M: Joerg Roedel <joro@8bytes.org>
+M: Will Deacon <will@kernel.org>
L: iommu@lists.linux-foundation.org
S: Maintained
T: git git://git.kernel.org/pub/scm/linux/kernel/git/joro/iommu.git
F: arch/s390/include/asm/gmap.h
F: arch/s390/include/asm/kvm*
F: arch/s390/include/uapi/asm/kvm*
+F: arch/s390/kernel/uv.c
F: arch/s390/kvm/
F: arch/s390/mm/gmap.c
F: tools/testing/selftests/kvm/*/s390x/
F: arch/mips/lantiq
F: drivers/soc/lantiq
-LAPB module
-L: linux-x25@vger.kernel.org
-S: Orphan
-F: Documentation/networking/lapb-module.rst
-F: include/*/lapb.h
-F: net/lapb/
-
LASI 53c700 driver for PARISC
M: "James E.J. Bottomley" <James.Bottomley@HansenPartnership.com>
L: linux-scsi@vger.kernel.org
L: linux-crypto@vger.kernel.org
S: Maintained
F: drivers/crypto/marvell/
+F: include/linux/soc/marvell/octeontx2/
MARVELL GIGABIT ETHERNET DRIVERS (skge/sky2)
M: Mirko Lindner <mlindner@marvell.com>
L: netdev@vger.kernel.org
S: Supported
F: drivers/net/ethernet/marvell/octeontx2/nic/
+F: include/linux/soc/marvell/octeontx2/
MARVELL OCTEONTX2 RVU ADMIN FUNCTION DRIVER
M: Sunil Goutham <sgoutham@marvell.com>
M: Microchip Linux Driver Support <UNGLinuxDriver@microchip.com>
L: netdev@vger.kernel.org
S: Maintained
-F: Documentation/devicetree/bindings/net/dsa/ksz.txt
+F: Documentation/devicetree/bindings/net/dsa/microchip,ksz.yaml
F: drivers/net/dsa/microchip/*
F: include/linux/platform_data/microchip-ksz.h
F: net/dsa/tag_ksz.c
M: Ilias Apalodimas <ilias.apalodimas@linaro.org>
L: netdev@vger.kernel.org
S: Supported
+F: Documentation/networking/page_pool.rst
F: include/net/page_pool.h
+F: include/trace/events/page_pool.h
F: net/core/page_pool.c
PANASONIC LAPTOP ACPI EXTRAS DRIVER
F: include/trace/events/pwc.h
PWM FAN DRIVER
-M: Kamil Debski <kamil@wypas.org>
M: Bartlomiej Zolnierkiewicz <b.zolnierkie@samsung.com>
L: linux-hwmon@vger.kernel.org
S: Supported
F: drivers/net/wireless/realtek/rtlwifi/
REALTEK WIRELESS DRIVER (rtw88)
-M: Yan-Hsuan Chuang <yhchuang@realtek.com>
+M: Yan-Hsuan Chuang <tony0620emma@gmail.com>
L: linux-wireless@vger.kernel.org
S: Maintained
F: drivers/net/wireless/realtek/rtw88/
F: drivers/nfc/s3fwrn5
SAMSUNG S5C73M3 CAMERA DRIVER
-M: Kyungmin Park <kyungmin.park@samsung.com>
M: Andrzej Hajda <a.hajda@samsung.com>
L: linux-media@vger.kernel.org
S: Supported
F: drivers/media/i2c/s5c73m3/*
SAMSUNG S5K5BAF CAMERA DRIVER
-M: Kyungmin Park <kyungmin.park@samsung.com>
M: Andrzej Hajda <a.hajda@samsung.com>
L: linux-media@vger.kernel.org
S: Supported
F: drivers/crypto/s5p-sss.c
SAMSUNG S5P/EXYNOS4 SOC SERIES CAMERA SUBSYSTEM DRIVERS
-M: Kyungmin Park <kyungmin.park@samsung.com>
M: Sylwester Nawrocki <s.nawrocki@samsung.com>
L: linux-media@vger.kernel.org
S: Supported
F: drivers/thermal/samsung/
SAMSUNG USB2 PHY DRIVER
-M: Kamil Debski <kamil@wypas.org>
M: Sylwester Nawrocki <s.nawrocki@samsung.com>
L: linux-kernel@vger.kernel.org
S: Supported
F: include/linux/slimbus.h
SFC NETWORK DRIVER
-M: Solarflare linux maintainers <linux-net-drivers@solarflare.com>
-M: Edward Cree <ecree@solarflare.com>
-M: Martin Habets <mhabets@solarflare.com>
+M: Edward Cree <ecree.xilinx@gmail.com>
+M: Martin Habets <habetsm.xilinx@gmail.com>
L: netdev@vger.kernel.org
S: Supported
F: drivers/net/ethernet/sfc/
S: Supported
F: drivers/usb/class/usblp.c
+USB RAW GADGET DRIVER
+R: Andrey Konovalov <andreyknvl@gmail.com>
+L: linux-usb@vger.kernel.org
+S: Maintained
+F: Documentation/usb/raw-gadget.rst
+F: drivers/usb/gadget/legacy/raw_gadget.c
+F: include/uapi/linux/usb/raw_gadget.h
+
USB QMI WWAN NETWORK DRIVER
M: Bjørn Mork <bjorn@mork.no>
L: netdev@vger.kernel.org
S: Maintained
N: axp[128]
-X.25 NETWORK LAYER
-M: Andrew Hendry <andrew.hendry@gmail.com>
+X.25 STACK
+M: Martin Schiller <ms@dev.tdt.de>
L: linux-x25@vger.kernel.org
-S: Odd Fixes
+S: Maintained
+F: Documentation/networking/lapb-module.rst
F: Documentation/networking/x25*
+F: drivers/net/wan/hdlc_x25.c
+F: drivers/net/wan/lapbether.c
+F: include/*/lapb.h
F: include/net/x25*
+F: include/uapi/linux/x25.h
+F: net/lapb/
F: net/x25/
X86 ARCHITECTURE (32-BIT AND 64-BIT)
L: bpf@vger.kernel.org
S: Supported
F: include/net/xdp.h
+F: include/net/xdp_priv.h
F: include/trace/events/xdp.h
F: kernel/bpf/cpumap.c
F: kernel/bpf/devmap.c
F: net/core/xdp.c
-N: xdp
-K: xdp
+F: samples/bpf/xdp*
+F: tools/testing/selftests/bpf/*xdp*
+F: tools/testing/selftests/bpf/*/*xdp*
+F: drivers/net/ethernet/*/*/*/*/*xdp*
+F: drivers/net/ethernet/*/*/*xdp*
+K: (?:\b|_)xdp(?:\b|_)
XDP SOCKETS (AF_XDP)
M: Björn Töpel <bjorn.topel@intel.com>
L: netdev@vger.kernel.org
L: bpf@vger.kernel.org
S: Maintained
+F: Documentation/networking/af_xdp.rst
F: include/net/xdp_sock*
F: include/net/xsk_buff_pool.h
F: include/uapi/linux/if_xdp.h
+F: include/uapi/linux/xdp_diag.h
+F: include/net/netns/xdp.h
F: net/xdp/
F: samples/bpf/xdpsock*
F: tools/lib/bpf/xsk*
VERSION = 5
PATCHLEVEL = 10
SUBLEVEL = 0
-EXTRAVERSION = -rc3
+EXTRAVERSION = -rc6
NAME = Kleptomaniac Octopus
# *DOCUMENTATION*
OBJCOPY = llvm-objcopy
OBJDUMP = llvm-objdump
READELF = llvm-readelf
-OBJSIZE = llvm-size
STRIP = llvm-strip
else
CC = $(CROSS_COMPILE)gcc
OBJCOPY = $(CROSS_COMPILE)objcopy
OBJDUMP = $(CROSS_COMPILE)objdump
READELF = $(CROSS_COMPILE)readelf
-OBJSIZE = $(CROSS_COMPILE)size
STRIP = $(CROSS_COMPILE)strip
endif
PAHOLE = pahole
CLANG_FLAGS :=
export ARCH SRCARCH CONFIG_SHELL BASH HOSTCC KBUILD_HOSTCFLAGS CROSS_COMPILE LD CC
-export CPP AR NM STRIP OBJCOPY OBJDUMP OBJSIZE READELF PAHOLE RESOLVE_BTFIDS LEX YACC AWK INSTALLKERNEL
+export CPP AR NM STRIP OBJCOPY OBJDUMP READELF PAHOLE RESOLVE_BTFIDS LEX YACC AWK INSTALLKERNEL
export PERL PYTHON PYTHON3 CHECK CHECKFLAGS MAKE UTS_MACHINE HOSTCXX
export KGZIP KBZIP2 KLZOP LZMA LZ4 XZ ZSTD
export KBUILD_HOSTCXXFLAGS KBUILD_HOSTLDFLAGS KBUILD_HOSTLDLIBS LDFLAGS_MODULE
void arch_cpu_idle(void)
{
wtint(0);
- local_irq_enable();
+ raw_local_irq_enable();
}
void arch_cpu_idle_dead(void)
x <<= 2;
r -= 2;
}
- if (!(x & 0x80000000u)) {
- x <<= 1;
+ if (!(x & 0x80000000u))
r -= 1;
- }
return r;
}
#ifdef CONFIG_ARC_HAS_PAE40
#define PTE_BITS_NON_RWX_IN_PD1 (0xff00000000 | PAGE_MASK | _PAGE_CACHEABLE)
+#define MAX_POSSIBLE_PHYSMEM_BITS 40
#else
#define PTE_BITS_NON_RWX_IN_PD1 (PAGE_MASK | _PAGE_CACHEABLE)
+#define MAX_POSSIBLE_PHYSMEM_BITS 32
#endif
/**************************************************************************
#ifdef CONFIG_ARC_DW2_UNWIND
-static void seed_unwind_frame_info(struct task_struct *tsk,
- struct pt_regs *regs,
- struct unwind_frame_info *frame_info)
+static int
+seed_unwind_frame_info(struct task_struct *tsk, struct pt_regs *regs,
+ struct unwind_frame_info *frame_info)
{
- /*
- * synchronous unwinding (e.g. dump_stack)
- * - uses current values of SP and friends
- */
- if (tsk == NULL && regs == NULL) {
+ if (regs) {
+ /*
+ * Asynchronous unwinding of intr/exception
+ * - Just uses the pt_regs passed
+ */
+ frame_info->task = tsk;
+
+ frame_info->regs.r27 = regs->fp;
+ frame_info->regs.r28 = regs->sp;
+ frame_info->regs.r31 = regs->blink;
+ frame_info->regs.r63 = regs->ret;
+ frame_info->call_frame = 0;
+ } else if (tsk == NULL || tsk == current) {
+ /*
+ * synchronous unwinding (e.g. dump_stack)
+ * - uses current values of SP and friends
+ */
unsigned long fp, sp, blink, ret;
frame_info->task = current;
frame_info->regs.r31 = blink;
frame_info->regs.r63 = ret;
frame_info->call_frame = 0;
- } else if (regs == NULL) {
+ } else {
/*
- * Asynchronous unwinding of sleeping task
- * - Gets SP etc from task's pt_regs (saved bottom of kernel
- * mode stack of task)
+ * Asynchronous unwinding of a likely sleeping task
+ * - first ensure it is actually sleeping
+ * - if so, it will be in __switch_to, kernel mode SP of task
+ * is safe-kept and BLINK at a well known location in there
*/
+ if (tsk->state == TASK_RUNNING)
+ return -1;
+
frame_info->task = tsk;
frame_info->regs.r27 = TSK_K_FP(tsk);
frame_info->regs.r28 += 60;
frame_info->call_frame = 0;
- } else {
- /*
- * Asynchronous unwinding of intr/exception
- * - Just uses the pt_regs passed
- */
- frame_info->task = tsk;
-
- frame_info->regs.r27 = regs->fp;
- frame_info->regs.r28 = regs->sp;
- frame_info->regs.r31 = regs->blink;
- frame_info->regs.r63 = regs->ret;
- frame_info->call_frame = 0;
}
+ return 0;
}
#endif
unsigned int address;
struct unwind_frame_info frame_info;
- seed_unwind_frame_info(tsk, regs, &frame_info);
+ if (seed_unwind_frame_info(tsk, regs, &frame_info))
+ return 0;
while (1) {
address = UNW_PC(&frame_info);
* -Changes related to MMU v2 (Rel 4.8)
*
* Vineetg: Aug 29th 2008
- * -In TLB Flush operations (Metal Fix MMU) there is a explict command to
+ * -In TLB Flush operations (Metal Fix MMU) there is a explicit command to
* flush Micro-TLBS. If TLB Index Reg is invalid prior to TLBIVUTLB cmd,
* it fails. Thus need to load it with ANY valid value before invoking
* TLBIVUTLB cmd
*
* Vineetg: Aug 21th 2008:
* -Reduced the duration of IRQ lockouts in TLB Flush routines
- * -Multiple copies of TLB erase code seperated into a "single" function
+ * -Multiple copies of TLB erase code separated into a "single" function
* -In TLB Flush routines, interrupt disabling moved UP to retrieve ASID
* in interrupt-safe region.
*
*
* Although J-TLB is 2 way set assoc, ARC700 caches J-TLB into uTLBS which has
* much higher associativity. u-D-TLB is 8 ways, u-I-TLB is 4 ways.
- * Given this, the thrasing problem should never happen because once the 3
+ * Given this, the thrashing problem should never happen because once the 3
* J-TLB entries are created (even though 3rd will knock out one of the prev
* two), the u-D-TLB and u-I-TLB will have what is required to accomplish memcpy
*
* There was however an obscure hardware bug, where uTLB flush would
* fail when a prior probe for J-TLB (both totally unrelated) would
* return lkup err - because the entry didn't exist in MMU.
- * The Workround was to set Index reg with some valid value, prior to
+ * The Workaround was to set Index reg with some valid value, prior to
* flush. This was fixed in MMU v3
*/
unsigned int idx;
}
/*
- * Flush the entrie MM for userland. The fastest way is to move to Next ASID
+ * Flush the entire MM for userland. The fastest way is to move to Next ASID
*/
noinline void local_flush_tlb_mm(struct mm_struct *mm)
{
* Difference between this and Kernel Range Flush is
* -Here the fastest way (if range is too large) is to move to next ASID
* without doing any explicit Shootdown
- * -In case of kernel Flush, entry has to be shot down explictly
+ * -In case of kernel Flush, entry has to be shot down explicitly
*/
void local_flush_tlb_range(struct vm_area_struct *vma, unsigned long start,
unsigned long end)
* Super Page size is configurable in hardware (4K to 16M), but fixed once
* RTL builds.
*
- * The exact THP size a Linx configuration will support is a function of:
+ * The exact THP size a Linux configuration will support is a function of:
* - MMU page size (typical 8K, RTL fixed)
* - software page walker address split between PGD:PTE:PFN (typical
* 11:8:13, but can be changed with 1 line)
#endif
-/* Read the Cache Build Confuration Registers, Decode them and save into
+/* Read the Cache Build Configuration Registers, Decode them and save into
* the cpuinfo structure for later use.
* No Validation is done here, simply read/convert the BCRs
*/
pr_info("%s", arc_mmu_mumbojumbo(0, str, sizeof(str)));
/*
- * Can't be done in processor.h due to header include depenedencies
+ * Can't be done in processor.h due to header include dependencies
*/
BUILD_BUG_ON(!IS_ALIGNED((CONFIG_ARC_KVADDR_SIZE << 20), PMD_SIZE));
/*
* stack top size sanity check,
- * Can't be done in processor.h due to header include depenedencies
+ * Can't be done in processor.h due to header include dependencies
*/
BUILD_BUG_ON(!IS_ALIGNED(STACK_TOP, PMD_SIZE));
* the duplicate one.
* -Knob to be verbose abt it.(TODO: hook them up to debugfs)
*/
-volatile int dup_pd_silent; /* Be slient abt it or complain (default) */
+volatile int dup_pd_silent; /* Be silent abt it or complain (default) */
void do_tlb_overlap_fault(unsigned long cause, unsigned long address,
struct pt_regs *regs)
/***********************************************************************
* Diagnostic Routines
- * -Called from Low Level TLB Hanlders if things don;t look good
+ * -Called from Low Level TLB Handlers if things don;t look good
**********************************************************************/
#ifdef CONFIG_ARC_DBG_TLB_PARANOIA
@ issued from HYP mode take us to the correct handler code. We
@ will disable the MMU before jumping to the kernel proper.
@
+ ARM( bic r1, r1, #(1 << 30) ) @ clear HSCTLR.TE
+ THUMB( orr r1, r1, #(1 << 30) ) @ set HSCTLR.TE
+ mcr p15, 4, r1, c1, c0, 0
adr r0, __hyp_reentry_vectors
mcr p15, 4, r0, c12, c0, 0 @ set HYP vector base (HVBAR)
isb
ranges = <0x0 0x100000 0x8000>;
mac_sw: switch@0 {
- compatible = "ti,am4372-cpsw","ti,cpsw-switch";
+ compatible = "ti,am4372-cpsw-switch", "ti,cpsw-switch";
reg = <0x0 0x4000>;
ranges = <0 0 0x4000>;
clocks = <&cpsw_125mhz_gclk>, <&dpll_clksel_mac_clk>;
interrupts = <GIC_SPI 67 IRQ_TYPE_LEVEL_HIGH>,
<GIC_SPI 68 IRQ_TYPE_LEVEL_HIGH>;
interrupt-names = "int0", "int1";
- clocks = <&mcan_clk>, <&l3_iclk_div>;
- clock-names = "cclk", "hclk";
+ clocks = <&l3_iclk_div>, <&mcan_clk>;
+ clock-names = "hclk", "cclk";
bosch,mram-cfg = <0x0 0 0 32 0 0 1 1>;
};
};
};
&clock {
- clocks = <&clock CLK_XUSBXTI>;
assigned-clocks = <&clock CLK_FOUT_EPLL>;
assigned-clock-rates = <45158401>;
};
MX50_PAD_CSPI_MISO__CSPI_MISO 0x00
MX50_PAD_CSPI_MOSI__CSPI_MOSI 0x00
MX50_PAD_CSPI_SS0__GPIO4_11 0xc4
- MX50_PAD_ECSPI1_MOSI__CSPI_SS1 0xf4
+ MX50_PAD_ECSPI1_MOSI__GPIO4_13 0x84
>;
};
#size-cells = <0>;
/* Microchip KSZ9031RNX PHY */
- rgmii_phy: ethernet-phy@4 {
- reg = <4>;
+ rgmii_phy: ethernet-phy@0 {
+ reg = <0>;
interrupts-extended = <&gpio1 28 IRQ_TYPE_LEVEL_LOW>;
reset-gpios = <&gpio1 25 GPIO_ACTIVE_LOW>;
reset-assert-us = <10000>;
&fec {
pinctrl-names = "default";
pinctrl-0 = <&pinctrl_enet>;
- phy-mode = "rgmii";
+ phy-mode = "rgmii-id";
status = "okay";
};
linux,code = <KEY_A>;
gpios = <&gpiof 3 GPIO_ACTIVE_LOW>;
};
+
+ /*
+ * The EXTi IRQ line 0 is shared with PMIC,
+ * so mark this as polled GPIO key.
+ */
+ button-2 {
+ label = "TA3-GPIO-C";
+ linux,code = <KEY_C>;
+ gpios = <&gpiog 0 GPIO_ACTIVE_LOW>;
+ };
};
gpio-keys {
wakeup-source;
};
- button-2 {
- label = "TA3-GPIO-C";
- linux,code = <KEY_C>;
- gpios = <&gpioi 11 GPIO_ACTIVE_LOW>;
- wakeup-source;
- };
-
button-3 {
label = "TA4-GPIO-D";
linux,code = <KEY_D>;
led-0 {
label = "green:led5";
- gpios = <&gpiog 2 GPIO_ACTIVE_HIGH>;
+ gpios = <&gpioc 6 GPIO_ACTIVE_HIGH>;
default-state = "off";
};
gpio = <&gpiog 3 GPIO_ACTIVE_LOW>;
regulator-always-on;
regulator-boot-on;
+ vin-supply = <&vdd>;
};
};
vdda: ldo1 {
regulator-name = "vdda";
+ regulator-always-on;
regulator-min-microvolt = <2900000>;
regulator-max-microvolt = <2900000>;
interrupts = <IT_CURLIM_LDO1 0>;
};
};
+&dts {
+ status = "okay";
+};
+
&i2c4 {
pinctrl-names = "default";
pinctrl-0 = <&i2c4_pins_a>;
pinctrl-names = "default";
pinctrl-0 = <&gmac_rgmii_pins>;
phy-handle = <&phy1>;
- phy-mode = "rgmii";
+ phy-mode = "rgmii-id";
status = "okay";
};
pinctrl-names = "default";
pinctrl-0 = <&gmac_rgmii_pins>;
phy-handle = <&phy1>;
- phy-mode = "rgmii";
+ phy-mode = "rgmii-id";
phy-supply = <®_gmac_3v3>;
status = "okay";
};
pinctrl-names = "default";
pinctrl-0 = <&gmac_rgmii_pins>;
phy-handle = <&phy1>;
- phy-mode = "rgmii";
+ phy-mode = "rgmii-id";
status = "okay";
};
pinctrl-0 = <&emac_rgmii_pins>;
phy-supply = <®_sw>;
phy-handle = <&rgmii_phy>;
- phy-mode = "rgmii";
+ phy-mode = "rgmii-id";
allwinner,rx-delay-ps = <700>;
allwinner,tx-delay-ps = <700>;
status = "okay";
pinctrl-0 = <&emac_rgmii_pins>;
phy-supply = <®_dldo4>;
phy-handle = <&rgmii_phy>;
- phy-mode = "rgmii";
+ phy-mode = "rgmii-id";
status = "okay";
};
};
};
-&emac {
- /* LEDs changed to active high on the plus */
- /delete-property/ allwinner,leds-active-low;
-};
-
&mmc1 {
vmmc-supply = <®_vcc3v3>;
bus-width = <4>;
pinctrl-0 = <&emac_rgmii_pins>;
phy-supply = <®_gmac_3v3>;
phy-handle = <&ext_rgmii_phy>;
- phy-mode = "rgmii";
+ phy-mode = "rgmii-id";
status = "okay";
};
pinctrl-names = "default";
pinctrl-0 = <&gmac_rgmii_pins>;
phy-handle = <&phy1>;
- phy-mode = "rgmii";
+ phy-mode = "rgmii-id";
phy-supply = <®_dc1sw>;
status = "okay";
};
pinctrl-names = "default";
pinctrl-0 = <&gmac_rgmii_pins>;
phy-handle = <&phy1>;
- phy-mode = "rgmii";
+ phy-mode = "rgmii-id";
phy-supply = <®_cldo1>;
status = "okay";
};
pinctrl-names = "default";
pinctrl-0 = <&gmac_rgmii_pins>;
phy-handle = <&phy1>;
- phy-mode = "rgmii";
+ phy-mode = "rgmii-id";
phy-supply = <®_cldo1>;
status = "okay";
};
pinctrl-0 = <&emac_rgmii_pins>;
phy-supply = <®_gmac_3v3>;
phy-handle = <&ext_rgmii_phy>;
- phy-mode = "rgmii";
+ phy-mode = "rgmii-id";
status = "okay";
};
};
};
+&mdio1 {
+ clock-frequency = <5000000>;
+};
&iomuxc {
pinctrl_gpio_e6185_eeprom_sel: pinctrl-gpio-e6185-eeprom-spi0 {
CONFIG_HDLC_FR=m
CONFIG_HDLC_PPP=m
CONFIG_HDLC_X25=m
-CONFIG_DLCI=m
CONFIG_WAN_ROUTER_DRIVERS=m
CONFIG_ATM_TCP=m
# CONFIG_INPUT_KEYBOARD is not set
unsigned long val, void *data);
/* optinsn template addresses */
-extern __visible kprobe_opcode_t optprobe_template_entry;
-extern __visible kprobe_opcode_t optprobe_template_val;
-extern __visible kprobe_opcode_t optprobe_template_call;
-extern __visible kprobe_opcode_t optprobe_template_end;
-extern __visible kprobe_opcode_t optprobe_template_sub_sp;
-extern __visible kprobe_opcode_t optprobe_template_add_sp;
-extern __visible kprobe_opcode_t optprobe_template_restore_begin;
-extern __visible kprobe_opcode_t optprobe_template_restore_orig_insn;
-extern __visible kprobe_opcode_t optprobe_template_restore_end;
+extern __visible kprobe_opcode_t optprobe_template_entry[];
+extern __visible kprobe_opcode_t optprobe_template_val[];
+extern __visible kprobe_opcode_t optprobe_template_call[];
+extern __visible kprobe_opcode_t optprobe_template_end[];
+extern __visible kprobe_opcode_t optprobe_template_sub_sp[];
+extern __visible kprobe_opcode_t optprobe_template_add_sp[];
+extern __visible kprobe_opcode_t optprobe_template_restore_begin[];
+extern __visible kprobe_opcode_t optprobe_template_restore_orig_insn[];
+extern __visible kprobe_opcode_t optprobe_template_restore_end[];
#define MAX_OPTIMIZED_LENGTH 4
#define MAX_OPTINSN_SIZE \
- ((unsigned long)&optprobe_template_end - \
- (unsigned long)&optprobe_template_entry)
+ ((unsigned long)optprobe_template_end - \
+ (unsigned long)optprobe_template_entry)
#define RELATIVEJUMP_SIZE 4
struct arch_optimized_insn {
#define PTE_HWTABLE_OFF (PTE_HWTABLE_PTRS * sizeof(pte_t))
#define PTE_HWTABLE_SIZE (PTRS_PER_PTE * sizeof(u32))
+#define MAX_POSSIBLE_PHYSMEM_BITS 32
+
/*
* PMD_SHIFT determines the size of the area a second-level page table can map
* PGDIR_SHIFT determines what a third-level page table entry can map
#define PTE_HWTABLE_OFF (0)
#define PTE_HWTABLE_SIZE (PTRS_PER_PTE * sizeof(u64))
+#define MAX_POSSIBLE_PHYSMEM_BITS 40
+
/*
* PGDIR_SHIFT determines the size a top-level page table entry can map.
*/
}
void perf_get_regs_user(struct perf_regs *regs_user,
- struct pt_regs *regs,
- struct pt_regs *regs_user_copy)
+ struct pt_regs *regs)
{
regs_user->regs = task_pt_regs(current);
regs_user->abi = perf_reg_abi(current);
arm_pm_idle();
else
cpu_do_idle();
- local_irq_enable();
+ raw_local_irq_enable();
}
void arch_cpu_idle_prepare(void)
depends on ARCH_MULTI_V6
select ARCH_OMAP2PLUS
select CPU_V6
- select PM_GENERIC_DOMAINS if PM
select SOC_HAS_OMAP2_SDRC
config ARCH_OMAP3
select OMAP_DM_TIMER
select OMAP_GPMC
select PINCTRL
+ select PM_GENERIC_DOMAINS if PM
+ select PM_GENERIC_DOMAINS_OF if PM
select RESET_CONTROLLER
select SOC_BUS
select TI_SYSC
if (mpuss_can_lose_context) {
error = cpu_cluster_pm_enter();
if (error) {
- omap_set_pwrdm_state(mpu_pd, PWRDM_POWER_ON);
- goto cpu_cluster_pm_out;
+ index = 0;
+ cx = state_ptr + index;
+ pwrdm_set_logic_retst(mpu_pd, cx->mpu_logic_state);
+ omap_set_pwrdm_state(mpu_pd, cx->mpu_state);
+ mpuss_can_lose_context = 0;
}
}
}
omap4_enter_lowpower(dev->cpu, cx->cpu_state);
cpu_done[dev->cpu] = true;
-cpu_cluster_pm_out:
/* Wakeup CPU1 only if it is not offlined */
if (dev->cpu == 0 && cpumask_test_cpu(1, cpu_online_mask)) {
"optprobe_template_end:\n");
#define TMPL_VAL_IDX \
- ((unsigned long *)&optprobe_template_val - (unsigned long *)&optprobe_template_entry)
+ ((unsigned long *)optprobe_template_val - (unsigned long *)optprobe_template_entry)
#define TMPL_CALL_IDX \
- ((unsigned long *)&optprobe_template_call - (unsigned long *)&optprobe_template_entry)
+ ((unsigned long *)optprobe_template_call - (unsigned long *)optprobe_template_entry)
#define TMPL_END_IDX \
- ((unsigned long *)&optprobe_template_end - (unsigned long *)&optprobe_template_entry)
+ ((unsigned long *)optprobe_template_end - (unsigned long *)optprobe_template_entry)
#define TMPL_ADD_SP \
- ((unsigned long *)&optprobe_template_add_sp - (unsigned long *)&optprobe_template_entry)
+ ((unsigned long *)optprobe_template_add_sp - (unsigned long *)optprobe_template_entry)
#define TMPL_SUB_SP \
- ((unsigned long *)&optprobe_template_sub_sp - (unsigned long *)&optprobe_template_entry)
+ ((unsigned long *)optprobe_template_sub_sp - (unsigned long *)optprobe_template_entry)
#define TMPL_RESTORE_BEGIN \
- ((unsigned long *)&optprobe_template_restore_begin - (unsigned long *)&optprobe_template_entry)
+ ((unsigned long *)optprobe_template_restore_begin - (unsigned long *)optprobe_template_entry)
#define TMPL_RESTORE_ORIGN_INSN \
- ((unsigned long *)&optprobe_template_restore_orig_insn - (unsigned long *)&optprobe_template_entry)
+ ((unsigned long *)optprobe_template_restore_orig_insn - (unsigned long *)optprobe_template_entry)
#define TMPL_RESTORE_END \
- ((unsigned long *)&optprobe_template_restore_end - (unsigned long *)&optprobe_template_entry)
+ ((unsigned long *)optprobe_template_restore_end - (unsigned long *)optprobe_template_entry)
/*
* ARM can always optimize an instruction when using ARM ISA, except
}
/* Copy arch-dep-instance from template. */
- memcpy(code, (unsigned long *)&optprobe_template_entry,
+ memcpy(code, (unsigned long *)optprobe_template_entry,
TMPL_END_IDX * sizeof(kprobe_opcode_t));
/* Adjust buffer according to instruction. */
&emac {
pinctrl-names = "default";
pinctrl-0 = <&rgmii_pins>;
- phy-mode = "rgmii";
+ phy-mode = "rgmii-id";
phy-handle = <&ext_rgmii_phy>;
phy-supply = <®_dc1sw>;
status = "okay";
&emac {
pinctrl-names = "default";
pinctrl-0 = <&rgmii_pins>;
- phy-mode = "rgmii";
+ phy-mode = "rgmii-id";
phy-handle = <&ext_rgmii_phy>;
phy-supply = <®_gmac_3v3>;
status = "okay";
&emac {
pinctrl-names = "default";
pinctrl-0 = <&rgmii_pins>;
- phy-mode = "rgmii";
+ phy-mode = "rgmii-txid";
phy-handle = <&ext_rgmii_phy>;
status = "okay";
};
status = "okay";
port {
- #address-cells = <1>;
- #size-cells = <0>;
-
csi_ep: endpoint {
remote-endpoint = <&ov5640_ep>;
bus-width = <8>;
pinctrl-0 = <&emac_rgmii_pins>;
phy-supply = <®_gmac_3v3>;
phy-handle = <&ext_rgmii_phy>;
- phy-mode = "rgmii";
+ phy-mode = "rgmii-id";
/delete-property/ allwinner,leds-active-low;
status = "okay";
};
pinctrl-0 = <&emac_rgmii_pins>;
phy-supply = <®_gmac_3v3>;
phy-handle = <&ext_rgmii_phy>;
- phy-mode = "rgmii";
+ phy-mode = "rgmii-id";
status = "okay";
};
pinctrl-0 = <&emac_rgmii_pins>;
phy-supply = <®_gmac_3v3>;
phy-handle = <&ext_rgmii_phy>;
- phy-mode = "rgmii";
+ phy-mode = "rgmii-id";
status = "okay";
};
&emac {
pinctrl-names = "default";
pinctrl-0 = <&ext_rgmii_pins>;
- phy-mode = "rgmii";
+ phy-mode = "rgmii-id";
phy-handle = <&ext_rgmii_phy>;
phy-supply = <®_aldo2>;
status = "okay";
&emac {
pinctrl-names = "default";
pinctrl-0 = <&ext_rgmii_pins>;
- phy-mode = "rgmii";
+ phy-mode = "rgmii-id";
phy-handle = <&ext_rgmii_phy>;
phy-supply = <®_gmac_3v3>;
allwinner,rx-delay-ps = <200>;
flash@0 {
#address-cells = <1>;
#size-cells = <1>;
- compatible = "n25q00a";
+ compatible = "micron,mt25qu02g", "jedec,spi-nor";
reg = <0>;
spi-max-frequency = <100000000>;
flash@0 {
#address-cells = <1>;
#size-cells = <1>;
- compatible = "n25q00a";
+ compatible = "micron,mt25qu02g", "jedec,spi-nor";
reg = <0>;
spi-max-frequency = <100000000>;
usb {
compatible = "simple-bus";
dma-ranges;
- #address-cells = <1>;
- #size-cells = <1>;
- ranges = <0x0 0x0 0x68500000 0x00400000>;
+ #address-cells = <2>;
+ #size-cells = <2>;
+ ranges = <0x0 0x0 0x0 0x68500000 0x0 0x00400000>;
usbphy0: usb-phy@0 {
compatible = "brcm,sr-usb-combo-phy";
- reg = <0x00000000 0x100>;
+ reg = <0x0 0x00000000 0x0 0x100>;
#phy-cells = <1>;
status = "disabled";
};
xhci0: usb@1000 {
compatible = "generic-xhci";
- reg = <0x00001000 0x1000>;
+ reg = <0x0 0x00001000 0x0 0x1000>;
interrupts = <GIC_SPI 256 IRQ_TYPE_LEVEL_HIGH>;
phys = <&usbphy0 1>, <&usbphy0 0>;
phy-names = "phy0", "phy1";
bdc0: usb@2000 {
compatible = "brcm,bdc-v0.16";
- reg = <0x00002000 0x1000>;
+ reg = <0x0 0x00002000 0x0 0x1000>;
interrupts = <GIC_SPI 259 IRQ_TYPE_LEVEL_HIGH>;
phys = <&usbphy0 0>, <&usbphy0 1>;
phy-names = "phy0", "phy1";
usbphy1: usb-phy@10000 {
compatible = "brcm,sr-usb-combo-phy";
- reg = <0x00010000 0x100>;
+ reg = <0x0 0x00010000 0x0 0x100>;
#phy-cells = <1>;
status = "disabled";
};
usbphy2: usb-phy@20000 {
compatible = "brcm,sr-usb-hs-phy";
- reg = <0x00020000 0x100>;
+ reg = <0x0 0x00020000 0x0 0x100>;
#phy-cells = <0>;
status = "disabled";
};
xhci1: usb@11000 {
compatible = "generic-xhci";
- reg = <0x00011000 0x1000>;
+ reg = <0x0 0x00011000 0x0 0x1000>;
interrupts = <GIC_SPI 263 IRQ_TYPE_LEVEL_HIGH>;
phys = <&usbphy1 1>, <&usbphy2>, <&usbphy1 0>;
phy-names = "phy0", "phy1", "phy2";
bdc1: usb@21000 {
compatible = "brcm,bdc-v0.16";
- reg = <0x00021000 0x1000>;
+ reg = <0x0 0x00021000 0x0 0x1000>;
interrupts = <GIC_SPI 266 IRQ_TYPE_LEVEL_HIGH>;
phys = <&usbphy2>;
phy-names = "phy0";
compatible = "fsl,ls1028a-rcpm", "fsl,qoriq-rcpm-2.1+";
reg = <0x0 0x1e34040 0x0 0x1c>;
#fsl,rcpm-wakeup-cells = <7>;
+ little-endian;
};
ftm_alarm0: timer@2800000 {
compatible = "fsl,ls1088a-rcpm", "fsl,qoriq-rcpm-2.1+";
reg = <0x0 0x1e34040 0x0 0x18>;
#fsl,rcpm-wakeup-cells = <6>;
+ little-endian;
};
ftm_alarm0: timer@2800000 {
compatible = "fsl,ls208xa-rcpm", "fsl,qoriq-rcpm-2.1+";
reg = <0x0 0x1e34040 0x0 0x18>;
#fsl,rcpm-wakeup-cells = <6>;
+ little-endian;
};
ftm_alarm0: timer@2800000 {
pmic@4b {
compatible = "rohm,bd71847";
reg = <0x4b>;
+ pinctrl-names = "default";
pinctrl-0 = <&pinctrl_pmic>;
interrupt-parent = <&gpio1>;
interrupts = <3 IRQ_TYPE_LEVEL_LOW>;
host-wakeup-gpios = <&gpio2 8 GPIO_ACTIVE_HIGH>;
device-wakeup-gpios = <&gpio2 7 GPIO_ACTIVE_HIGH>;
clocks = <&osc_32k>;
+ max-speed = <4000000>;
clock-names = "extclk";
};
};
pmic@4b {
compatible = "rohm,bd71847";
reg = <0x4b>;
+ pinctrl-names = "default";
pinctrl-0 = <&pinctrl_pmic>;
interrupt-parent = <&gpio1>;
interrupts = <3 IRQ_TYPE_LEVEL_LOW>;
pmic@4b {
compatible = "rohm,bd71847";
reg = <0x4b>;
+ pinctrl-names = "default";
pinctrl-0 = <&pinctrl_pmic>;
interrupt-parent = <&gpio2>;
- /*
- * The interrupt is not correct. It should be level low,
- * however with internal pull up this causes IRQ storm.
- */
- interrupts = <8 IRQ_TYPE_EDGE_RISING>;
+ interrupts = <8 IRQ_TYPE_LEVEL_LOW>;
rohm,reset-snvs-powered;
#clock-cells = <0>;
pinctrl_pmic: pmicirqgrp {
fsl,pins = <
- MX8MM_IOMUXC_SD1_DATA6_GPIO2_IO8 0x41
+ MX8MM_IOMUXC_SD1_DATA6_GPIO2_IO8 0x141
>;
};
opp-1600000000 {
opp-hz = /bits/ 64 <1600000000>;
- opp-microvolt = <900000>;
+ opp-microvolt = <950000>;
opp-supported-hw = <0xc>, <0x7>;
clock-latency-ns = <150000>;
opp-suspend;
pmic@4b {
compatible = "rohm,bd71847";
reg = <0x4b>;
+ pinctrl-names = "default";
pinctrl-0 = <&pinctrl_pmic>;
interrupt-parent = <&gpio1>;
interrupts = <3 IRQ_TYPE_LEVEL_LOW>;
pmic: pmic@25 {
compatible = "nxp,pca9450b";
reg = <0x25>;
+ pinctrl-names = "default";
pinctrl-0 = <&pinctrl_pmic>;
interrupt-parent = <&gpio1>;
interrupts = <3 IRQ_TYPE_LEVEL_LOW>;
pmic@4b {
compatible = "rohm,bd71847";
reg = <0x4b>;
+ pinctrl-names = "default";
pinctrl-0 = <&pinctrl_pmic>;
interrupt-parent = <&gpio2>;
- /*
- * The interrupt is not correct. It should be level low,
- * however with internal pull up this causes IRQ storm.
- */
- interrupts = <8 IRQ_TYPE_EDGE_RISING>;
+ interrupts = <8 IRQ_TYPE_LEVEL_LOW>;
rohm,reset-snvs-powered;
regulators {
pinctrl_pmic: pmicirqgrp {
fsl,pins = <
- MX8MN_IOMUXC_SD1_DATA6_GPIO2_IO8 0x101
+ MX8MN_IOMUXC_SD1_DATA6_GPIO2_IO8 0x141
>;
};
#index-cells = <1>;
reg = <0x32e40200 0x200>;
};
-
- usbotg2: usb@32e50000 {
- compatible = "fsl,imx8mn-usb", "fsl,imx7d-usb";
- reg = <0x32e50000 0x200>;
- interrupts = <GIC_SPI 41 IRQ_TYPE_LEVEL_HIGH>;
- clocks = <&clk IMX8MN_CLK_USB1_CTRL_ROOT>;
- clock-names = "usb1_ctrl_root_clk";
- assigned-clocks = <&clk IMX8MN_CLK_USB_BUS>,
- <&clk IMX8MN_CLK_USB_CORE_REF>;
- assigned-clock-parents = <&clk IMX8MN_SYS_PLL2_500M>,
- <&clk IMX8MN_SYS_PLL1_100M>;
- fsl,usbphy = <&usbphynop2>;
- fsl,usbmisc = <&usbmisc2 0>;
- status = "disabled";
- };
-
- usbmisc2: usbmisc@32e50200 {
- compatible = "fsl,imx8mn-usbmisc", "fsl,imx7d-usbmisc";
- #index-cells = <1>;
- reg = <0x32e50200 0x200>;
- };
-
};
dma_apbh: dma-controller@33000000 {
assigned-clock-parents = <&clk IMX8MN_SYS_PLL1_100M>;
clock-names = "main_clk";
};
-
- usbphynop2: usbphynop2 {
- compatible = "usb-nop-xceiv";
- clocks = <&clk IMX8MN_CLK_USB_PHY_REF>;
- assigned-clocks = <&clk IMX8MN_CLK_USB_PHY_REF>;
- assigned-clock-parents = <&clk IMX8MN_SYS_PLL1_100M>;
- clock-names = "main_clk";
- };
};
clock-names = "fmanclk";
fsl,qman-channel-range = <0x800 0x10>;
ptimer-handle = <&ptp_timer0>;
+ dma-coherent;
muram@0 {
compatible = "fsl,fman-muram";
flash@0 {
#address-cells = <1>;
#size-cells = <1>;
- compatible = "mt25qu02g";
+ compatible = "micron,mt25qu02g", "jedec,spi-nor";
reg = <0>;
spi-max-frequency = <100000000>;
model = "NVIDIA Jetson TX2 Developer Kit";
compatible = "nvidia,p2771-0000", "nvidia,tegra186";
- aconnect {
- status = "okay";
-
- dma-controller@2930000 {
- status = "okay";
- };
-
- interrupt-controller@2a40000 {
- status = "okay";
- };
- };
-
i2c@3160000 {
power-monitor@42 {
compatible = "ti,ina3221";
status = "okay";
};
- serial@c280000 {
+ serial@3100000 {
status = "okay";
};
hsp_aon: hsp@c150000 {
compatible = "nvidia,tegra194-hsp", "nvidia,tegra186-hsp";
- reg = <0x0c150000 0xa0000>;
+ reg = <0x0c150000 0x90000>;
interrupts = <GIC_SPI 133 IRQ_TYPE_LEVEL_HIGH>,
<GIC_SPI 134 IRQ_TYPE_LEVEL_HIGH>,
<GIC_SPI 135 IRQ_TYPE_LEVEL_HIGH>,
vin-supply = <&vdd_5v0_sys>;
};
- vdd_usb_vbus_otg: regulator@11 {
- compatible = "regulator-fixed";
- regulator-name = "USB_VBUS_EN0";
- regulator-min-microvolt = <5000000>;
- regulator-max-microvolt = <5000000>;
- gpio = <&gpio TEGRA_GPIO(CC, 4) GPIO_ACTIVE_HIGH>;
- enable-active-high;
- vin-supply = <&vdd_5v0_sys>;
- };
-
vdd_hdmi: regulator@10 {
compatible = "regulator-fixed";
regulator-name = "VDD_HDMI_5V0";
enable-active-high;
vin-supply = <&vdd_3v3_sys>;
};
+
+ vdd_usb_vbus_otg: regulator@14 {
+ compatible = "regulator-fixed";
+ regulator-name = "USB_VBUS_EN0";
+ regulator-min-microvolt = <5000000>;
+ regulator-max-microvolt = <5000000>;
+ gpio = <&gpio TEGRA_GPIO(CC, 4) GPIO_ACTIVE_HIGH>;
+ enable-active-high;
+ vin-supply = <&vdd_5v0_sys>;
+ };
};
compatible = "nvidia,tegra234-vdk", "nvidia,tegra234";
aliases {
- sdhci3 = "/cbb@0/sdhci@3460000";
+ mmc3 = "/bus@0/mmc@3460000";
serial0 = &uarta;
};
stdout-path = "serial0:115200n8";
};
- cbb@0 {
+ bus@0 {
serial@3100000 {
status = "okay";
};
- sdhci@3460000 {
+ mmc@3460000 {
status = "okay";
bus-width = <8>;
non-removable;
};
soc: soc {
- #address-cells = <1>;
- #size-cells = <1>;
- ranges = <0 0 0 0xffffffff>;
+ #address-cells = <2>;
+ #size-cells = <2>;
+ ranges = <0 0 0 0 0x0 0xffffffff>;
dma-ranges;
compatible = "simple-bus";
prng: qrng@e1000 {
compatible = "qcom,prng-ee";
- reg = <0xe3000 0x1000>;
+ reg = <0x0 0xe3000 0x0 0x1000>;
clocks = <&gcc GCC_PRNG_AHB_CLK>;
clock-names = "core";
};
cryptobam: dma@704000 {
compatible = "qcom,bam-v1.7.0";
- reg = <0x00704000 0x20000>;
+ reg = <0x0 0x00704000 0x0 0x20000>;
interrupts = <GIC_SPI 207 IRQ_TYPE_LEVEL_HIGH>;
clocks = <&gcc GCC_CRYPTO_AHB_CLK>;
clock-names = "bam_clk";
crypto: crypto@73a000 {
compatible = "qcom,crypto-v5.1";
- reg = <0x0073a000 0x6000>;
+ reg = <0x0 0x0073a000 0x0 0x6000>;
clocks = <&gcc GCC_CRYPTO_AHB_CLK>,
<&gcc GCC_CRYPTO_AXI_CLK>,
<&gcc GCC_CRYPTO_CLK>;
tlmm: pinctrl@1000000 {
compatible = "qcom,ipq6018-pinctrl";
- reg = <0x01000000 0x300000>;
+ reg = <0x0 0x01000000 0x0 0x300000>;
interrupts = <GIC_SPI 208 IRQ_TYPE_LEVEL_HIGH>;
gpio-controller;
#gpio-cells = <2>;
gcc: gcc@1800000 {
compatible = "qcom,gcc-ipq6018";
- reg = <0x01800000 0x80000>;
+ reg = <0x0 0x01800000 0x0 0x80000>;
clocks = <&xo>, <&sleep_clk>;
clock-names = "xo", "sleep_clk";
#clock-cells = <1>;
tcsr_mutex_regs: syscon@1905000 {
compatible = "syscon";
- reg = <0x01905000 0x8000>;
+ reg = <0x0 0x01905000 0x0 0x8000>;
};
tcsr_q6: syscon@1945000 {
compatible = "syscon";
- reg = <0x01945000 0xe000>;
+ reg = <0x0 0x01945000 0x0 0xe000>;
};
blsp_dma: dma@7884000 {
compatible = "qcom,bam-v1.7.0";
- reg = <0x07884000 0x2b000>;
+ reg = <0x0 0x07884000 0x0 0x2b000>;
interrupts = <GIC_SPI 238 IRQ_TYPE_LEVEL_HIGH>;
clocks = <&gcc GCC_BLSP1_AHB_CLK>;
clock-names = "bam_clk";
blsp1_uart3: serial@78b1000 {
compatible = "qcom,msm-uartdm-v1.4", "qcom,msm-uartdm";
- reg = <0x078b1000 0x200>;
+ reg = <0x0 0x078b1000 0x0 0x200>;
interrupts = <GIC_SPI 306 IRQ_TYPE_LEVEL_HIGH>;
clocks = <&gcc GCC_BLSP1_UART3_APPS_CLK>,
<&gcc GCC_BLSP1_AHB_CLK>;
compatible = "qcom,spi-qup-v2.2.1";
#address-cells = <1>;
#size-cells = <0>;
- reg = <0x078b5000 0x600>;
+ reg = <0x0 0x078b5000 0x0 0x600>;
interrupts = <GIC_SPI 95 IRQ_TYPE_LEVEL_HIGH>;
spi-max-frequency = <50000000>;
clocks = <&gcc GCC_BLSP1_QUP1_SPI_APPS_CLK>,
compatible = "qcom,spi-qup-v2.2.1";
#address-cells = <1>;
#size-cells = <0>;
- reg = <0x078b6000 0x600>;
+ reg = <0x0 0x078b6000 0x0 0x600>;
interrupts = <GIC_SPI 96 IRQ_TYPE_LEVEL_HIGH>;
spi-max-frequency = <50000000>;
clocks = <&gcc GCC_BLSP1_QUP2_SPI_APPS_CLK>,
compatible = "qcom,i2c-qup-v2.2.1";
#address-cells = <1>;
#size-cells = <0>;
- reg = <0x078b6000 0x600>;
+ reg = <0x0 0x078b6000 0x0 0x600>;
interrupts = <GIC_SPI 96 IRQ_TYPE_LEVEL_HIGH>;
clocks = <&gcc GCC_BLSP1_AHB_CLK>,
<&gcc GCC_BLSP1_QUP2_I2C_APPS_CLK>;
compatible = "qcom,i2c-qup-v2.2.1";
#address-cells = <1>;
#size-cells = <0>;
- reg = <0x078b7000 0x600>;
+ reg = <0x0 0x078b7000 0x0 0x600>;
interrupts = <GIC_SPI 97 IRQ_TYPE_LEVEL_HIGH>;
clocks = <&gcc GCC_BLSP1_AHB_CLK>,
<&gcc GCC_BLSP1_QUP3_I2C_APPS_CLK>;
compatible = "qcom,msm-qgic2";
interrupt-controller;
#interrupt-cells = <0x3>;
- reg = <0x0b000000 0x1000>, /*GICD*/
- <0x0b002000 0x1000>, /*GICC*/
- <0x0b001000 0x1000>, /*GICH*/
- <0x0b004000 0x1000>; /*GICV*/
+ reg = <0x0 0x0b000000 0x0 0x1000>, /*GICD*/
+ <0x0 0x0b002000 0x0 0x1000>, /*GICC*/
+ <0x0 0x0b001000 0x0 0x1000>, /*GICH*/
+ <0x0 0x0b004000 0x0 0x1000>; /*GICV*/
interrupts = <GIC_PPI 9 IRQ_TYPE_LEVEL_HIGH>;
};
watchdog@b017000 {
compatible = "qcom,kpss-wdt";
interrupts = <GIC_SPI 3 IRQ_TYPE_EDGE_RISING>;
- reg = <0x0b017000 0x40>;
+ reg = <0x0 0x0b017000 0x0 0x40>;
clocks = <&sleep_clk>;
timeout-sec = <10>;
};
apcs_glb: mailbox@b111000 {
compatible = "qcom,ipq6018-apcs-apps-global";
- reg = <0x0b111000 0x1000>;
+ reg = <0x0 0x0b111000 0x0 0x1000>;
#clock-cells = <1>;
clocks = <&a53pll>, <&xo>;
clock-names = "pll", "xo";
a53pll: clock@b116000 {
compatible = "qcom,ipq6018-a53pll";
- reg = <0x0b116000 0x40>;
+ reg = <0x0 0x0b116000 0x0 0x40>;
#clock-cells = <0>;
clocks = <&xo>;
clock-names = "xo";
};
timer@b120000 {
- #address-cells = <1>;
- #size-cells = <1>;
+ #address-cells = <2>;
+ #size-cells = <2>;
ranges;
compatible = "arm,armv7-timer-mem";
- reg = <0x0b120000 0x1000>;
+ reg = <0x0 0x0b120000 0x0 0x1000>;
clock-frequency = <19200000>;
frame@b120000 {
frame-number = <0>;
interrupts = <GIC_SPI 8 IRQ_TYPE_LEVEL_HIGH>,
<GIC_SPI 7 IRQ_TYPE_LEVEL_HIGH>;
- reg = <0x0b121000 0x1000>,
- <0x0b122000 0x1000>;
+ reg = <0x0 0x0b121000 0x0 0x1000>,
+ <0x0 0x0b122000 0x0 0x1000>;
};
frame@b123000 {
frame-number = <1>;
interrupts = <GIC_SPI 9 IRQ_TYPE_LEVEL_HIGH>;
- reg = <0xb123000 0x1000>;
+ reg = <0x0 0xb123000 0x0 0x1000>;
status = "disabled";
};
frame@b124000 {
frame-number = <2>;
interrupts = <GIC_SPI 10 IRQ_TYPE_LEVEL_HIGH>;
- reg = <0x0b124000 0x1000>;
+ reg = <0x0 0x0b124000 0x0 0x1000>;
status = "disabled";
};
frame@b125000 {
frame-number = <3>;
interrupts = <GIC_SPI 11 IRQ_TYPE_LEVEL_HIGH>;
- reg = <0x0b125000 0x1000>;
+ reg = <0x0 0x0b125000 0x0 0x1000>;
status = "disabled";
};
frame@b126000 {
frame-number = <4>;
interrupts = <GIC_SPI 12 IRQ_TYPE_LEVEL_HIGH>;
- reg = <0x0b126000 0x1000>;
+ reg = <0x0 0x0b126000 0x0 0x1000>;
status = "disabled";
};
frame@b127000 {
frame-number = <5>;
interrupts = <GIC_SPI 13 IRQ_TYPE_LEVEL_HIGH>;
- reg = <0x0b127000 0x1000>;
+ reg = <0x0 0x0b127000 0x0 0x1000>;
status = "disabled";
};
frame@b128000 {
frame-number = <6>;
interrupts = <GIC_SPI 14 IRQ_TYPE_LEVEL_HIGH>;
- reg = <0x0b128000 0x1000>;
+ reg = <0x0 0x0b128000 0x0 0x1000>;
status = "disabled";
};
};
q6v5_wcss: remoteproc@cd00000 {
compatible = "qcom,ipq8074-wcss-pil";
- reg = <0x0cd00000 0x4040>,
- <0x004ab000 0x20>;
+ reg = <0x0 0x0cd00000 0x0 0x4040>,
+ <0x0 0x004ab000 0x0 0x20>;
reg-names = "qdsp6",
"rmb";
interrupts-extended = <&intc GIC_SPI 325 IRQ_TYPE_EDGE_RISING>,
clock-frequency = <0>;
};
+ audio_clk_b: audio_clk_b {
+ compatible = "fixed-clock";
+ #clock-cells = <0>;
+ clock-frequency = <0>;
+ };
+
audio_clk_c: audio_clk_c {
compatible = "fixed-clock";
#clock-cells = <0>;
interrupts = <RK_PB2 IRQ_TYPE_LEVEL_LOW>;
pinctrl-names = "default";
pinctrl-0 = <&pmic_int>;
- rockchip,system-power-controller;
wakeup-source;
#clock-cells = <1>;
clock-output-names = "rk808-clkout1", "xin32k";
gmac_clk: gmac-clock {
compatible = "fixed-clock";
clock-frequency = <125000000>;
- clock-output-names = "gmac_clk";
+ clock-output-names = "gmac_clkin";
#clock-cells = <0>;
};
label = "red:diy";
gpios = <&gpio0 RK_PB5 GPIO_ACTIVE_HIGH>;
default-state = "off";
- linux,default-trigger = "mmc1";
+ linux,default-trigger = "mmc2";
};
yellow_led: led-2 {
label = "yellow:yellow-led";
gpios = <&gpio0 RK_PA2 GPIO_ACTIVE_HIGH>;
default-state = "off";
- linux,default-trigger = "mmc0";
+ linux,default-trigger = "mmc1";
};
};
i2c6 = &i2c6;
i2c7 = &i2c7;
i2c8 = &i2c8;
+ mmc0 = &sdio0;
+ mmc1 = &sdmmc;
+ mmc2 = &sdhci;
serial0 = &uart0;
serial1 = &uart1;
serial2 = &uart2;
/*
* CPU feature detected at boot time based on feature of one or more CPUs.
* All possible conflicts for a late CPU are ignored.
+ * NOTE: this means that a late CPU with the feature will *not* cause the
+ * capability to be advertised by cpus_have_*cap()!
*/
#define ARM64_CPUCAP_WEAK_LOCAL_CPU_FEATURE \
(ARM64_CPUCAP_SCOPE_LOCAL_CPU | \
#define QCOM_CPU_PART_FALKOR_V1 0x800
#define QCOM_CPU_PART_FALKOR 0xC00
#define QCOM_CPU_PART_KRYO 0x200
+#define QCOM_CPU_PART_KRYO_2XX_GOLD 0x800
+#define QCOM_CPU_PART_KRYO_2XX_SILVER 0x801
#define QCOM_CPU_PART_KRYO_3XX_SILVER 0x803
#define QCOM_CPU_PART_KRYO_4XX_GOLD 0x804
#define QCOM_CPU_PART_KRYO_4XX_SILVER 0x805
#define MIDR_QCOM_FALKOR_V1 MIDR_CPU_MODEL(ARM_CPU_IMP_QCOM, QCOM_CPU_PART_FALKOR_V1)
#define MIDR_QCOM_FALKOR MIDR_CPU_MODEL(ARM_CPU_IMP_QCOM, QCOM_CPU_PART_FALKOR)
#define MIDR_QCOM_KRYO MIDR_CPU_MODEL(ARM_CPU_IMP_QCOM, QCOM_CPU_PART_KRYO)
+#define MIDR_QCOM_KRYO_2XX_GOLD MIDR_CPU_MODEL(ARM_CPU_IMP_QCOM, QCOM_CPU_PART_KRYO_2XX_GOLD)
+#define MIDR_QCOM_KRYO_2XX_SILVER MIDR_CPU_MODEL(ARM_CPU_IMP_QCOM, QCOM_CPU_PART_KRYO_2XX_SILVER)
#define MIDR_QCOM_KRYO_3XX_SILVER MIDR_CPU_MODEL(ARM_CPU_IMP_QCOM, QCOM_CPU_PART_KRYO_3XX_SILVER)
#define MIDR_QCOM_KRYO_4XX_GOLD MIDR_CPU_MODEL(ARM_CPU_IMP_QCOM, QCOM_CPU_PART_KRYO_4XX_GOLD)
#define MIDR_QCOM_KRYO_4XX_SILVER MIDR_CPU_MODEL(ARM_CPU_IMP_QCOM, QCOM_CPU_PART_KRYO_4XX_SILVER)
{
unsigned long flags = regs->pstate & DAIF_MASK;
+ if (interrupts_enabled(regs))
+ trace_hardirqs_on();
+
/*
* We can't use local_daif_restore(regs->pstate) here as
* system_has_prio_mask_debugging() won't restore the I bit if it can
return esr;
}
+asmlinkage void noinstr enter_el1_irq_or_nmi(struct pt_regs *regs);
+asmlinkage void noinstr exit_el1_irq_or_nmi(struct pt_regs *regs);
asmlinkage void enter_from_user_mode(void);
+asmlinkage void exit_to_user_mode(void);
+void arm64_enter_nmi(struct pt_regs *regs);
+void arm64_exit_nmi(struct pt_regs *regs);
void do_mem_abort(unsigned long addr, unsigned int esr, struct pt_regs *regs);
void do_undefinstr(struct pt_regs *regs);
void do_bti(struct pt_regs *regs);
*/
unsigned long *pmu_filter;
unsigned int pmuver;
+
+ u8 pfr0_csv2;
};
struct kvm_vcpu_fault_info {
#define pte_valid(pte) (!!(pte_val(pte) & PTE_VALID))
#define pte_valid_not_user(pte) \
((pte_val(pte) & (PTE_VALID | PTE_USER)) == PTE_VALID)
-#define pte_valid_young(pte) \
- ((pte_val(pte) & (PTE_VALID | PTE_AF)) == (PTE_VALID | PTE_AF))
#define pte_valid_user(pte) \
((pte_val(pte) & (PTE_VALID | PTE_USER)) == (PTE_VALID | PTE_USER))
* Could the pte be present in the TLB? We must check mm_tlb_flush_pending
* so that we don't erroneously return false for pages that have been
* remapped as PROT_NONE but are yet to be flushed from the TLB.
+ * Note that we can't make any assumptions based on the state of the access
+ * flag, since ptep_clear_flush_young() elides a DSB when invalidating the
+ * TLB.
*/
#define pte_accessible(mm, pte) \
- (mm_tlb_flush_pending(mm) ? pte_present(pte) : pte_valid_young(pte))
+ (mm_tlb_flush_pending(mm) ? pte_present(pte) : pte_valid(pte))
/*
* p??_access_permitted() is true for valid user mappings (subject to the
return pmd;
}
-static inline pte_t pte_wrprotect(pte_t pte)
-{
- pte = clear_pte_bit(pte, __pgprot(PTE_WRITE));
- pte = set_pte_bit(pte, __pgprot(PTE_RDONLY));
- return pte;
-}
-
static inline pte_t pte_mkwrite(pte_t pte)
{
pte = set_pte_bit(pte, __pgprot(PTE_WRITE));
return pte;
}
+static inline pte_t pte_wrprotect(pte_t pte)
+{
+ /*
+ * If hardware-dirty (PTE_WRITE/DBM bit set and PTE_RDONLY
+ * clear), set the PTE_DIRTY bit.
+ */
+ if (pte_hw_dirty(pte))
+ pte = pte_mkdirty(pte);
+
+ pte = clear_pte_bit(pte, __pgprot(PTE_WRITE));
+ pte = set_pte_bit(pte, __pgprot(PTE_RDONLY));
+ return pte;
+}
+
static inline pte_t pte_mkold(pte_t pte)
{
return clear_pte_bit(pte, __pgprot(PTE_AF));
pte = READ_ONCE(*ptep);
do {
old_pte = pte;
- /*
- * If hardware-dirty (PTE_WRITE/DBM bit set and PTE_RDONLY
- * clear), set the PTE_DIRTY bit.
- */
- if (pte_hw_dirty(pte))
- pte = pte_mkdirty(pte);
pte = pte_wrprotect(pte);
pte_val(pte) = cmpxchg_relaxed(&pte_val(*ptep),
pte_val(old_pte), pte_val(pte));
#ifndef _ARM_PROBES_H
#define _ARM_PROBES_H
+#include <asm/insn.h>
+
typedef u32 probe_opcode_t;
typedef void (probes_handler_t) (u32 opcode, long addr, struct pt_regs *);
/* Only valid when ARM64_HAS_IRQ_PRIO_MASKING is enabled. */
u64 pmr_save;
u64 stackframe[2];
+
+ /* Only valid for some EL1 exceptions. */
+ u64 lockdep_hardirqs;
+ u64 exit_rcu;
};
static inline bool in_syscall(struct pt_regs const *regs)
#define SYS_CONTEXTIDR_EL1 sys_reg(3, 0, 13, 0, 1)
#define SYS_TPIDR_EL1 sys_reg(3, 0, 13, 0, 4)
+#define SYS_SCXTNUM_EL1 sys_reg(3, 0, 13, 0, 7)
+
#define SYS_CNTKCTL_EL1 sys_reg(3, 0, 14, 1, 0)
#define SYS_CCSIDR_EL1 sys_reg(3, 1, 0, 0, 0)
#define SYS_TPIDR_EL0 sys_reg(3, 3, 13, 0, 2)
#define SYS_TPIDRRO_EL0 sys_reg(3, 3, 13, 0, 3)
+#define SYS_SCXTNUM_EL0 sys_reg(3, 3, 13, 0, 7)
+
/* Definitions for system register interface to AMU for ARMv8.4 onwards */
#define SYS_AM_EL0(crm, op2) sys_reg(3, 3, 13, (crm), (op2))
#define SYS_AMCR_EL0 SYS_AM_EL0(2, 0)
#define SYS_TFSR_EL1_TF0_SHIFT 0
#define SYS_TFSR_EL1_TF1_SHIFT 1
#define SYS_TFSR_EL1_TF0 (UL(1) << SYS_TFSR_EL1_TF0_SHIFT)
-#define SYS_TFSR_EL1_TF1 (UK(2) << SYS_TFSR_EL1_TF1_SHIFT)
+#define SYS_TFSR_EL1_TF1 (UL(1) << SYS_TFSR_EL1_TF1_SHIFT)
/* Safe value for MPIDR_EL1: Bit31:RES1, Bit30:U:0, Bit24:MT:0 */
#define SYS_MPIDR_SAFE_VAL (BIT(31))
MIDR_REV_RANGE(MIDR_CORTEX_A53, 0, 0, 4),
/* Brahma-B53 r0p[0] */
MIDR_REV(MIDR_BRAHMA_B53, 0, 0),
+ /* Kryo2XX Silver rAp4 */
+ MIDR_REV(MIDR_QCOM_KRYO_2XX_SILVER, 0xa, 0x4),
{},
};
#endif
MIDR_ALL_VERSIONS(MIDR_CORTEX_A73),
MIDR_ALL_VERSIONS(MIDR_HISI_TSV110),
MIDR_ALL_VERSIONS(MIDR_NVIDIA_CARMEL),
+ MIDR_ALL_VERSIONS(MIDR_QCOM_KRYO_2XX_GOLD),
+ MIDR_ALL_VERSIONS(MIDR_QCOM_KRYO_2XX_SILVER),
MIDR_ALL_VERSIONS(MIDR_QCOM_KRYO_3XX_SILVER),
MIDR_ALL_VERSIONS(MIDR_QCOM_KRYO_4XX_SILVER),
{ /* sentinel */ }
#include <asm/mmu.h>
#include <asm/sysreg.h>
-static void notrace el1_abort(struct pt_regs *regs, unsigned long esr)
+/*
+ * This is intended to match the logic in irqentry_enter(), handling the kernel
+ * mode transitions only.
+ */
+static void noinstr enter_from_kernel_mode(struct pt_regs *regs)
+{
+ regs->exit_rcu = false;
+
+ if (!IS_ENABLED(CONFIG_TINY_RCU) && is_idle_task(current)) {
+ lockdep_hardirqs_off(CALLER_ADDR0);
+ rcu_irq_enter();
+ trace_hardirqs_off_finish();
+
+ regs->exit_rcu = true;
+ return;
+ }
+
+ lockdep_hardirqs_off(CALLER_ADDR0);
+ rcu_irq_enter_check_tick();
+ trace_hardirqs_off_finish();
+}
+
+/*
+ * This is intended to match the logic in irqentry_exit(), handling the kernel
+ * mode transitions only, and with preemption handled elsewhere.
+ */
+static void noinstr exit_to_kernel_mode(struct pt_regs *regs)
+{
+ lockdep_assert_irqs_disabled();
+
+ if (interrupts_enabled(regs)) {
+ if (regs->exit_rcu) {
+ trace_hardirqs_on_prepare();
+ lockdep_hardirqs_on_prepare(CALLER_ADDR0);
+ rcu_irq_exit();
+ lockdep_hardirqs_on(CALLER_ADDR0);
+ return;
+ }
+
+ trace_hardirqs_on();
+ } else {
+ if (regs->exit_rcu)
+ rcu_irq_exit();
+ }
+}
+
+void noinstr arm64_enter_nmi(struct pt_regs *regs)
+{
+ regs->lockdep_hardirqs = lockdep_hardirqs_enabled();
+
+ __nmi_enter();
+ lockdep_hardirqs_off(CALLER_ADDR0);
+ lockdep_hardirq_enter();
+ rcu_nmi_enter();
+
+ trace_hardirqs_off_finish();
+ ftrace_nmi_enter();
+}
+
+void noinstr arm64_exit_nmi(struct pt_regs *regs)
+{
+ bool restore = regs->lockdep_hardirqs;
+
+ ftrace_nmi_exit();
+ if (restore) {
+ trace_hardirqs_on_prepare();
+ lockdep_hardirqs_on_prepare(CALLER_ADDR0);
+ }
+
+ rcu_nmi_exit();
+ lockdep_hardirq_exit();
+ if (restore)
+ lockdep_hardirqs_on(CALLER_ADDR0);
+ __nmi_exit();
+}
+
+asmlinkage void noinstr enter_el1_irq_or_nmi(struct pt_regs *regs)
+{
+ if (IS_ENABLED(CONFIG_ARM64_PSEUDO_NMI) && !interrupts_enabled(regs))
+ arm64_enter_nmi(regs);
+ else
+ enter_from_kernel_mode(regs);
+}
+
+asmlinkage void noinstr exit_el1_irq_or_nmi(struct pt_regs *regs)
+{
+ if (IS_ENABLED(CONFIG_ARM64_PSEUDO_NMI) && !interrupts_enabled(regs))
+ arm64_exit_nmi(regs);
+ else
+ exit_to_kernel_mode(regs);
+}
+
+static void noinstr el1_abort(struct pt_regs *regs, unsigned long esr)
{
unsigned long far = read_sysreg(far_el1);
+ enter_from_kernel_mode(regs);
local_daif_inherit(regs);
far = untagged_addr(far);
do_mem_abort(far, esr, regs);
+ local_daif_mask();
+ exit_to_kernel_mode(regs);
}
-NOKPROBE_SYMBOL(el1_abort);
-static void notrace el1_pc(struct pt_regs *regs, unsigned long esr)
+static void noinstr el1_pc(struct pt_regs *regs, unsigned long esr)
{
unsigned long far = read_sysreg(far_el1);
+ enter_from_kernel_mode(regs);
local_daif_inherit(regs);
do_sp_pc_abort(far, esr, regs);
+ local_daif_mask();
+ exit_to_kernel_mode(regs);
}
-NOKPROBE_SYMBOL(el1_pc);
-static void notrace el1_undef(struct pt_regs *regs)
+static void noinstr el1_undef(struct pt_regs *regs)
{
+ enter_from_kernel_mode(regs);
local_daif_inherit(regs);
do_undefinstr(regs);
+ local_daif_mask();
+ exit_to_kernel_mode(regs);
}
-NOKPROBE_SYMBOL(el1_undef);
-static void notrace el1_inv(struct pt_regs *regs, unsigned long esr)
+static void noinstr el1_inv(struct pt_regs *regs, unsigned long esr)
{
+ enter_from_kernel_mode(regs);
local_daif_inherit(regs);
bad_mode(regs, 0, esr);
+ local_daif_mask();
+ exit_to_kernel_mode(regs);
}
-NOKPROBE_SYMBOL(el1_inv);
-static void notrace el1_dbg(struct pt_regs *regs, unsigned long esr)
+static void noinstr arm64_enter_el1_dbg(struct pt_regs *regs)
+{
+ regs->lockdep_hardirqs = lockdep_hardirqs_enabled();
+
+ lockdep_hardirqs_off(CALLER_ADDR0);
+ rcu_nmi_enter();
+
+ trace_hardirqs_off_finish();
+}
+
+static void noinstr arm64_exit_el1_dbg(struct pt_regs *regs)
+{
+ bool restore = regs->lockdep_hardirqs;
+
+ if (restore) {
+ trace_hardirqs_on_prepare();
+ lockdep_hardirqs_on_prepare(CALLER_ADDR0);
+ }
+
+ rcu_nmi_exit();
+ if (restore)
+ lockdep_hardirqs_on(CALLER_ADDR0);
+}
+
+static void noinstr el1_dbg(struct pt_regs *regs, unsigned long esr)
{
unsigned long far = read_sysreg(far_el1);
if (system_uses_irq_prio_masking())
gic_write_pmr(GIC_PRIO_IRQON | GIC_PRIO_PSR_I_SET);
+ arm64_enter_el1_dbg(regs);
do_debug_exception(far, esr, regs);
+ arm64_exit_el1_dbg(regs);
}
-NOKPROBE_SYMBOL(el1_dbg);
-static void notrace el1_fpac(struct pt_regs *regs, unsigned long esr)
+static void noinstr el1_fpac(struct pt_regs *regs, unsigned long esr)
{
+ enter_from_kernel_mode(regs);
local_daif_inherit(regs);
do_ptrauth_fault(regs, esr);
+ local_daif_mask();
+ exit_to_kernel_mode(regs);
}
-NOKPROBE_SYMBOL(el1_fpac);
-asmlinkage void notrace el1_sync_handler(struct pt_regs *regs)
+asmlinkage void noinstr el1_sync_handler(struct pt_regs *regs)
{
unsigned long esr = read_sysreg(esr_el1);
el1_inv(regs, esr);
}
}
-NOKPROBE_SYMBOL(el1_sync_handler);
-static void notrace el0_da(struct pt_regs *regs, unsigned long esr)
+asmlinkage void noinstr enter_from_user_mode(void)
+{
+ lockdep_hardirqs_off(CALLER_ADDR0);
+ CT_WARN_ON(ct_state() != CONTEXT_USER);
+ user_exit_irqoff();
+ trace_hardirqs_off_finish();
+}
+
+asmlinkage void noinstr exit_to_user_mode(void)
+{
+ trace_hardirqs_on_prepare();
+ lockdep_hardirqs_on_prepare(CALLER_ADDR0);
+ user_enter_irqoff();
+ lockdep_hardirqs_on(CALLER_ADDR0);
+}
+
+static void noinstr el0_da(struct pt_regs *regs, unsigned long esr)
{
unsigned long far = read_sysreg(far_el1);
- user_exit_irqoff();
+ enter_from_user_mode();
local_daif_restore(DAIF_PROCCTX);
far = untagged_addr(far);
do_mem_abort(far, esr, regs);
}
-NOKPROBE_SYMBOL(el0_da);
-static void notrace el0_ia(struct pt_regs *regs, unsigned long esr)
+static void noinstr el0_ia(struct pt_regs *regs, unsigned long esr)
{
unsigned long far = read_sysreg(far_el1);
if (!is_ttbr0_addr(far))
arm64_apply_bp_hardening();
- user_exit_irqoff();
+ enter_from_user_mode();
local_daif_restore(DAIF_PROCCTX);
do_mem_abort(far, esr, regs);
}
-NOKPROBE_SYMBOL(el0_ia);
-static void notrace el0_fpsimd_acc(struct pt_regs *regs, unsigned long esr)
+static void noinstr el0_fpsimd_acc(struct pt_regs *regs, unsigned long esr)
{
- user_exit_irqoff();
+ enter_from_user_mode();
local_daif_restore(DAIF_PROCCTX);
do_fpsimd_acc(esr, regs);
}
-NOKPROBE_SYMBOL(el0_fpsimd_acc);
-static void notrace el0_sve_acc(struct pt_regs *regs, unsigned long esr)
+static void noinstr el0_sve_acc(struct pt_regs *regs, unsigned long esr)
{
- user_exit_irqoff();
+ enter_from_user_mode();
local_daif_restore(DAIF_PROCCTX);
do_sve_acc(esr, regs);
}
-NOKPROBE_SYMBOL(el0_sve_acc);
-static void notrace el0_fpsimd_exc(struct pt_regs *regs, unsigned long esr)
+static void noinstr el0_fpsimd_exc(struct pt_regs *regs, unsigned long esr)
{
- user_exit_irqoff();
+ enter_from_user_mode();
local_daif_restore(DAIF_PROCCTX);
do_fpsimd_exc(esr, regs);
}
-NOKPROBE_SYMBOL(el0_fpsimd_exc);
-static void notrace el0_sys(struct pt_regs *regs, unsigned long esr)
+static void noinstr el0_sys(struct pt_regs *regs, unsigned long esr)
{
- user_exit_irqoff();
+ enter_from_user_mode();
local_daif_restore(DAIF_PROCCTX);
do_sysinstr(esr, regs);
}
-NOKPROBE_SYMBOL(el0_sys);
-static void notrace el0_pc(struct pt_regs *regs, unsigned long esr)
+static void noinstr el0_pc(struct pt_regs *regs, unsigned long esr)
{
unsigned long far = read_sysreg(far_el1);
if (!is_ttbr0_addr(instruction_pointer(regs)))
arm64_apply_bp_hardening();
- user_exit_irqoff();
+ enter_from_user_mode();
local_daif_restore(DAIF_PROCCTX);
do_sp_pc_abort(far, esr, regs);
}
-NOKPROBE_SYMBOL(el0_pc);
-static void notrace el0_sp(struct pt_regs *regs, unsigned long esr)
+static void noinstr el0_sp(struct pt_regs *regs, unsigned long esr)
{
- user_exit_irqoff();
+ enter_from_user_mode();
local_daif_restore(DAIF_PROCCTX);
do_sp_pc_abort(regs->sp, esr, regs);
}
-NOKPROBE_SYMBOL(el0_sp);
-static void notrace el0_undef(struct pt_regs *regs)
+static void noinstr el0_undef(struct pt_regs *regs)
{
- user_exit_irqoff();
+ enter_from_user_mode();
local_daif_restore(DAIF_PROCCTX);
do_undefinstr(regs);
}
-NOKPROBE_SYMBOL(el0_undef);
-static void notrace el0_bti(struct pt_regs *regs)
+static void noinstr el0_bti(struct pt_regs *regs)
{
- user_exit_irqoff();
+ enter_from_user_mode();
local_daif_restore(DAIF_PROCCTX);
do_bti(regs);
}
-NOKPROBE_SYMBOL(el0_bti);
-static void notrace el0_inv(struct pt_regs *regs, unsigned long esr)
+static void noinstr el0_inv(struct pt_regs *regs, unsigned long esr)
{
- user_exit_irqoff();
+ enter_from_user_mode();
local_daif_restore(DAIF_PROCCTX);
bad_el0_sync(regs, 0, esr);
}
-NOKPROBE_SYMBOL(el0_inv);
-static void notrace el0_dbg(struct pt_regs *regs, unsigned long esr)
+static void noinstr el0_dbg(struct pt_regs *regs, unsigned long esr)
{
/* Only watchpoints write FAR_EL1, otherwise its UNKNOWN */
unsigned long far = read_sysreg(far_el1);
if (system_uses_irq_prio_masking())
gic_write_pmr(GIC_PRIO_IRQON | GIC_PRIO_PSR_I_SET);
- user_exit_irqoff();
+ enter_from_user_mode();
do_debug_exception(far, esr, regs);
local_daif_restore(DAIF_PROCCTX_NOIRQ);
}
-NOKPROBE_SYMBOL(el0_dbg);
-static void notrace el0_svc(struct pt_regs *regs)
+static void noinstr el0_svc(struct pt_regs *regs)
{
if (system_uses_irq_prio_masking())
gic_write_pmr(GIC_PRIO_IRQON | GIC_PRIO_PSR_I_SET);
+ enter_from_user_mode();
do_el0_svc(regs);
}
-NOKPROBE_SYMBOL(el0_svc);
-static void notrace el0_fpac(struct pt_regs *regs, unsigned long esr)
+static void noinstr el0_fpac(struct pt_regs *regs, unsigned long esr)
{
- user_exit_irqoff();
+ enter_from_user_mode();
local_daif_restore(DAIF_PROCCTX);
do_ptrauth_fault(regs, esr);
}
-NOKPROBE_SYMBOL(el0_fpac);
-asmlinkage void notrace el0_sync_handler(struct pt_regs *regs)
+asmlinkage void noinstr el0_sync_handler(struct pt_regs *regs)
{
unsigned long esr = read_sysreg(esr_el1);
el0_inv(regs, esr);
}
}
-NOKPROBE_SYMBOL(el0_sync_handler);
#ifdef CONFIG_COMPAT
-static void notrace el0_cp15(struct pt_regs *regs, unsigned long esr)
+static void noinstr el0_cp15(struct pt_regs *regs, unsigned long esr)
{
- user_exit_irqoff();
+ enter_from_user_mode();
local_daif_restore(DAIF_PROCCTX);
do_cp15instr(esr, regs);
}
-NOKPROBE_SYMBOL(el0_cp15);
-static void notrace el0_svc_compat(struct pt_regs *regs)
+static void noinstr el0_svc_compat(struct pt_regs *regs)
{
if (system_uses_irq_prio_masking())
gic_write_pmr(GIC_PRIO_IRQON | GIC_PRIO_PSR_I_SET);
+ enter_from_user_mode();
do_el0_svc_compat(regs);
}
-NOKPROBE_SYMBOL(el0_svc_compat);
-asmlinkage void notrace el0_sync_compat_handler(struct pt_regs *regs)
+asmlinkage void noinstr el0_sync_compat_handler(struct pt_regs *regs)
{
unsigned long esr = read_sysreg(esr_el1);
el0_inv(regs, esr);
}
}
-NOKPROBE_SYMBOL(el0_sync_compat_handler);
#endif /* CONFIG_COMPAT */
#include <asm/unistd.h>
/*
- * Context tracking subsystem. Used to instrument transitions
- * between user and kernel mode.
+ * Context tracking and irqflag tracing need to instrument transitions between
+ * user and kernel mode.
*/
- .macro ct_user_exit_irqoff
-#ifdef CONFIG_CONTEXT_TRACKING
+ .macro user_exit_irqoff
+#if defined(CONFIG_CONTEXT_TRACKING) || defined(CONFIG_TRACE_IRQFLAGS)
bl enter_from_user_mode
#endif
.endm
- .macro ct_user_enter
-#ifdef CONFIG_CONTEXT_TRACKING
- bl context_tracking_user_enter
+ .macro user_enter_irqoff
+#if defined(CONFIG_CONTEXT_TRACKING) || defined(CONFIG_TRACE_IRQFLAGS)
+ bl exit_to_user_mode
#endif
.endm
alternative_else_nop_endif
ldp x21, x22, [sp, #S_PC] // load ELR, SPSR
- .if \el == 0
- ct_user_enter
- .endif
#ifdef CONFIG_ARM64_SW_TTBR0_PAN
alternative_if_not ARM64_HAS_PAN
gic_prio_irq_setup pmr=x20, tmp=x1
enable_da_f
-#ifdef CONFIG_ARM64_PSEUDO_NMI
- test_irqs_unmasked res=x0, pmr=x20
- cbz x0, 1f
- bl asm_nmi_enter
-1:
-#endif
-
-#ifdef CONFIG_TRACE_IRQFLAGS
- bl trace_hardirqs_off
-#endif
+ mov x0, sp
+ bl enter_el1_irq_or_nmi
irq_handler
1:
#endif
-#ifdef CONFIG_ARM64_PSEUDO_NMI
- /*
- * When using IRQ priority masking, we can get spurious interrupts while
- * PMR is set to GIC_PRIO_IRQOFF. An NMI might also have occurred in a
- * section with interrupts disabled. Skip tracing in those cases.
- */
- test_irqs_unmasked res=x0, pmr=x20
- cbz x0, 1f
- bl asm_nmi_exit
-1:
-#endif
-
-#ifdef CONFIG_TRACE_IRQFLAGS
-#ifdef CONFIG_ARM64_PSEUDO_NMI
- test_irqs_unmasked res=x0, pmr=x20
- cbnz x0, 1f
-#endif
- bl trace_hardirqs_on
-1:
-#endif
+ mov x0, sp
+ bl exit_el1_irq_or_nmi
kernel_exit 1
SYM_CODE_END(el1_irq)
kernel_entry 0
el0_irq_naked:
gic_prio_irq_setup pmr=x20, tmp=x0
- ct_user_exit_irqoff
+ user_exit_irqoff
enable_da_f
-#ifdef CONFIG_TRACE_IRQFLAGS
- bl trace_hardirqs_off
-#endif
-
tbz x22, #55, 1f
bl do_el0_irq_bp_hardening
1:
irq_handler
-#ifdef CONFIG_TRACE_IRQFLAGS
- bl trace_hardirqs_on
-#endif
b ret_to_user
SYM_CODE_END(el0_irq)
el0_error_naked:
mrs x25, esr_el1
gic_prio_kentry_setup tmp=x2
- ct_user_exit_irqoff
+ user_exit_irqoff
enable_dbg
mov x0, sp
mov x1, x25
SYM_CODE_START_LOCAL(ret_to_user)
disable_daif
gic_prio_kentry_setup tmp=x3
- ldr x1, [tsk, #TSK_TI_FLAGS]
- and x2, x1, #_TIF_WORK_MASK
+#ifdef CONFIG_TRACE_IRQFLAGS
+ bl trace_hardirqs_off
+#endif
+ ldr x19, [tsk, #TSK_TI_FLAGS]
+ and x2, x19, #_TIF_WORK_MASK
cbnz x2, work_pending
finish_ret_to_user:
+ user_enter_irqoff
/* Ignore asynchronous tag check faults in the uaccess routines */
clear_mte_async_tcf
- enable_step_tsk x1, x2
+ enable_step_tsk x19, x2
#ifdef CONFIG_GCC_PLUGIN_STACKLEAK
bl stackleak_erase
#endif
*/
work_pending:
mov x0, sp // 'regs'
+ mov x1, x19
bl do_notify_resume
-#ifdef CONFIG_TRACE_IRQFLAGS
- bl trace_hardirqs_on // enabled while in userspace
-#endif
- ldr x1, [tsk, #TSK_TI_FLAGS] // re-check for single-step
+ ldr x19, [tsk, #TSK_TI_FLAGS] // re-check for single-step
b finish_ret_to_user
SYM_CODE_END(ret_to_user)
local_daif_restore(DAIF_PROCCTX_NOIRQ);
}
}
-
-/*
- * Stubs to make nmi_enter/exit() code callable from ASM
- */
-asmlinkage void notrace asm_nmi_enter(void)
-{
- nmi_enter();
-}
-NOKPROBE_SYMBOL(asm_nmi_enter);
-
-asmlinkage void notrace asm_nmi_exit(void)
-{
- nmi_exit();
-}
-NOKPROBE_SYMBOL(asm_nmi_exit);
kernel_segment->mem, kbuf.bufsz,
kernel_segment->memsz);
- return 0;
+ return NULL;
}
#ifdef CONFIG_KEXEC_IMAGE_VERIFY_SIG
}
void perf_get_regs_user(struct perf_regs *regs_user,
- struct pt_regs *regs,
- struct pt_regs *regs_user_copy)
+ struct pt_regs *regs)
{
regs_user->regs = task_pt_regs(current);
regs_user->abi = perf_reg_abi(current);
void (*arm_pm_restart)(enum reboot_mode reboot_mode, const char *cmd);
-static void __cpu_do_idle(void)
+static void noinstr __cpu_do_idle(void)
{
dsb(sy);
wfi();
}
-static void __cpu_do_idle_irqprio(void)
+static void noinstr __cpu_do_idle_irqprio(void)
{
unsigned long pmr;
unsigned long daif_bits;
* ensure that interrupts are not masked at the PMR (because the core will
* not wake up if we block the wake up signal in the interrupt controller).
*/
-void cpu_do_idle(void)
+void noinstr cpu_do_idle(void)
{
if (system_uses_irq_prio_masking())
__cpu_do_idle_irqprio();
/*
* This is our default idle handler.
*/
-void arch_cpu_idle(void)
+void noinstr arch_cpu_idle(void)
{
/*
* This should do all the clock switching and wait for interrupt
* tricks
*/
cpu_do_idle();
- local_irq_enable();
+ raw_local_irq_enable();
}
#ifdef CONFIG_HOTPLUG_CPU
bool prev32, next32;
u64 val;
- if (!(IS_ENABLED(CONFIG_ARM64_ERRATUM_1418040) &&
- cpus_have_const_cap(ARM64_WORKAROUND_1418040)))
+ if (!IS_ENABLED(CONFIG_ARM64_ERRATUM_1418040))
return;
prev32 = is_compat_thread(task_thread_info(prev));
next32 = is_compat_thread(task_thread_info(next));
- if (prev32 == next32)
+ if (prev32 == next32 || !this_cpu_has_cap(ARM64_WORKAROUND_1418040))
return;
val = read_sysreg(cntkctl_el1);
MIDR_ALL_VERSIONS(MIDR_CORTEX_A55),
MIDR_ALL_VERSIONS(MIDR_BRAHMA_B53),
MIDR_ALL_VERSIONS(MIDR_HISI_TSV110),
+ MIDR_ALL_VERSIONS(MIDR_QCOM_KRYO_2XX_SILVER),
MIDR_ALL_VERSIONS(MIDR_QCOM_KRYO_3XX_SILVER),
MIDR_ALL_VERSIONS(MIDR_QCOM_KRYO_4XX_SILVER),
{ /* sentinel */ }
static void cpu_psci_cpu_die(unsigned int cpu)
{
- int ret;
/*
* There are no known implementations of PSCI actually using the
* power state field, pass a sensible default for now.
u32 state = PSCI_POWER_STATE_TYPE_POWER_DOWN <<
PSCI_0_2_POWER_STATE_TYPE_SHIFT;
- ret = psci_ops.cpu_off(state);
-
- pr_crit("unable to power off CPU%u (%d)\n", cpu, ret);
+ psci_ops.cpu_off(state);
}
static int cpu_psci_cpu_kill(unsigned int cpu)
#include <linux/uaccess.h>
#include <asm/alternative.h>
+#include <asm/exception.h>
#include <asm/kprobes.h>
#include <asm/mmu.h>
#include <asm/ptrace.h>
}
-asmlinkage __kprobes notrace unsigned long
+asmlinkage noinstr unsigned long
__sdei_handler(struct pt_regs *regs, struct sdei_registered_event *arg)
{
unsigned long ret;
- nmi_enter();
+ arm64_enter_nmi(regs);
ret = _sdei_handler(regs, arg);
- nmi_exit();
+ arm64_exit_nmi(regs);
return ret;
}
/* Mark this CPU absent */
set_cpu_present(cpu, 0);
+ rcu_report_dead(cpu);
if (IS_ENABLED(CONFIG_HOTPLUG_CPU)) {
update_cpu_boot_status(CPU_KILL_ME);
cortex_a76_erratum_1463225_svc_handler();
local_daif_restore(DAIF_PROCCTX);
- user_exit();
if (system_supports_mte() && (flags & _TIF_MTE_ASYNC_FAULT)) {
/*
#include <asm/daifflags.h>
#include <asm/debug-monitors.h>
#include <asm/esr.h>
+#include <asm/exception.h>
#include <asm/extable.h>
#include <asm/insn.h>
#include <asm/kprobes.h>
* bad_mode handles the impossible case in the exception vector. This is always
* fatal.
*/
-asmlinkage void bad_mode(struct pt_regs *regs, int reason, unsigned int esr)
+asmlinkage void notrace bad_mode(struct pt_regs *regs, int reason, unsigned int esr)
{
+ arm64_enter_nmi(regs);
+
console_verbose();
pr_crit("Bad mode in %s handler detected on CPU%d, code 0x%08x -- %s\n",
DEFINE_PER_CPU(unsigned long [OVERFLOW_STACK_SIZE/sizeof(long)], overflow_stack)
__aligned(16);
-asmlinkage void handle_bad_stack(struct pt_regs *regs)
+asmlinkage void noinstr handle_bad_stack(struct pt_regs *regs)
{
unsigned long tsk_stk = (unsigned long)current->stack;
unsigned long irq_stk = (unsigned long)this_cpu_read(irq_stack_ptr);
unsigned int esr = read_sysreg(esr_el1);
unsigned long far = read_sysreg(far_el1);
+ arm64_enter_nmi(regs);
+
console_verbose();
pr_emerg("Insufficient stack space to handle exception!");
}
}
-asmlinkage void do_serror(struct pt_regs *regs, unsigned int esr)
+asmlinkage void noinstr do_serror(struct pt_regs *regs, unsigned int esr)
{
- nmi_enter();
+ arm64_enter_nmi(regs);
/* non-RAS errors are not containable */
if (!arm64_is_ras_serror(esr) || arm64_is_fatal_ras_serror(regs, esr))
arm64_serror_panic(regs, esr);
- nmi_exit();
-}
-
-asmlinkage void enter_from_user_mode(void)
-{
- CT_WARN_ON(ct_state() != CONTEXT_USER);
- user_exit_irqoff();
+ arm64_exit_nmi(regs);
}
-NOKPROBE_SYMBOL(enter_from_user_mode);
/* GENERIC_BUG traps */
return vgic_present ? kvm_vgic_get_max_vcpus() : KVM_MAX_VCPUS;
}
+static void set_default_csv2(struct kvm *kvm)
+{
+ /*
+ * The default is to expose CSV2 == 1 if the HW isn't affected.
+ * Although this is a per-CPU feature, we make it global because
+ * asymmetric systems are just a nuisance.
+ *
+ * Userspace can override this as long as it doesn't promise
+ * the impossible.
+ */
+ if (arm64_get_spectre_v2_state() == SPECTRE_UNAFFECTED)
+ kvm->arch.pfr0_csv2 = 1;
+}
+
/**
* kvm_arch_init_vm - initializes a VM data structure
* @kvm: pointer to the KVM struct
/* The maximum number of VCPUs is limited by the host's GIC model */
kvm->arch.max_vcpus = kvm_arm_default_max_vcpus();
+ set_default_csv2(kvm);
+
return ret;
out_free_stage2_pgd:
kvm_free_stage2_pgd(&kvm->arch.mmu);
SECTIONS {
HYP_SECTION(.text)
+ /*
+ * .hyp..data..percpu needs to be page aligned to maintain the same
+ * alignment for when linking into vmlinux.
+ */
+ . = ALIGN(PAGE_SIZE);
HYP_SECTION_NAME(.data..percpu) : {
PERCPU_INPUT(L1_CACHE_BYTES)
}
{ SYS_DESC(SYS_PMEVTYPERn_EL0(n)), \
access_pmu_evtyper, reset_unknown, (PMEVTYPER0_EL0 + n), }
-static bool access_amu(struct kvm_vcpu *vcpu, struct sys_reg_params *p,
- const struct sys_reg_desc *r)
+static bool undef_access(struct kvm_vcpu *vcpu, struct sys_reg_params *p,
+ const struct sys_reg_desc *r)
{
kvm_inject_undefined(vcpu);
}
/* Macro to expand the AMU counter and type registers*/
-#define AMU_AMEVCNTR0_EL0(n) { SYS_DESC(SYS_AMEVCNTR0_EL0(n)), access_amu }
-#define AMU_AMEVTYPER0_EL0(n) { SYS_DESC(SYS_AMEVTYPER0_EL0(n)), access_amu }
-#define AMU_AMEVCNTR1_EL0(n) { SYS_DESC(SYS_AMEVCNTR1_EL0(n)), access_amu }
-#define AMU_AMEVTYPER1_EL0(n) { SYS_DESC(SYS_AMEVTYPER1_EL0(n)), access_amu }
-
-static bool trap_ptrauth(struct kvm_vcpu *vcpu,
- struct sys_reg_params *p,
- const struct sys_reg_desc *rd)
-{
- /*
- * If we land here, that is because we didn't fixup the access on exit
- * by allowing the PtrAuth sysregs. The only way this happens is when
- * the guest does not have PtrAuth support enabled.
- */
- kvm_inject_undefined(vcpu);
-
- return false;
-}
+#define AMU_AMEVCNTR0_EL0(n) { SYS_DESC(SYS_AMEVCNTR0_EL0(n)), undef_access }
+#define AMU_AMEVTYPER0_EL0(n) { SYS_DESC(SYS_AMEVTYPER0_EL0(n)), undef_access }
+#define AMU_AMEVCNTR1_EL0(n) { SYS_DESC(SYS_AMEVCNTR1_EL0(n)), undef_access }
+#define AMU_AMEVTYPER1_EL0(n) { SYS_DESC(SYS_AMEVTYPER1_EL0(n)), undef_access }
static unsigned int ptrauth_visibility(const struct kvm_vcpu *vcpu,
const struct sys_reg_desc *rd)
return vcpu_has_ptrauth(vcpu) ? 0 : REG_HIDDEN;
}
+/*
+ * If we land here on a PtrAuth access, that is because we didn't
+ * fixup the access on exit by allowing the PtrAuth sysregs. The only
+ * way this happens is when the guest does not have PtrAuth support
+ * enabled.
+ */
#define __PTRAUTH_KEY(k) \
- { SYS_DESC(SYS_## k), trap_ptrauth, reset_unknown, k, \
+ { SYS_DESC(SYS_## k), undef_access, reset_unknown, k, \
.visibility = ptrauth_visibility}
#define PTRAUTH_KEY(k) \
if (!vcpu_has_sve(vcpu))
val &= ~(0xfUL << ID_AA64PFR0_SVE_SHIFT);
val &= ~(0xfUL << ID_AA64PFR0_AMU_SHIFT);
- if (!(val & (0xfUL << ID_AA64PFR0_CSV2_SHIFT)) &&
- arm64_get_spectre_v2_state() == SPECTRE_UNAFFECTED)
- val |= (1UL << ID_AA64PFR0_CSV2_SHIFT);
+ val &= ~(0xfUL << ID_AA64PFR0_CSV2_SHIFT);
+ val |= ((u64)vcpu->kvm->arch.pfr0_csv2 << ID_AA64PFR0_CSV2_SHIFT);
} else if (id == SYS_ID_AA64PFR1_EL1) {
val &= ~(0xfUL << ID_AA64PFR1_MTE_SHIFT);
} else if (id == SYS_ID_AA64ISAR1_EL1 && !vcpu_has_ptrauth(vcpu)) {
return REG_HIDDEN;
}
+static int set_id_aa64pfr0_el1(struct kvm_vcpu *vcpu,
+ const struct sys_reg_desc *rd,
+ const struct kvm_one_reg *reg, void __user *uaddr)
+{
+ const u64 id = sys_reg_to_index(rd);
+ int err;
+ u64 val;
+ u8 csv2;
+
+ err = reg_from_user(&val, uaddr, id);
+ if (err)
+ return err;
+
+ /*
+ * Allow AA64PFR0_EL1.CSV2 to be set from userspace as long as
+ * it doesn't promise more than what is actually provided (the
+ * guest could otherwise be covered in ectoplasmic residue).
+ */
+ csv2 = cpuid_feature_extract_unsigned_field(val, ID_AA64PFR0_CSV2_SHIFT);
+ if (csv2 > 1 ||
+ (csv2 && arm64_get_spectre_v2_state() != SPECTRE_UNAFFECTED))
+ return -EINVAL;
+
+ /* We can only differ with CSV2, and anything else is an error */
+ val ^= read_id_reg(vcpu, rd, false);
+ val &= ~(0xFUL << ID_AA64PFR0_CSV2_SHIFT);
+ if (val)
+ return -EINVAL;
+
+ vcpu->kvm->arch.pfr0_csv2 = csv2;
+
+ return 0;
+}
+
/*
* cpufeature ID register user accessors
*
return true;
}
-static bool access_mte_regs(struct kvm_vcpu *vcpu, struct sys_reg_params *p,
- const struct sys_reg_desc *r)
-{
- kvm_inject_undefined(vcpu);
- return false;
-}
-
/* sys_reg_desc initialiser for known cpufeature ID registers */
#define ID_SANITISED(name) { \
SYS_DESC(SYS_##name), \
/* AArch64 ID registers */
/* CRm=4 */
- ID_SANITISED(ID_AA64PFR0_EL1),
+ { SYS_DESC(SYS_ID_AA64PFR0_EL1), .access = access_id_reg,
+ .get_user = get_id_reg, .set_user = set_id_aa64pfr0_el1, },
ID_SANITISED(ID_AA64PFR1_EL1),
ID_UNALLOCATED(4,2),
ID_UNALLOCATED(4,3),
{ SYS_DESC(SYS_ACTLR_EL1), access_actlr, reset_actlr, ACTLR_EL1 },
{ SYS_DESC(SYS_CPACR_EL1), NULL, reset_val, CPACR_EL1, 0 },
- { SYS_DESC(SYS_RGSR_EL1), access_mte_regs },
- { SYS_DESC(SYS_GCR_EL1), access_mte_regs },
+ { SYS_DESC(SYS_RGSR_EL1), undef_access },
+ { SYS_DESC(SYS_GCR_EL1), undef_access },
{ SYS_DESC(SYS_ZCR_EL1), NULL, reset_val, ZCR_EL1, 0, .visibility = sve_visibility },
{ SYS_DESC(SYS_TTBR0_EL1), access_vm_reg, reset_unknown, TTBR0_EL1 },
{ SYS_DESC(SYS_ERXMISC0_EL1), trap_raz_wi },
{ SYS_DESC(SYS_ERXMISC1_EL1), trap_raz_wi },
- { SYS_DESC(SYS_TFSR_EL1), access_mte_regs },
- { SYS_DESC(SYS_TFSRE0_EL1), access_mte_regs },
+ { SYS_DESC(SYS_TFSR_EL1), undef_access },
+ { SYS_DESC(SYS_TFSRE0_EL1), undef_access },
{ SYS_DESC(SYS_FAR_EL1), access_vm_reg, reset_unknown, FAR_EL1 },
{ SYS_DESC(SYS_PAR_EL1), NULL, reset_unknown, PAR_EL1 },
{ SYS_DESC(SYS_CONTEXTIDR_EL1), access_vm_reg, reset_val, CONTEXTIDR_EL1, 0 },
{ SYS_DESC(SYS_TPIDR_EL1), NULL, reset_unknown, TPIDR_EL1 },
+ { SYS_DESC(SYS_SCXTNUM_EL1), undef_access },
+
{ SYS_DESC(SYS_CNTKCTL_EL1), NULL, reset_val, CNTKCTL_EL1, 0},
{ SYS_DESC(SYS_CCSIDR_EL1), access_ccsidr },
{ SYS_DESC(SYS_TPIDR_EL0), NULL, reset_unknown, TPIDR_EL0 },
{ SYS_DESC(SYS_TPIDRRO_EL0), NULL, reset_unknown, TPIDRRO_EL0 },
- { SYS_DESC(SYS_AMCR_EL0), access_amu },
- { SYS_DESC(SYS_AMCFGR_EL0), access_amu },
- { SYS_DESC(SYS_AMCGCR_EL0), access_amu },
- { SYS_DESC(SYS_AMUSERENR_EL0), access_amu },
- { SYS_DESC(SYS_AMCNTENCLR0_EL0), access_amu },
- { SYS_DESC(SYS_AMCNTENSET0_EL0), access_amu },
- { SYS_DESC(SYS_AMCNTENCLR1_EL0), access_amu },
- { SYS_DESC(SYS_AMCNTENSET1_EL0), access_amu },
+ { SYS_DESC(SYS_SCXTNUM_EL0), undef_access },
+
+ { SYS_DESC(SYS_AMCR_EL0), undef_access },
+ { SYS_DESC(SYS_AMCFGR_EL0), undef_access },
+ { SYS_DESC(SYS_AMCGCR_EL0), undef_access },
+ { SYS_DESC(SYS_AMUSERENR_EL0), undef_access },
+ { SYS_DESC(SYS_AMCNTENCLR0_EL0), undef_access },
+ { SYS_DESC(SYS_AMCNTENSET0_EL0), undef_access },
+ { SYS_DESC(SYS_AMCNTENCLR1_EL0), undef_access },
+ { SYS_DESC(SYS_AMCNTENSET1_EL0), undef_access },
AMU_AMEVCNTR0_EL0(0),
AMU_AMEVCNTR0_EL0(1),
AMU_AMEVCNTR0_EL0(2),
return extract_bytes(value, addr & 7, len);
}
+static unsigned long vgic_uaccess_read_v3r_typer(struct kvm_vcpu *vcpu,
+ gpa_t addr, unsigned int len)
+{
+ unsigned long mpidr = kvm_vcpu_get_mpidr_aff(vcpu);
+ int target_vcpu_id = vcpu->vcpu_id;
+ u64 value;
+
+ value = (u64)(mpidr & GENMASK(23, 0)) << 32;
+ value |= ((target_vcpu_id & 0xffff) << 8);
+
+ if (vgic_has_its(vcpu->kvm))
+ value |= GICR_TYPER_PLPIS;
+
+ /* reporting of the Last bit is not supported for userspace */
+ return extract_bytes(value, addr & 7, len);
+}
+
static unsigned long vgic_mmio_read_v3r_iidr(struct kvm_vcpu *vcpu,
gpa_t addr, unsigned int len)
{
REGISTER_DESC_WITH_LENGTH(GICR_IIDR,
vgic_mmio_read_v3r_iidr, vgic_mmio_write_wi, 4,
VGIC_ACCESS_32bit),
- REGISTER_DESC_WITH_LENGTH(GICR_TYPER,
- vgic_mmio_read_v3r_typer, vgic_mmio_write_wi, 8,
+ REGISTER_DESC_WITH_LENGTH_UACCESS(GICR_TYPER,
+ vgic_mmio_read_v3r_typer, vgic_mmio_write_wi,
+ vgic_uaccess_read_v3r_typer, vgic_mmio_uaccess_write_wi, 8,
VGIC_ACCESS_64bit | VGIC_ACCESS_32bit),
REGISTER_DESC_WITH_LENGTH(GICR_WAKER,
vgic_mmio_read_raz, vgic_mmio_write_wi, 4,
*/
static void debug_exception_enter(struct pt_regs *regs)
{
- /*
- * Tell lockdep we disabled irqs in entry.S. Do nothing if they were
- * already disabled to preserve the last enabled/disabled addresses.
- */
- if (interrupts_enabled(regs))
- trace_hardirqs_off();
-
- if (user_mode(regs)) {
- RCU_LOCKDEP_WARN(!rcu_is_watching(), "entry code didn't wake RCU");
- } else {
- /*
- * We might have interrupted pretty much anything. In
- * fact, if we're a debug exception, we can even interrupt
- * NMI processing. We don't want this code makes in_nmi()
- * to return true, but we need to notify RCU.
- */
- rcu_nmi_enter();
- }
-
preempt_disable();
/* This code is a bit fragile. Test it. */
static void debug_exception_exit(struct pt_regs *regs)
{
preempt_enable_no_resched();
-
- if (!user_mode(regs))
- rcu_nmi_exit();
-
- if (interrupts_enabled(regs))
- trace_hardirqs_on();
}
NOKPROBE_SYMBOL(debug_exception_exit);
free_empty_tables(start, end, PAGE_OFFSET, PAGE_END);
}
+static bool inside_linear_region(u64 start, u64 size)
+{
+ /*
+ * Linear mapping region is the range [PAGE_OFFSET..(PAGE_END - 1)]
+ * accommodating both its ends but excluding PAGE_END. Max physical
+ * range which can be mapped inside this linear mapping range, must
+ * also be derived from its end points.
+ */
+ return start >= __pa(_PAGE_OFFSET(vabits_actual)) &&
+ (start + size - 1) <= __pa(PAGE_END - 1);
+}
+
int arch_add_memory(int nid, u64 start, u64 size,
struct mhp_params *params)
{
int ret, flags = 0;
+ if (!inside_linear_region(start, size)) {
+ pr_err("[%llx %llx] is outside linear mapping region\n", start, start + size);
+ return -EINVAL;
+ }
+
if (rodata_full || debug_pagealloc_enabled())
flags = NO_BLOCK_MAPPINGS | NO_CONT_MAPPINGS;
}
void perf_get_regs_user(struct perf_regs *regs_user,
- struct pt_regs *regs,
- struct pt_regs *regs_user_copy)
+ struct pt_regs *regs)
{
regs_user->regs = task_pt_regs(current);
regs_user->abi = perf_reg_abi(current);
#ifdef CONFIG_CPU_PM_STOP
asm volatile("stop\n");
#endif
- local_irq_enable();
+ raw_local_irq_enable();
}
#endif
*/
void arch_cpu_idle(void)
{
- local_irq_enable();
+ raw_local_irq_enable();
__asm__("sleep");
}
{
__vmwait();
/* interrupts wake us up, but irqs are still disabled */
- local_irq_enable();
+ raw_local_irq_enable();
}
/*
#endif
#endif /* CONFIG_SPARSEMEM */
+
+#ifdef CONFIG_MEMORY_HOTPLUG
+int memory_add_physaddr_to_nid(u64 addr);
+#define memory_add_physaddr_to_nid memory_add_physaddr_to_nid
+#endif
+
#endif /* _ASM_IA64_SPARSEMEM_H */
if (mark_idle)
(*mark_idle)(1);
- safe_halt();
+ raw_safe_halt();
if (mark_idle)
(*mark_idle)(0);
void arch_cpu_idle(void)
{
- local_irq_enable();
+ raw_local_irq_enable();
}
{
struct clk_init_data id;
struct clk_hw *h;
+ struct clk *clk;
h = kzalloc(sizeof(*h), GFP_KERNEL);
if (!h)
id.ops = &alchemy_clkops_cpu;
h->init = &id;
- return clk_register(NULL, h);
+ clk = clk_register(NULL, h);
+ if (IS_ERR(clk)) {
+ pr_err("failed to register clock\n");
+ kfree(h);
+ }
+
+ return clk;
}
/* AUXPLLs ************************************************************/
CONFIG_DSCC4=m
CONFIG_DSCC4_PCISYNC=y
CONFIG_DSCC4_PCI_RST=y
-CONFIG_DLCI=m
CONFIG_LAPBETHER=m
# CONFIG_INPUT_KEYBOARD is not set
# CONFIG_INPUT_MOUSE is not set
CONFIG_DSCC4=m
CONFIG_DSCC4_PCISYNC=y
CONFIG_DSCC4_PCI_RST=y
-CONFIG_DLCI=m
CONFIG_LAPBETHER=m
# CONFIG_KEYBOARD_ATKBD is not set
CONFIG_KEYBOARD_GPIO=y
#if defined(CONFIG_XPA)
+#define MAX_POSSIBLE_PHYSMEM_BITS 40
#define pte_pfn(x) (((unsigned long)((x).pte_high >> _PFN_SHIFT)) | (unsigned long)((x).pte_low << _PAGE_PRESENT_SHIFT))
static inline pte_t
pfn_pte(unsigned long pfn, pgprot_t prot)
#elif defined(CONFIG_PHYS_ADDR_T_64BIT) && defined(CONFIG_CPU_MIPS32)
+#define MAX_POSSIBLE_PHYSMEM_BITS 36
#define pte_pfn(x) ((unsigned long)((x).pte_high >> 6))
static inline pte_t pfn_pte(unsigned long pfn, pgprot_t prot)
#else
+#define MAX_POSSIBLE_PHYSMEM_BITS 32
#ifdef CONFIG_CPU_VR41XX
#define pte_pfn(x) ((unsigned long)((x).pte >> (PAGE_SHIFT + 2)))
#define pfn_pte(pfn, prot) __pte(((pfn) << (PAGE_SHIFT + 2)) | pgprot_val(prot))
{
unsigned long cfg = read_c0_conf();
write_c0_conf(cfg | R30XX_CONF_HALT);
- local_irq_enable();
+ raw_local_irq_enable();
}
static void __cpuidle r39xx_wait(void)
{
if (!need_resched())
write_c0_conf(read_c0_conf() | TX39_CONF_HALT);
- local_irq_enable();
+ raw_local_irq_enable();
}
void __cpuidle r4k_wait(void)
{
- local_irq_enable();
+ raw_local_irq_enable();
__r4k_wait();
}
" .set arch=r4000 \n"
" wait \n"
" .set pop \n");
- local_irq_enable();
+ raw_local_irq_enable();
}
/*
" wait \n"
" mtc0 $1, $12 # stalls until W stage \n"
" .set pop \n");
- local_irq_enable();
+ raw_local_irq_enable();
}
/*
if (cpu_wait)
cpu_wait();
else
- local_irq_enable();
+ raw_local_irq_enable();
}
#ifdef CONFIG_CPU_IDLE
static void __init bootmem_init(void)
{
phys_addr_t ramstart, ramend;
- phys_addr_t start, end;
- u64 i;
+ unsigned long start, end;
+ int i;
ramstart = memblock_start_of_DRAM();
ramend = memblock_end_of_DRAM();
min_low_pfn = ARCH_PFN_OFFSET;
max_pfn = PFN_DOWN(ramend);
- for_each_mem_range(i, &start, &end) {
+ for_each_mem_pfn_range(i, MAX_NUMNODES, &start, &end, NULL) {
/*
* Skip highmem here so we get an accurate max_low_pfn if low
* memory stops short of high memory.
}
return mask == PM_HUGE_MASK;
}
+EXPORT_SYMBOL(has_transparent_hugepage);
#endif /* CONFIG_TRANSPARENT_HUGEPAGE */
void arch_cpu_idle(void)
{
- local_irq_enable();
+ raw_local_irq_enable();
}
/*
*/
void arch_cpu_idle(void)
{
- local_irq_enable();
+ raw_local_irq_enable();
if (mfspr(SPR_UPR) & SPR_UPR_PMP)
mtspr(SPR_PMR, mfspr(SPR_PMR) | SPR_PMR_DME);
}
void __cpuidle arch_cpu_idle(void)
{
- local_irq_enable();
+ raw_local_irq_enable();
/* nop on real hardware, qemu will idle sleep. */
asm volatile("or %%r10,%%r10,%%r10\n":::);
cpu-as-$(CONFIG_40x) += -Wa,-m405
cpu-as-$(CONFIG_44x) += -Wa,-m440
cpu-as-$(CONFIG_ALTIVEC) += $(call as-option,-Wa$(comma)-maltivec)
-cpu-as-$(CONFIG_E200) += -Wa,-me200
cpu-as-$(CONFIG_E500) += -Wa,-me500
# When using '-many -mpower4' gas will first try and find a matching power4
*/
#ifdef CONFIG_PTE_64BIT
#define PTE_RPN_MASK (~((1ULL << PTE_RPN_SHIFT) - 1))
+#define MAX_POSSIBLE_PHYSMEM_BITS 36
#else
#define PTE_RPN_MASK (~((1UL << PTE_RPN_SHIFT) - 1))
+#define MAX_POSSIBLE_PHYSMEM_BITS 32
#endif
/*
#endif
.endm
+#ifdef CONFIG_PPC_KUAP
.macro kuap_check_amr gpr1, gpr2
#ifdef CONFIG_PPC_KUAP_DEBUG
BEGIN_MMU_FTR_SECTION_NESTED(67)
END_MMU_FTR_SECTION_NESTED_IFSET(MMU_FTR_RADIX_KUAP, 67)
#endif
.endm
+#endif
.macro kuap_save_amr_and_lock gpr1, gpr2, use_cr, msr_pr_cr
#ifdef CONFIG_PPC_KUAP
#else /* !__ASSEMBLY__ */
+#include <linux/jump_label.h>
+
+DECLARE_STATIC_KEY_FALSE(uaccess_flush_key);
+
#ifdef CONFIG_PPC_KUAP
#include <asm/mmu.h>
static inline unsigned long get_kuap(void)
{
+ /*
+ * We return AMR_KUAP_BLOCKED when we don't support KUAP because
+ * prevent_user_access_return needs to return AMR_KUAP_BLOCKED to
+ * cause restore_user_access to do a flush.
+ *
+ * This has no effect in terms of actually blocking things on hash,
+ * so it doesn't break anything.
+ */
if (!early_mmu_has_feature(MMU_FTR_RADIX_KUAP))
- return 0;
+ return AMR_KUAP_BLOCKED;
return mfspr(SPRN_AMR);
}
isync();
}
+static inline bool
+bad_kuap_fault(struct pt_regs *regs, unsigned long address, bool is_write)
+{
+ return WARN(mmu_has_feature(MMU_FTR_RADIX_KUAP) &&
+ (regs->kuap & (is_write ? AMR_KUAP_BLOCK_WRITE : AMR_KUAP_BLOCK_READ)),
+ "Bug: %s fault blocked by AMR!", is_write ? "Write" : "Read");
+}
+#else /* CONFIG_PPC_KUAP */
+static inline void kuap_restore_amr(struct pt_regs *regs, unsigned long amr) { }
+
+static inline unsigned long kuap_get_and_check_amr(void)
+{
+ return 0UL;
+}
+
+static inline unsigned long get_kuap(void)
+{
+ return AMR_KUAP_BLOCKED;
+}
+
+static inline void set_kuap(unsigned long value) { }
+#endif /* !CONFIG_PPC_KUAP */
+
static __always_inline void allow_user_access(void __user *to, const void __user *from,
unsigned long size, unsigned long dir)
{
unsigned long size, unsigned long dir)
{
set_kuap(AMR_KUAP_BLOCKED);
+ if (static_branch_unlikely(&uaccess_flush_key))
+ do_uaccess_flush();
}
static inline unsigned long prevent_user_access_return(void)
unsigned long flags = get_kuap();
set_kuap(AMR_KUAP_BLOCKED);
+ if (static_branch_unlikely(&uaccess_flush_key))
+ do_uaccess_flush();
return flags;
}
static inline void restore_user_access(unsigned long flags)
{
set_kuap(flags);
+ if (static_branch_unlikely(&uaccess_flush_key) && flags == AMR_KUAP_BLOCKED)
+ do_uaccess_flush();
}
-
-static inline bool
-bad_kuap_fault(struct pt_regs *regs, unsigned long address, bool is_write)
-{
- return WARN(mmu_has_feature(MMU_FTR_RADIX_KUAP) &&
- (regs->kuap & (is_write ? AMR_KUAP_BLOCK_WRITE : AMR_KUAP_BLOCK_READ)),
- "Bug: %s fault blocked by AMR!", is_write ? "Write" : "Read");
-}
-#else /* CONFIG_PPC_KUAP */
-static inline void kuap_restore_amr(struct pt_regs *regs, unsigned long amr)
-{
-}
-
-static inline void kuap_check_amr(void)
-{
-}
-
-static inline unsigned long kuap_get_and_check_amr(void)
-{
- return 0;
-}
-#endif /* CONFIG_PPC_KUAP */
-
#endif /* __ASSEMBLY__ */
#endif /* _ASM_POWERPC_BOOK3S_64_KUP_RADIX_H */
nop; \
nop
+#define ENTRY_FLUSH_SLOT \
+ ENTRY_FLUSH_FIXUP_SECTION; \
+ nop; \
+ nop; \
+ nop;
+
/*
* r10 must be free to use, r13 must be paca
*/
#define INTERRUPT_TO_KERNEL \
- STF_ENTRY_BARRIER_SLOT
+ STF_ENTRY_BARRIER_SLOT; \
+ ENTRY_FLUSH_SLOT
/*
* Macros for annotating the expected destination of (h)rfid
RFSCV; \
b rfscv_flush_fallback
+#else /* __ASSEMBLY__ */
+/* Prototype for function defined in exceptions-64s.S */
+void do_uaccess_flush(void);
#endif /* __ASSEMBLY__ */
#endif /* _ASM_POWERPC_EXCEPTION_H */
FTR_ENTRY_OFFSET 955b-956b; \
.popsection;
+#define UACCESS_FLUSH_FIXUP_SECTION \
+959: \
+ .pushsection __uaccess_flush_fixup,"a"; \
+ .align 2; \
+960: \
+ FTR_ENTRY_OFFSET 959b-960b; \
+ .popsection;
+
+#define ENTRY_FLUSH_FIXUP_SECTION \
+957: \
+ .pushsection __entry_flush_fixup,"a"; \
+ .align 2; \
+958: \
+ FTR_ENTRY_OFFSET 957b-958b; \
+ .popsection;
+
#define RFI_FLUSH_FIXUP_SECTION \
951: \
.pushsection __rfi_flush_fixup,"a"; \
#include <linux/types.h>
extern long stf_barrier_fallback;
+extern long entry_flush_fallback;
extern long __start___stf_entry_barrier_fixup, __stop___stf_entry_barrier_fixup;
extern long __start___stf_exit_barrier_fixup, __stop___stf_exit_barrier_fixup;
+extern long __start___uaccess_flush_fixup, __stop___uaccess_flush_fixup;
+extern long __start___entry_flush_fixup, __stop___entry_flush_fixup;
extern long __start___rfi_flush_fixup, __stop___rfi_flush_fixup;
extern long __start___barrier_nospec_fixup, __stop___barrier_nospec_fixup;
extern long __start__btb_flush_fixup, __stop__btb_flush_fixup;
#define KUAP_CURRENT_WRITE 8
#define KUAP_CURRENT (KUAP_CURRENT_READ | KUAP_CURRENT_WRITE)
-#ifdef CONFIG_PPC64
+#ifdef CONFIG_PPC_BOOK3S_64
#include <asm/book3s/64/kup-radix.h>
#endif
#ifdef CONFIG_PPC_8xx
.macro kuap_check current, gpr
.endm
+.macro kuap_check_amr gpr1, gpr2
+.endm
+
#endif
#else /* !__ASSEMBLY__ */
void setup_kuap(bool disabled);
#else
static inline void setup_kuap(bool disabled) { }
+
+static inline bool
+bad_kuap_fault(struct pt_regs *regs, unsigned long address, bool is_write)
+{
+ return false;
+}
+
+static inline void kuap_check_amr(void) { }
+
+/*
+ * book3s/64/kup-radix.h defines these functions for the !KUAP case to flush
+ * the L1D cache after user accesses. Only include the empty stubs for other
+ * platforms.
+ */
+#ifndef CONFIG_PPC_BOOK3S_64
static inline void allow_user_access(void __user *to, const void __user *from,
unsigned long size, unsigned long dir) { }
static inline void prevent_user_access(void __user *to, const void __user *from,
unsigned long size, unsigned long dir) { }
static inline unsigned long prevent_user_access_return(void) { return 0UL; }
static inline void restore_user_access(unsigned long flags) { }
-static inline bool
-bad_kuap_fault(struct pt_regs *regs, unsigned long address, bool is_write)
-{
- return false;
-}
+#endif /* CONFIG_PPC_BOOK3S_64 */
#endif /* CONFIG_PPC_KUAP */
static inline void allow_read_from_user(const void __user *from, unsigned long size)
#define __HAVE_ARCH_RESERVED_KERNEL_PAGES
#endif
+#ifdef CONFIG_MEMORY_HOTPLUG
+extern int create_section_mapping(unsigned long start, unsigned long end,
+ int nid, pgprot_t prot);
+#endif
+
#endif /* __KERNEL__ */
#endif /* _ASM_MMZONE_H_ */
*/
#if defined(CONFIG_PPC32) && defined(CONFIG_PTE_64BIT)
#define PTE_RPN_MASK (~((1ULL << PTE_RPN_SHIFT) - 1))
+#define MAX_POSSIBLE_PHYSMEM_BITS 36
#else
#define PTE_RPN_MASK (~((1UL << PTE_RPN_SHIFT) - 1))
+#define MAX_POSSIBLE_PHYSMEM_BITS 32
#endif
/*
// Software required to flush link stack on context switch
#define SEC_FTR_FLUSH_LINK_STACK 0x0000000000001000ull
+// The L1-D cache should be flushed when entering the kernel
+#define SEC_FTR_L1D_FLUSH_ENTRY 0x0000000000004000ull
+
+// The L1-D cache should be flushed after user accesses from the kernel
+#define SEC_FTR_L1D_FLUSH_UACCESS 0x0000000000008000ull
// Features enabled by default
#define SEC_FTR_DEFAULT \
(SEC_FTR_L1D_FLUSH_HV | \
SEC_FTR_L1D_FLUSH_PR | \
SEC_FTR_BNDS_CHK_SPEC_BAR | \
+ SEC_FTR_L1D_FLUSH_ENTRY | \
+ SEC_FTR_L1D_FLUSH_UACCESS | \
SEC_FTR_FAVOUR_SECURITY)
#endif /* _ASM_POWERPC_SECURITY_FEATURES_H */
};
void setup_rfi_flush(enum l1d_flush_type, bool enable);
+void setup_entry_flush(bool enable);
+void setup_uaccess_flush(bool enable);
void do_rfi_flush_fixups(enum l1d_flush_type types);
#ifdef CONFIG_PPC_BARRIER_NOSPEC
void setup_barrier_nospec(void);
#else
static inline void setup_barrier_nospec(void) { };
#endif
+void do_uaccess_flush_fixups(enum l1d_flush_type types);
+void do_entry_flush_fixups(enum l1d_flush_type types);
void do_barrier_nospec_fixups(bool enable);
extern bool barrier_nospec_enabled;
#endif /* CONFIG_SPARSEMEM */
#ifdef CONFIG_MEMORY_HOTPLUG
-extern int create_section_mapping(unsigned long start, unsigned long end,
- int nid, pgprot_t prot);
extern int remove_section_mapping(unsigned long start, unsigned long end);
+extern int memory_add_physaddr_to_nid(u64 start);
+#define memory_add_physaddr_to_nid memory_add_physaddr_to_nid
#ifdef CONFIG_NUMA
extern int hot_add_scn_to_nid(unsigned long scn_addr);
}
#endif /* CONFIG_NUMA */
#endif /* CONFIG_MEMORY_HOTPLUG */
-
#endif /* __KERNEL__ */
#endif /* _ASM_POWERPC_SPARSEMEM_H */
* Vectors for the FWNMI option. Share common code.
*/
TRAMP_REAL_BEGIN(system_reset_fwnmi)
- /* XXX: fwnmi guest could run a nested/PR guest, so why no test? */
- __IKVM_REAL(system_reset)=0
GEN_INT_ENTRY system_reset, virt=0
#endif /* CONFIG_PPC_PSERIES */
* If none is found, do a Linux page fault. Linux page faults can happen in
* kernel mode due to user copy operations of course.
*
+ * KVM: The KVM HDSI handler may perform a load with MSR[DR]=1 in guest
+ * MMU context, which may cause a DSI in the host, which must go to the
+ * KVM handler. MSR[IR] is not enabled, so the real-mode handler will
+ * always be used regardless of AIL setting.
+ *
* - Radix MMU
* The hardware loads from the Linux page table directly, so a fault goes
* immediately to Linux page fault.
IVEC=0x300
IDAR=1
IDSISR=1
-#ifdef CONFIG_KVM_BOOK3S_PR_POSSIBLE
IKVM_SKIP=1
IKVM_REAL=1
-#endif
INT_DEFINE_END(data_access)
EXC_REAL_BEGIN(data_access, 0x300, 0x80)
* ppc64_bolted_size (first segment). The kernel handler must avoid stomping
* on user-handler data structures.
*
+ * KVM: Same as 0x300, DSLB must test for KVM guest.
+ *
* A dedicated save area EXSLB is used (XXX: but it actually need not be
* these days, we could use EXGEN).
*/
IAREA=PACA_EXSLB
IRECONCILE=0
IDAR=1
-#ifdef CONFIG_KVM_BOOK3S_PR_POSSIBLE
IKVM_SKIP=1
IKVM_REAL=1
-#endif
INT_DEFINE_END(data_access_slb)
EXC_REAL_BEGIN(data_access_slb, 0x380, 0x80)
.endr
blr
-TRAMP_REAL_BEGIN(rfi_flush_fallback)
- SET_SCRATCH0(r13);
- GET_PACA(r13);
- std r1,PACA_EXRFI+EX_R12(r13)
- ld r1,PACAKSAVE(r13)
- std r9,PACA_EXRFI+EX_R9(r13)
- std r10,PACA_EXRFI+EX_R10(r13)
- std r11,PACA_EXRFI+EX_R11(r13)
- mfctr r9
+/* Clobbers r10, r11, ctr */
+.macro L1D_DISPLACEMENT_FLUSH
ld r10,PACA_RFI_FLUSH_FALLBACK_AREA(r13)
ld r11,PACA_L1D_FLUSH_SIZE(r13)
srdi r11,r11,(7 + 3) /* 128 byte lines, unrolled 8x */
sync
/*
- * The load adresses are at staggered offsets within cachelines,
+ * The load addresses are at staggered offsets within cachelines,
* which suits some pipelines better (on others it should not
* hurt).
*/
ld r11,(0x80 + 8)*7(r10)
addi r10,r10,0x80*8
bdnz 1b
+.endm
+TRAMP_REAL_BEGIN(entry_flush_fallback)
+ std r9,PACA_EXRFI+EX_R9(r13)
+ std r10,PACA_EXRFI+EX_R10(r13)
+ std r11,PACA_EXRFI+EX_R11(r13)
+ mfctr r9
+ L1D_DISPLACEMENT_FLUSH
+ mtctr r9
+ ld r9,PACA_EXRFI+EX_R9(r13)
+ ld r10,PACA_EXRFI+EX_R10(r13)
+ ld r11,PACA_EXRFI+EX_R11(r13)
+ blr
+
+TRAMP_REAL_BEGIN(rfi_flush_fallback)
+ SET_SCRATCH0(r13);
+ GET_PACA(r13);
+ std r1,PACA_EXRFI+EX_R12(r13)
+ ld r1,PACAKSAVE(r13)
+ std r9,PACA_EXRFI+EX_R9(r13)
+ std r10,PACA_EXRFI+EX_R10(r13)
+ std r11,PACA_EXRFI+EX_R11(r13)
+ mfctr r9
+ L1D_DISPLACEMENT_FLUSH
mtctr r9
ld r9,PACA_EXRFI+EX_R9(r13)
ld r10,PACA_EXRFI+EX_R10(r13)
std r10,PACA_EXRFI+EX_R10(r13)
std r11,PACA_EXRFI+EX_R11(r13)
mfctr r9
- ld r10,PACA_RFI_FLUSH_FALLBACK_AREA(r13)
- ld r11,PACA_L1D_FLUSH_SIZE(r13)
- srdi r11,r11,(7 + 3) /* 128 byte lines, unrolled 8x */
- mtctr r11
- DCBT_BOOK3S_STOP_ALL_STREAM_IDS(r11) /* Stop prefetch streams */
-
- /* order ld/st prior to dcbt stop all streams with flushing */
- sync
-
- /*
- * The load adresses are at staggered offsets within cachelines,
- * which suits some pipelines better (on others it should not
- * hurt).
- */
-1:
- ld r11,(0x80 + 8)*0(r10)
- ld r11,(0x80 + 8)*1(r10)
- ld r11,(0x80 + 8)*2(r10)
- ld r11,(0x80 + 8)*3(r10)
- ld r11,(0x80 + 8)*4(r10)
- ld r11,(0x80 + 8)*5(r10)
- ld r11,(0x80 + 8)*6(r10)
- ld r11,(0x80 + 8)*7(r10)
- addi r10,r10,0x80*8
- bdnz 1b
-
+ L1D_DISPLACEMENT_FLUSH
mtctr r9
ld r9,PACA_EXRFI+EX_R9(r13)
ld r10,PACA_EXRFI+EX_R10(r13)
RFSCV
USE_TEXT_SECTION()
- MASKED_INTERRUPT
- MASKED_INTERRUPT hsrr=1
+
+_GLOBAL(do_uaccess_flush)
+ UACCESS_FLUSH_FIXUP_SECTION
+ nop
+ nop
+ nop
+ blr
+ L1D_DISPLACEMENT_FLUSH
+ blr
+_ASM_NOKPROBE_SYMBOL(do_uaccess_flush)
+EXPORT_SYMBOL(do_uaccess_flush)
+
+
+MASKED_INTERRUPT
+MASKED_INTERRUPT hsrr=1
#ifdef CONFIG_KVM_BOOK3S_64_HANDLER
kvmppc_skip_interrupt:
bl initial_bats
bl load_segment_registers
BEGIN_MMU_FTR_SECTION
+ bl reloc_offset
bl early_hash_table
END_MMU_FTR_SECTION_IFSET(MMU_FTR_HPTE_TABLE)
#if defined(CONFIG_BOOTX_TEXT)
ori r6, r6, 3 /* 256kB table */
mtspr SPRN_SDR1, r6
lis r6, early_hash@h
- lis r3, Hash@ha
+ addis r3, r3, Hash@ha
stw r6, Hash@l(r3)
blr
* interrupts enabled, some don't.
*/
if (irqs_disabled())
- local_irq_enable();
+ raw_local_irq_enable();
} else {
- local_irq_enable();
+ raw_local_irq_enable();
/*
* Go into low thread priority and possibly
* low power mode.
static enum l1d_flush_type enabled_flush_types;
static void *l1d_flush_fallback_area;
static bool no_rfi_flush;
+static bool no_entry_flush;
+static bool no_uaccess_flush;
bool rfi_flush;
+bool entry_flush;
+bool uaccess_flush;
+DEFINE_STATIC_KEY_FALSE(uaccess_flush_key);
+EXPORT_SYMBOL(uaccess_flush_key);
static int __init handle_no_rfi_flush(char *p)
{
}
early_param("no_rfi_flush", handle_no_rfi_flush);
+static int __init handle_no_entry_flush(char *p)
+{
+ pr_info("entry-flush: disabled on command line.");
+ no_entry_flush = true;
+ return 0;
+}
+early_param("no_entry_flush", handle_no_entry_flush);
+
+static int __init handle_no_uaccess_flush(char *p)
+{
+ pr_info("uaccess-flush: disabled on command line.");
+ no_uaccess_flush = true;
+ return 0;
+}
+early_param("no_uaccess_flush", handle_no_uaccess_flush);
+
/*
* The RFI flush is not KPTI, but because users will see doco that says to use
* nopti we hijack that option here to also disable the RFI flush.
rfi_flush = enable;
}
+void entry_flush_enable(bool enable)
+{
+ if (enable) {
+ do_entry_flush_fixups(enabled_flush_types);
+ on_each_cpu(do_nothing, NULL, 1);
+ } else {
+ do_entry_flush_fixups(L1D_FLUSH_NONE);
+ }
+
+ entry_flush = enable;
+}
+
+void uaccess_flush_enable(bool enable)
+{
+ if (enable) {
+ do_uaccess_flush_fixups(enabled_flush_types);
+ static_branch_enable(&uaccess_flush_key);
+ on_each_cpu(do_nothing, NULL, 1);
+ } else {
+ static_branch_disable(&uaccess_flush_key);
+ do_uaccess_flush_fixups(L1D_FLUSH_NONE);
+ }
+
+ uaccess_flush = enable;
+}
+
static void __ref init_fallback_flush(void)
{
u64 l1d_size, limit;
enabled_flush_types = types;
- if (!no_rfi_flush && !cpu_mitigations_off())
+ if (!cpu_mitigations_off() && !no_rfi_flush)
rfi_flush_enable(enable);
}
+void setup_entry_flush(bool enable)
+{
+ if (cpu_mitigations_off())
+ return;
+
+ if (!no_entry_flush)
+ entry_flush_enable(enable);
+}
+
+void setup_uaccess_flush(bool enable)
+{
+ if (cpu_mitigations_off())
+ return;
+
+ if (!no_uaccess_flush)
+ uaccess_flush_enable(enable);
+}
+
#ifdef CONFIG_DEBUG_FS
static int rfi_flush_set(void *data, u64 val)
{
DEFINE_SIMPLE_ATTRIBUTE(fops_rfi_flush, rfi_flush_get, rfi_flush_set, "%llu\n");
+static int entry_flush_set(void *data, u64 val)
+{
+ bool enable;
+
+ if (val == 1)
+ enable = true;
+ else if (val == 0)
+ enable = false;
+ else
+ return -EINVAL;
+
+ /* Only do anything if we're changing state */
+ if (enable != entry_flush)
+ entry_flush_enable(enable);
+
+ return 0;
+}
+
+static int entry_flush_get(void *data, u64 *val)
+{
+ *val = entry_flush ? 1 : 0;
+ return 0;
+}
+
+DEFINE_SIMPLE_ATTRIBUTE(fops_entry_flush, entry_flush_get, entry_flush_set, "%llu\n");
+
+static int uaccess_flush_set(void *data, u64 val)
+{
+ bool enable;
+
+ if (val == 1)
+ enable = true;
+ else if (val == 0)
+ enable = false;
+ else
+ return -EINVAL;
+
+ /* Only do anything if we're changing state */
+ if (enable != uaccess_flush)
+ uaccess_flush_enable(enable);
+
+ return 0;
+}
+
+static int uaccess_flush_get(void *data, u64 *val)
+{
+ *val = uaccess_flush ? 1 : 0;
+ return 0;
+}
+
+DEFINE_SIMPLE_ATTRIBUTE(fops_uaccess_flush, uaccess_flush_get, uaccess_flush_set, "%llu\n");
+
static __init int rfi_flush_debugfs_init(void)
{
debugfs_create_file("rfi_flush", 0600, powerpc_debugfs_root, NULL, &fops_rfi_flush);
+ debugfs_create_file("entry_flush", 0600, powerpc_debugfs_root, NULL, &fops_entry_flush);
+ debugfs_create_file("uaccess_flush", 0600, powerpc_debugfs_root, NULL, &fops_uaccess_flush);
return 0;
}
device_initcall(rfi_flush_debugfs_init);
#include <linux/err.h>
#include <asm/asm-prototypes.h>
-#include <asm/book3s/64/kup-radix.h>
+#include <asm/kup.h>
#include <asm/cputime.h>
#include <asm/hw_irq.h>
#include <asm/kprobes.h>
__stop___stf_entry_barrier_fixup = .;
}
+ . = ALIGN(8);
+ __uaccess_flush_fixup : AT(ADDR(__uaccess_flush_fixup) - LOAD_OFFSET) {
+ __start___uaccess_flush_fixup = .;
+ *(__uaccess_flush_fixup)
+ __stop___uaccess_flush_fixup = .;
+ }
+
+ . = ALIGN(8);
+ __entry_flush_fixup : AT(ADDR(__entry_flush_fixup) - LOAD_OFFSET) {
+ __start___entry_flush_fixup = .;
+ *(__entry_flush_fixup)
+ __stop___entry_flush_fixup = .;
+ }
+
. = ALIGN(8);
__stf_exit_barrier_fixup : AT(ADDR(__stf_exit_barrier_fixup) - LOAD_OFFSET) {
__start___stf_exit_barrier_fixup = .;
}
state = &sb->irq_state[src];
+
+ /* Some sanity checking */
+ if (!state->valid) {
+ pr_devel("%s: source %lx invalid !\n", __func__, irq);
+ return VM_FAULT_SIGBUS;
+ }
+
kvmppc_xive_select_irq(state, &hw_num, &xd);
arch_spin_lock(&sb->lock);
do_stf_exit_barrier_fixups(types);
}
+void do_uaccess_flush_fixups(enum l1d_flush_type types)
+{
+ unsigned int instrs[4], *dest;
+ long *start, *end;
+ int i;
+
+ start = PTRRELOC(&__start___uaccess_flush_fixup);
+ end = PTRRELOC(&__stop___uaccess_flush_fixup);
+
+ instrs[0] = 0x60000000; /* nop */
+ instrs[1] = 0x60000000; /* nop */
+ instrs[2] = 0x60000000; /* nop */
+ instrs[3] = 0x4e800020; /* blr */
+
+ i = 0;
+ if (types == L1D_FLUSH_FALLBACK) {
+ instrs[3] = 0x60000000; /* nop */
+ /* fallthrough to fallback flush */
+ }
+
+ if (types & L1D_FLUSH_ORI) {
+ instrs[i++] = 0x63ff0000; /* ori 31,31,0 speculation barrier */
+ instrs[i++] = 0x63de0000; /* ori 30,30,0 L1d flush*/
+ }
+
+ if (types & L1D_FLUSH_MTTRIG)
+ instrs[i++] = 0x7c12dba6; /* mtspr TRIG2,r0 (SPR #882) */
+
+ for (i = 0; start < end; start++, i++) {
+ dest = (void *)start + *start;
+
+ pr_devel("patching dest %lx\n", (unsigned long)dest);
+
+ patch_instruction((struct ppc_inst *)dest, ppc_inst(instrs[0]));
+
+ patch_instruction((struct ppc_inst *)(dest + 1), ppc_inst(instrs[1]));
+ patch_instruction((struct ppc_inst *)(dest + 2), ppc_inst(instrs[2]));
+ patch_instruction((struct ppc_inst *)(dest + 3), ppc_inst(instrs[3]));
+ }
+
+ printk(KERN_DEBUG "uaccess-flush: patched %d locations (%s flush)\n", i,
+ (types == L1D_FLUSH_NONE) ? "no" :
+ (types == L1D_FLUSH_FALLBACK) ? "fallback displacement" :
+ (types & L1D_FLUSH_ORI) ? (types & L1D_FLUSH_MTTRIG)
+ ? "ori+mttrig type"
+ : "ori type" :
+ (types & L1D_FLUSH_MTTRIG) ? "mttrig type"
+ : "unknown");
+}
+
+void do_entry_flush_fixups(enum l1d_flush_type types)
+{
+ unsigned int instrs[3], *dest;
+ long *start, *end;
+ int i;
+
+ start = PTRRELOC(&__start___entry_flush_fixup);
+ end = PTRRELOC(&__stop___entry_flush_fixup);
+
+ instrs[0] = 0x60000000; /* nop */
+ instrs[1] = 0x60000000; /* nop */
+ instrs[2] = 0x60000000; /* nop */
+
+ i = 0;
+ if (types == L1D_FLUSH_FALLBACK) {
+ instrs[i++] = 0x7d4802a6; /* mflr r10 */
+ instrs[i++] = 0x60000000; /* branch patched below */
+ instrs[i++] = 0x7d4803a6; /* mtlr r10 */
+ }
+
+ if (types & L1D_FLUSH_ORI) {
+ instrs[i++] = 0x63ff0000; /* ori 31,31,0 speculation barrier */
+ instrs[i++] = 0x63de0000; /* ori 30,30,0 L1d flush*/
+ }
+
+ if (types & L1D_FLUSH_MTTRIG)
+ instrs[i++] = 0x7c12dba6; /* mtspr TRIG2,r0 (SPR #882) */
+
+ for (i = 0; start < end; start++, i++) {
+ dest = (void *)start + *start;
+
+ pr_devel("patching dest %lx\n", (unsigned long)dest);
+
+ patch_instruction((struct ppc_inst *)dest, ppc_inst(instrs[0]));
+
+ if (types == L1D_FLUSH_FALLBACK)
+ patch_branch((struct ppc_inst *)(dest + 1), (unsigned long)&entry_flush_fallback,
+ BRANCH_SET_LINK);
+ else
+ patch_instruction((struct ppc_inst *)(dest + 1), ppc_inst(instrs[1]));
+
+ patch_instruction((struct ppc_inst *)(dest + 2), ppc_inst(instrs[2]));
+ }
+
+ printk(KERN_DEBUG "entry-flush: patched %d locations (%s flush)\n", i,
+ (types == L1D_FLUSH_NONE) ? "no" :
+ (types == L1D_FLUSH_FALLBACK) ? "fallback displacement" :
+ (types & L1D_FLUSH_ORI) ? (types & L1D_FLUSH_MTTRIG)
+ ? "ori+mttrig type"
+ : "ori type" :
+ (types & L1D_FLUSH_MTTRIG) ? "mttrig type"
+ : "unknown");
+}
+
void do_rfi_flush_fixups(enum l1d_flush_type types)
{
unsigned int instrs[3], *dest;
#include <asm/rtas.h>
#include <asm/kasan.h>
#include <asm/svm.h>
+#include <asm/mmzone.h>
#include <mm/mmu_decl.h>
/* If this is a valid record, create the sample */
struct perf_output_handle handle;
- if (perf_output_begin(&handle, event, header.size))
+ if (perf_output_begin(&handle, &data, event, header.size))
return;
perf_output_sample(&handle, &header, &data, event);
}
void perf_get_regs_user(struct perf_regs *regs_user,
- struct pt_regs *regs,
- struct pt_regs *regs_user_copy)
+ struct pt_regs *regs)
{
regs_user->regs = task_pt_regs(current);
regs_user->abi = (regs_user->regs) ? perf_reg_abi(current) :
security_ftr_clear(SEC_FTR_BNDS_CHK_SPEC_BAR);
}
-static void pnv_setup_rfi_flush(void)
+static void pnv_setup_security_mitigations(void)
{
struct device_node *np, *fw_features;
enum l1d_flush_type type;
type = L1D_FLUSH_ORI;
}
+ /*
+ * If we are non-Power9 bare metal, we don't need to flush on kernel
+ * entry or after user access: they fix a P9 specific vulnerability.
+ */
+ if (!pvr_version_is(PVR_POWER9)) {
+ security_ftr_clear(SEC_FTR_L1D_FLUSH_ENTRY);
+ security_ftr_clear(SEC_FTR_L1D_FLUSH_UACCESS);
+ }
+
enable = security_ftr_enabled(SEC_FTR_FAVOUR_SECURITY) && \
(security_ftr_enabled(SEC_FTR_L1D_FLUSH_PR) || \
security_ftr_enabled(SEC_FTR_L1D_FLUSH_HV));
setup_rfi_flush(type, enable);
setup_count_cache_flush();
+
+ enable = security_ftr_enabled(SEC_FTR_FAVOUR_SECURITY) &&
+ security_ftr_enabled(SEC_FTR_L1D_FLUSH_ENTRY);
+ setup_entry_flush(enable);
+
+ enable = security_ftr_enabled(SEC_FTR_FAVOUR_SECURITY) &&
+ security_ftr_enabled(SEC_FTR_L1D_FLUSH_UACCESS);
+ setup_uaccess_flush(enable);
+
+ setup_stf_barrier();
}
static void __init pnv_check_guarded_cores(void)
{
set_arch_panic_timeout(10, ARCH_PANIC_TIMEOUT);
- pnv_setup_rfi_flush();
- setup_stf_barrier();
+ pnv_setup_security_mitigations();
/* Initialize SMP */
pnv_smp_init();
cpus_read_unlock();
- /* Possibly switch to a new RFI flush type */
- pseries_setup_rfi_flush();
+ /* Possibly switch to a new L1 flush type */
+ pseries_setup_security_mitigations();
/* Reinitialise system information for hv-24x7 */
read_24x7_sys_info();
int dlpar_workqueue_init(void);
-void pseries_setup_rfi_flush(void);
+void pseries_setup_security_mitigations(void);
void pseries_lpar_read_hblkrm_characteristics(void);
#endif /* _PSERIES_PSERIES_H */
security_ftr_clear(SEC_FTR_BNDS_CHK_SPEC_BAR);
}
-void pseries_setup_rfi_flush(void)
+void pseries_setup_security_mitigations(void)
{
struct h_cpu_char_result result;
enum l1d_flush_type types;
setup_rfi_flush(types, enable);
setup_count_cache_flush();
+
+ enable = security_ftr_enabled(SEC_FTR_FAVOUR_SECURITY) &&
+ security_ftr_enabled(SEC_FTR_L1D_FLUSH_ENTRY);
+ setup_entry_flush(enable);
+
+ enable = security_ftr_enabled(SEC_FTR_FAVOUR_SECURITY) &&
+ security_ftr_enabled(SEC_FTR_L1D_FLUSH_UACCESS);
+ setup_uaccess_flush(enable);
+
+ setup_stf_barrier();
}
#ifdef CONFIG_PCI_IOV
fwnmi_init();
- pseries_setup_rfi_flush();
- setup_stf_barrier();
+ pseries_setup_security_mitigations();
pseries_lpar_read_hblkrm_characteristics();
/* By default, only probe PCI (can be overridden by rtas_pci) */
#define PGDIR_SIZE (_AC(1, UL) << PGDIR_SHIFT)
#define PGDIR_MASK (~(PGDIR_SIZE - 1))
+#define MAX_POSSIBLE_PHYSMEM_BITS 34
+
#endif /* _ASM_RISCV_PGTABLE_32_H */
#ifndef __ASSEMBLY__
+#include <asm/barrier.h>
+
static inline void cpu_relax(void)
{
#ifdef __riscv_muldiv
}
void perf_get_regs_user(struct perf_regs *regs_user,
- struct pt_regs *regs,
- struct pt_regs *regs_user_copy)
+ struct pt_regs *regs)
{
regs_user->regs = task_pt_regs(current);
regs_user->abi = perf_reg_abi(current);
void arch_cpu_idle(void)
{
wait_for_interrupt();
- local_irq_enable();
+ raw_local_irq_enable();
}
void show_regs(struct pt_regs *regs)
*cmdline_p = boot_command_line;
early_ioremap_setup();
+ jump_label_init();
parse_early_param();
efi_init();
$(obj)/vdso.so.dbg: $(src)/vdso.lds $(obj-vdso) FORCE
$(call if_changed,vdsold)
SYSCFLAGS_vdso.so.dbg = -shared -s -Wl,-soname=linux-vdso.so.1 \
- -Wl,--build-id -Wl,--hash-style=both
+ -Wl,--build-id=sha1 -Wl,--hash-style=both
# We also create a special relocatable object that should mirror the symbol
# table and layout of the linked DSO. With ld --just-symbols we can then
+CONFIG_UAPI_HEADER_TEST=y
CONFIG_SYSVIPC=y
CONFIG_POSIX_MQUEUE=y
CONFIG_WATCH_QUEUE=y
/* stack_frame offsets */
OFFSET(__SF_BACKCHAIN, stack_frame, back_chain);
OFFSET(__SF_GPRS, stack_frame, gprs);
- OFFSET(__SF_EMPTY, stack_frame, empty1);
- OFFSET(__SF_SIE_CONTROL, stack_frame, empty1[0]);
- OFFSET(__SF_SIE_SAVEAREA, stack_frame, empty1[1]);
- OFFSET(__SF_SIE_REASON, stack_frame, empty1[2]);
- OFFSET(__SF_SIE_FLAGS, stack_frame, empty1[3]);
+ OFFSET(__SF_EMPTY, stack_frame, empty1[0]);
+ OFFSET(__SF_SIE_CONTROL, stack_frame, empty1[1]);
+ OFFSET(__SF_SIE_SAVEAREA, stack_frame, empty1[2]);
+ OFFSET(__SF_SIE_REASON, stack_frame, empty1[3]);
+ OFFSET(__SF_SIE_FLAGS, stack_frame, empty1[4]);
BLANK();
OFFSET(__VDSO_GETCPU_VAL, vdso_per_cpu_data, getcpu_val);
BLANK();
#endif
LOCKDEP_SYS_EXIT
.Lsysc_tif:
+ DISABLE_INTS
TSTMSK __PT_FLAGS(%r11),_PIF_WORK
jnz .Lsysc_work
TSTMSK __TI_flags(%r12),_TIF_WORK
# One of the work bits is on. Find out which one.
#
.Lsysc_work:
+ ENABLE_INTS
TSTMSK __TI_flags(%r12),_TIF_NEED_RESCHED
jo .Lsysc_reschedule
TSTMSK __PT_FLAGS(%r11),_PIF_SYSCALL_RESTART
xc __PT_FLAGS(8,%r11),__PT_FLAGS(%r11)
TSTMSK __LC_CPU_FLAGS,_CIF_IGNORE_IRQ
jo .Lio_restore
-#if IS_ENABLED(CONFIG_TRACE_IRQFLAGS)
- tmhh %r8,0x300
- jz 1f
TRACE_IRQS_OFF
-1:
-#endif
xc __SF_BACKCHAIN(8,%r15),__SF_BACKCHAIN(%r15)
.Lio_loop:
lgr %r2,%r11 # pass pointer to pt_regs
TSTMSK __LC_CPU_FLAGS,_CIF_WORK
jnz .Lio_work
.Lio_restore:
-#if IS_ENABLED(CONFIG_TRACE_IRQFLAGS)
- tm __PT_PSW(%r11),3
- jno 0f
TRACE_IRQS_ON
-0:
-#endif
mvc __LC_RETURN_PSW(16),__PT_PSW(%r11)
tm __PT_PSW+1(%r11),0x01 # returning to user ?
jno .Lio_exit_kernel
xc __PT_FLAGS(8,%r11),__PT_FLAGS(%r11)
TSTMSK __LC_CPU_FLAGS,_CIF_IGNORE_IRQ
jo .Lio_restore
-#if IS_ENABLED(CONFIG_TRACE_IRQFLAGS)
- tmhh %r8,0x300
- jz 1f
TRACE_IRQS_OFF
-1:
-#endif
xc __SF_BACKCHAIN(8,%r15),__SF_BACKCHAIN(%r15)
lgr %r2,%r11 # pass pointer to pt_regs
lghi %r3,EXT_INTERRUPT
* %r4
*/
load_fpu_regs:
+ stnsm __SF_EMPTY(%r15),0xfc
lg %r4,__LC_CURRENT
aghi %r4,__TASK_thread
TSTMSK __LC_CPU_FLAGS,_CIF_FPU
.Lload_fpu_regs_done:
ni __LC_CPU_FLAGS+7,255-_CIF_FPU
.Lload_fpu_regs_exit:
+ ssm __SF_EMPTY(%r15)
BR_EX %r14
.Lload_fpu_regs_end:
ENDPROC(load_fpu_regs)
PSW_MASK_IO | PSW_MASK_EXT | PSW_MASK_MCHECK;
clear_cpu_flag(CIF_NOHZ_DELAY);
- local_irq_save(flags);
+ raw_local_irq_save(flags);
/* Call the assembler magic in entry.S */
psw_idle(idle, psw_mask);
- local_irq_restore(flags);
+ raw_local_irq_restore(flags);
/* Account time spent with enabled wait psw loaded as idle time. */
raw_write_seqcount_begin(&idle->seqcount);
void arch_cpu_idle(void)
{
enabled_wait();
- local_irq_enable();
+ raw_local_irq_enable();
}
void arch_cpu_idle_exit(void)
rcu_read_lock();
perf_prepare_sample(&header, data, event, regs);
- if (perf_output_begin(&handle, event, header.size))
+ if (perf_output_begin(&handle, data, event, header.size))
goto out;
/* Update the process ID (see also kernel/events/core.c) */
}
arch_initcall(init_cpum_sampling_pmu);
-core_param(cpum_sfb_size, CPUM_SF_MAX_SDB, sfb_size, 0640);
+core_param(cpum_sfb_size, CPUM_SF_MAX_SDB, sfb_size, 0644);
}
void perf_get_regs_user(struct perf_regs *regs_user,
- struct pt_regs *regs,
- struct pt_regs *regs_user_copy)
+ struct pt_regs *regs)
{
/*
* Use the regs from the first interruption and let
.paddr = paddr
};
- if (uv_call(0, (u64)&uvcb))
+ if (uv_call(0, (u64)&uvcb)) {
+ /*
+ * Older firmware uses 107/d as an indication of a non secure
+ * page. Let us emulate the newer variant (no-op).
+ */
+ if (uvcb.header.rc == 0x107 && uvcb.header.rrc == 0xd)
+ return 0;
return -EINVAL;
+ }
return 0;
}
struct kvm_s390_pv_unp unp = {};
r = -EINVAL;
- if (!kvm_s390_pv_is_protected(kvm))
+ if (!kvm_s390_pv_is_protected(kvm) || !mm_is_protected(kvm->mm))
break;
r = -EFAULT;
vcpu->arch.sie_block->pp = 0;
vcpu->arch.sie_block->fpf &= ~FPF_BPBC;
vcpu->arch.sie_block->todpr = 0;
- vcpu->arch.sie_block->cpnc = 0;
}
}
regs->etoken = 0;
regs->etoken_extension = 0;
- regs->diag318 = 0;
}
int kvm_arch_vcpu_ioctl_set_regs(struct kvm_vcpu *vcpu, struct kvm_regs *regs)
return -EIO;
}
kvm->arch.gmap->guest_handle = uvcb.guest_handle;
- atomic_set(&kvm->mm->context.is_protected, 1);
return 0;
}
*rrc = uvcb.header.rrc;
KVM_UV_EVENT(kvm, 3, "PROTVIRT VM SET PARMS: rc %x rrc %x",
*rc, *rrc);
+ if (!cc)
+ atomic_set(&kvm->mm->context.is_protected, 1);
return cc ? -EINVAL : 0;
}
static void __udelay_disabled(unsigned long long usecs)
{
- unsigned long cr0, cr0_new, psw_mask, flags;
+ unsigned long cr0, cr0_new, psw_mask;
struct s390_idle_data idle;
u64 end;
psw_mask = __extract_psw() | PSW_MASK_EXT | PSW_MASK_WAIT;
set_clock_comparator(end);
set_cpu_flag(CIF_IGNORE_IRQ);
- local_irq_save(flags);
psw_idle(&idle, psw_mask);
- local_irq_restore(flags);
+ trace_hardirqs_off();
clear_cpu_flag(CIF_IGNORE_IRQ);
set_clock_comparator(S390_lowcore.clock_comparator);
__ctl_load(cr0, 0, 0);
#include <linux/sched/mm.h>
void s390_reset_acc(struct mm_struct *mm)
{
+ if (!mm_is_protected(mm))
+ return;
/*
* we might be called during
* reset: we walk the pages and clear
{
struct msi_desc *entry = irq_get_msi_desc(data->irq);
struct msi_msg msg = entry->msg;
+ int cpu_addr = smp_cpu_get_cpu_address(cpumask_first(dest));
msg.address_lo &= 0xff0000ff;
- msg.address_lo |= (cpumask_first(dest) << 8);
+ msg.address_lo |= (cpu_addr << 8);
pci_write_msi_msg(data->irq, &msg);
return IRQ_SET_MASK_OK;
unsigned long bit;
struct msi_desc *msi;
struct msi_msg msg;
+ int cpu_addr;
int rc, irq;
zdev->aisb = -1UL;
handle_percpu_irq);
msg.data = hwirq - bit;
if (irq_delivery == DIRECTED) {
+ if (msi->affinity)
+ cpu = cpumask_first(&msi->affinity->mask);
+ else
+ cpu = 0;
+ cpu_addr = smp_cpu_get_cpu_address(cpu);
+
msg.address_lo = zdev->msi_addr & 0xff0000ff;
- msg.address_lo |= msi->affinity ?
- (cpumask_first(&msi->affinity->mask) << 8) : 0;
+ msg.address_lo |= (cpu_addr << 8);
+
for_each_possible_cpu(cpu) {
airq_iv_set_data(zpci_ibv[cpu], hwirq, irq);
}
void default_idle(void)
{
set_bl_bit();
- local_irq_enable();
+ raw_local_irq_enable();
/* Isn't this racy ? */
cpu_sleep();
clear_bl_bit();
register unsigned int address = (unsigned int)leon3_irqctrl_regs;
/* Interrupts need to be enabled to not hang the CPU */
- local_irq_enable();
+ raw_local_irq_enable();
__asm__ __volatile__ (
"wr %%g0, %%asr19\n"
static void pmc_leon_idle(void)
{
/* Interrupts need to be enabled to not hang the CPU */
- local_irq_enable();
+ raw_local_irq_enable();
/* For systems without power-down, this will be no-op */
__asm__ __volatile__ ("wr %g0, %asr19\n\t");
{
if (sparc_idle)
(*sparc_idle)();
- local_irq_enable();
+ raw_local_irq_enable();
}
/* XXX cli/sti -> local_irq_xxx here, check this works once SMP is fixed. */
{
if (tlb_type != hypervisor) {
touch_nmi_watchdog();
- local_irq_enable();
+ raw_local_irq_enable();
} else {
unsigned long pstate;
- local_irq_enable();
+ raw_local_irq_enable();
/* The sun4v sleeping code requires that we have PSTATE.IE cleared over
* the cpu sleep hypervisor call.
} while (0)
#ifdef CONFIG_3_LEVEL_PGTABLES
-#define __pmd_free_tlb(tlb,x, address) tlb_remove_page((tlb),virt_to_page(x))
+
+#define __pmd_free_tlb(tlb, pmd, address) \
+do { \
+ pgtable_pmd_page_dtor(virt_to_page(pmd)); \
+ tlb_remove_page((tlb),virt_to_page(pmd)); \
+} while (0) \
+
#endif
#endif
{
cpu_tasks[current_thread_info()->cpu].pid = os_getpid();
um_idle_sleep();
- local_irq_enable();
+ raw_local_irq_enable();
}
int __cant_sleep(void) {
u64 pebs_enabled = cpuc->pebs_enabled;
handled++;
- x86_pmu.drain_pebs(regs);
+ x86_pmu.drain_pebs(regs, &data);
status &= x86_pmu.intel_ctrl | GLOBAL_STATUS_TRACE_TOPAPMI;
/*
x86_add_quirk(intel_arch_events_quirk); /* Install first, so it runs last */
+ if (version >= 5) {
+ x86_pmu.intel_cap.anythread_deprecated = edx.split.anythread_deprecated;
+ if (x86_pmu.intel_cap.anythread_deprecated)
+ pr_cont(" AnyThread deprecated, ");
+ }
+
/*
* Install the hw-cache-events table:
*/
x86_pmu.intel_ctrl |=
((1LL << x86_pmu.num_counters_fixed)-1) << INTEL_PMC_IDX_FIXED;
+ /* AnyThread may be deprecated on arch perfmon v5 or later */
+ if (x86_pmu.intel_cap.anythread_deprecated)
+ x86_pmu.format_attrs = intel_arch_formats_attr;
+
if (x86_pmu.event_constraints) {
/*
* event on fixed counter2 (REF_CYCLES) only works on this
MODULE_LICENSE("GPL");
#define DEFINE_CSTATE_FORMAT_ATTR(_var, _name, _format) \
-static ssize_t __cstate_##_var##_show(struct kobject *kobj, \
- struct kobj_attribute *attr, \
+static ssize_t __cstate_##_var##_show(struct device *dev, \
+ struct device_attribute *attr, \
char *page) \
{ \
BUILD_BUG_ON(sizeof(_format) >= PAGE_SIZE); \
return sprintf(page, _format "\n"); \
} \
-static struct kobj_attribute format_attr_##_var = \
+static struct device_attribute format_attr_##_var = \
__ATTR(_name, 0444, __cstate_##_var##_show, NULL)
static ssize_t cstate_get_attr_cpumask(struct device *dev,
rcu_read_lock();
perf_prepare_sample(&header, &data, event, ®s);
- if (perf_output_begin(&handle, event, header.size *
- (top - base - skip)))
+ if (perf_output_begin(&handle, &data, event,
+ header.size * (top - base - skip)))
goto unlock;
for (at = base; at < top; at++) {
static inline void intel_pmu_drain_pebs_buffer(void)
{
- x86_pmu.drain_pebs(NULL);
+ struct perf_sample_data data;
+
+ x86_pmu.drain_pebs(NULL, &data);
}
/*
return 0;
}
-static void __intel_pmu_pebs_event(struct perf_event *event,
- struct pt_regs *iregs,
- void *base, void *top,
- int bit, int count,
- void (*setup_sample)(struct perf_event *,
- struct pt_regs *,
- void *,
- struct perf_sample_data *,
- struct pt_regs *))
+static __always_inline void
+__intel_pmu_pebs_event(struct perf_event *event,
+ struct pt_regs *iregs,
+ struct perf_sample_data *data,
+ void *base, void *top,
+ int bit, int count,
+ void (*setup_sample)(struct perf_event *,
+ struct pt_regs *,
+ void *,
+ struct perf_sample_data *,
+ struct pt_regs *))
{
struct cpu_hw_events *cpuc = this_cpu_ptr(&cpu_hw_events);
struct hw_perf_event *hwc = &event->hw;
- struct perf_sample_data data;
struct x86_perf_regs perf_regs;
struct pt_regs *regs = &perf_regs.regs;
void *at = get_next_pebs_record_by_bit(base, top, bit);
- struct pt_regs dummy_iregs;
+ static struct pt_regs dummy_iregs;
if (hwc->flags & PERF_X86_EVENT_AUTO_RELOAD) {
/*
iregs = &dummy_iregs;
while (count > 1) {
- setup_sample(event, iregs, at, &data, regs);
- perf_event_output(event, &data, regs);
+ setup_sample(event, iregs, at, data, regs);
+ perf_event_output(event, data, regs);
at += cpuc->pebs_record_size;
at = get_next_pebs_record_by_bit(at, top, bit);
count--;
}
- setup_sample(event, iregs, at, &data, regs);
+ setup_sample(event, iregs, at, data, regs);
if (iregs == &dummy_iregs) {
/*
* The PEBS records may be drained in the non-overflow context,
* last record the same as other PEBS records, and doesn't
* invoke the generic overflow handler.
*/
- perf_event_output(event, &data, regs);
+ perf_event_output(event, data, regs);
} else {
/*
* All but the last records are processed.
* The last one is left to be able to call the overflow handler.
*/
- if (perf_event_overflow(event, &data, regs))
+ if (perf_event_overflow(event, data, regs))
x86_pmu_stop(event, 0);
}
}
-static void intel_pmu_drain_pebs_core(struct pt_regs *iregs)
+static void intel_pmu_drain_pebs_core(struct pt_regs *iregs, struct perf_sample_data *data)
{
struct cpu_hw_events *cpuc = this_cpu_ptr(&cpu_hw_events);
struct debug_store *ds = cpuc->ds;
return;
}
- __intel_pmu_pebs_event(event, iregs, at, top, 0, n,
+ __intel_pmu_pebs_event(event, iregs, data, at, top, 0, n,
setup_pebs_fixed_sample_data);
}
}
}
-static void intel_pmu_drain_pebs_nhm(struct pt_regs *iregs)
+static void intel_pmu_drain_pebs_nhm(struct pt_regs *iregs, struct perf_sample_data *data)
{
struct cpu_hw_events *cpuc = this_cpu_ptr(&cpu_hw_events);
struct debug_store *ds = cpuc->ds;
}
if (counts[bit]) {
- __intel_pmu_pebs_event(event, iregs, base,
+ __intel_pmu_pebs_event(event, iregs, data, base,
top, bit, counts[bit],
setup_pebs_fixed_sample_data);
}
}
}
-static void intel_pmu_drain_pebs_icl(struct pt_regs *iregs)
+static void intel_pmu_drain_pebs_icl(struct pt_regs *iregs, struct perf_sample_data *data)
{
short counts[INTEL_PMC_IDX_FIXED + MAX_FIXED_PEBS_EVENTS] = {};
struct cpu_hw_events *cpuc = this_cpu_ptr(&cpu_hw_events);
if (WARN_ON_ONCE(!event->attr.precise_ip))
continue;
- __intel_pmu_pebs_event(event, iregs, base,
+ __intel_pmu_pebs_event(event, iregs, data, base,
top, bit, counts[bit],
setup_pebs_adaptive_sample_data);
}
return map;
}
-ssize_t uncore_event_show(struct kobject *kobj,
- struct kobj_attribute *attr, char *buf)
+ssize_t uncore_event_show(struct device *dev,
+ struct device_attribute *attr, char *buf)
{
struct uncore_event_desc *event =
container_of(attr, struct uncore_event_desc, attr);
#define UNCORE_BOX_FLAG_CFL8_CBOX_MSR_OFFS 2
struct uncore_event_desc {
- struct kobj_attribute attr;
+ struct device_attribute attr;
const char *config;
};
struct pci2phy_map *__find_pci2phy_map(int segment);
int uncore_pcibus_to_physid(struct pci_bus *bus);
-ssize_t uncore_event_show(struct kobject *kobj,
- struct kobj_attribute *attr, char *buf);
+ssize_t uncore_event_show(struct device *dev,
+ struct device_attribute *attr, char *buf);
static inline struct intel_uncore_pmu *dev_to_uncore_pmu(struct device *dev)
{
}
#define DEFINE_UNCORE_FORMAT_ATTR(_var, _name, _format) \
-static ssize_t __uncore_##_var##_show(struct kobject *kobj, \
- struct kobj_attribute *attr, \
+static ssize_t __uncore_##_var##_show(struct device *dev, \
+ struct device_attribute *attr, \
char *page) \
{ \
BUILD_BUG_ON(sizeof(_format) >= PAGE_SIZE); \
return sprintf(page, _format "\n"); \
} \
-static struct kobj_attribute format_attr_##_var = \
+static struct device_attribute format_attr_##_var = \
__ATTR(_name, 0444, __uncore_##_var##_show, NULL)
static inline bool uncore_pmc_fixed(int idx)
static struct freerunning_counters snb_uncore_imc_freerunning[] = {
[SNB_PCI_UNCORE_IMC_DATA_READS] = { SNB_UNCORE_PCI_IMC_DATA_READS_BASE,
0x0, 0x0, 1, 32 },
- [SNB_PCI_UNCORE_IMC_DATA_READS] = { SNB_UNCORE_PCI_IMC_DATA_WRITES_BASE,
+ [SNB_PCI_UNCORE_IMC_DATA_WRITES] = { SNB_UNCORE_PCI_IMC_DATA_WRITES_BASE,
0x0, 0x0, 1, 32 },
[SNB_PCI_UNCORE_IMC_GT_REQUESTS] = { SNB_UNCORE_PCI_IMC_GT_REQUESTS_BASE,
0x0, 0x0, 1, 32 },
u64 pebs_baseline:1;
u64 perf_metrics:1;
u64 pebs_output_pt_available:1;
+ u64 anythread_deprecated:1;
};
u64 capabilities;
};
int pebs_record_size;
int pebs_buffer_size;
int max_pebs_events;
- void (*drain_pebs)(struct pt_regs *regs);
+ void (*drain_pebs)(struct pt_regs *regs, struct perf_sample_data *data);
struct event_constraint *pebs_constraints;
void (*pebs_aliases)(struct perf_event *event);
unsigned long large_pebs_flags;
* any other bit is reserved
*/
#define RAPL_EVENT_MASK 0xFFULL
-
-#define DEFINE_RAPL_FORMAT_ATTR(_var, _name, _format) \
-static ssize_t __rapl_##_var##_show(struct kobject *kobj, \
- struct kobj_attribute *attr, \
- char *page) \
-{ \
- BUILD_BUG_ON(sizeof(_format) >= PAGE_SIZE); \
- return sprintf(page, _format "\n"); \
-} \
-static struct kobj_attribute format_attr_##_var = \
- __ATTR(_name, 0444, __rapl_##_var##_show, NULL)
-
#define RAPL_CNTR_WIDTH 32
#define RAPL_EVENT_ATTR_STR(_name, v, str) \
.attrs = attrs_empty,
};
-DEFINE_RAPL_FORMAT_ATTR(event, event, "config:0-7");
+PMU_FORMAT_ATTR(event, "config:0-7");
static struct attribute *rapl_formats_attr[] = {
&format_attr_event.attr,
NULL,
int cpuid_nent;
struct kvm_cpuid_entry2 *cpuid_entries;
+ unsigned long cr3_lm_rsvd_bits;
int maxphyaddr;
int max_tdp_level;
int kvm_set_spte_hva(struct kvm *kvm, unsigned long hva, pte_t pte);
int kvm_cpu_has_injectable_intr(struct kvm_vcpu *v);
int kvm_cpu_has_interrupt(struct kvm_vcpu *vcpu);
+int kvm_cpu_has_extint(struct kvm_vcpu *v);
int kvm_arch_interrupt_allowed(struct kvm_vcpu *vcpu);
int kvm_cpu_get_interrupt(struct kvm_vcpu *v);
void kvm_vcpu_reset(struct kvm_vcpu *vcpu, bool init_event);
static inline void __sti_mwait(unsigned long eax, unsigned long ecx)
{
- trace_hardirqs_on();
-
mds_idle_clear_cpu_buffers();
/* "mwait %eax, %ecx;" */
asm volatile("sti; .byte 0x0f, 0x01, 0xc9;"
struct {
unsigned int num_counters_fixed:5;
unsigned int bit_width_fixed:8;
- unsigned int reserved:19;
+ unsigned int reserved1:2;
+ unsigned int anythread_deprecated:1;
+ unsigned int reserved2:16;
} split;
unsigned int full;
};
#endif
#endif /* CONFIG_SPARSEMEM */
+
+#ifndef __ASSEMBLY__
+#ifdef CONFIG_NUMA_KEEP_MEMINFO
+extern int phys_to_target_node(phys_addr_t start);
+#define phys_to_target_node phys_to_target_node
+extern int memory_add_physaddr_to_nid(u64 start);
+#define memory_add_physaddr_to_nid memory_add_physaddr_to_nid
+#endif
+#endif /* __ASSEMBLY__ */
+
#endif /* _ASM_X86_SPARSEMEM_H */
#ifndef _ASM_X86_UV_UV_H
#define _ASM_X86_UV_UV_H
-#include <asm/tlbflush.h>
-
enum uv_system_type {UV_NONE, UV_LEGACY_APIC, UV_X2APIC};
-struct cpumask;
-struct mm_struct;
-struct flush_tlb_info;
-
#ifdef CONFIG_X86_UV
#include <linux/efi.h>
static inline int is_uv_hubbed(int uv) { return 0; }
static inline void uv_cpu_init(void) { }
static inline void uv_system_init(void) { }
-static inline const struct cpumask *
-uv_flush_tlb_others(const struct cpumask *cpumask,
- const struct flush_tlb_info *info)
-{ return cpumask; }
#endif /* X86_UV */
static int uv_node_id;
/* Unpack AT/OEM/TABLE ID's to be NULL terminated strings */
-static u8 uv_archtype[UV_AT_SIZE];
+static u8 uv_archtype[UV_AT_SIZE + 1];
static u8 oem_id[ACPI_OEM_ID_SIZE + 1];
static u8 oem_table_id[ACPI_OEM_TABLE_ID_SIZE + 1];
if (n > 0 && n < sizeof(uv_ate->archtype)) {
pr_info("UV: UVarchtype received from BIOS\n");
- uv_stringify(UV_AT_SIZE, uv_archtype, uv_ate->archtype);
+ uv_stringify(sizeof(uv_archtype), uv_archtype, uv_ate->archtype);
return 1;
}
return 0;
if (!early_get_arch_type())
/* If not use OEM ID for UVarchtype */
- uv_stringify(UV_AT_SIZE, uv_archtype, _oem_id);
+ uv_stringify(sizeof(uv_archtype), uv_archtype, oem_id);
/* Check if not hubbed */
if (strncmp(uv_archtype, "SGI", 3) != 0) {
if (boot_cpu_has(X86_FEATURE_IBPB)) {
setup_force_cpu_cap(X86_FEATURE_USE_IBPB);
+ spectre_v2_user_ibpb = mode;
switch (cmd) {
case SPECTRE_V2_USER_CMD_FORCE:
case SPECTRE_V2_USER_CMD_PRCTL_IBPB:
case SPECTRE_V2_USER_CMD_SECCOMP_IBPB:
static_branch_enable(&switch_mm_always_ibpb);
+ spectre_v2_user_ibpb = SPECTRE_V2_USER_STRICT;
break;
case SPECTRE_V2_USER_CMD_PRCTL:
case SPECTRE_V2_USER_CMD_AUTO:
pr_info("mitigation: Enabling %s Indirect Branch Prediction Barrier\n",
static_key_enabled(&switch_mm_always_ibpb) ?
"always-on" : "conditional");
-
- spectre_v2_user_ibpb = mode;
}
/*
* When there's any problem use only local no_way_out state.
*/
if (!lmce) {
- if (mce_end(order) < 0)
- no_way_out = worst >= MCE_PANIC_SEVERITY;
+ if (mce_end(order) < 0) {
+ if (!no_way_out)
+ no_way_out = worst >= MCE_PANIC_SEVERITY;
+ }
} else {
/*
* If there was a fatal machine check we should have
return find_matching_signature(mc, csig, cpf);
}
-/*
- * Given CPU signature and a microcode patch, this function finds if the
- * microcode patch has matching family and model with the CPU.
- *
- * %true - if there's a match
- * %false - otherwise
- */
-static bool microcode_matches(struct microcode_header_intel *mc_header,
- unsigned long sig)
-{
- unsigned long total_size = get_totalsize(mc_header);
- unsigned long data_size = get_datasize(mc_header);
- struct extended_sigtable *ext_header;
- unsigned int fam_ucode, model_ucode;
- struct extended_signature *ext_sig;
- unsigned int fam, model;
- int ext_sigcount, i;
-
- fam = x86_family(sig);
- model = x86_model(sig);
-
- fam_ucode = x86_family(mc_header->sig);
- model_ucode = x86_model(mc_header->sig);
-
- if (fam == fam_ucode && model == model_ucode)
- return true;
-
- /* Look for ext. headers: */
- if (total_size <= data_size + MC_HEADER_SIZE)
- return false;
-
- ext_header = (void *) mc_header + data_size + MC_HEADER_SIZE;
- ext_sig = (void *)ext_header + EXT_HEADER_SIZE;
- ext_sigcount = ext_header->count;
-
- for (i = 0; i < ext_sigcount; i++) {
- fam_ucode = x86_family(ext_sig->sig);
- model_ucode = x86_model(ext_sig->sig);
-
- if (fam == fam_ucode && model == model_ucode)
- return true;
-
- ext_sig++;
- }
- return false;
-}
-
static struct ucode_patch *memdup_patch(void *data, unsigned int size)
{
struct ucode_patch *p;
return p;
}
-static void save_microcode_patch(void *data, unsigned int size)
+static void save_microcode_patch(struct ucode_cpu_info *uci, void *data, unsigned int size)
{
struct microcode_header_intel *mc_hdr, *mc_saved_hdr;
struct ucode_patch *iter, *tmp, *p = NULL;
if (!p)
return;
+ if (!find_matching_signature(p->data, uci->cpu_sig.sig, uci->cpu_sig.pf))
+ return;
+
/*
* Save for early loading. On 32-bit, that needs to be a physical
* address as the APs are running from physical addresses, before
size -= mc_size;
- if (!microcode_matches(mc_header, uci->cpu_sig.sig)) {
+ if (!find_matching_signature(data, uci->cpu_sig.sig,
+ uci->cpu_sig.pf)) {
data += mc_size;
continue;
}
if (save) {
- save_microcode_patch(data, mc_size);
+ save_microcode_patch(uci, data, mc_size);
goto next;
}
* Save this microcode patch. It will be loaded early when a CPU is
* hot-added or resumes.
*/
-static void save_mc_for_early(u8 *mc, unsigned int size)
+static void save_mc_for_early(struct ucode_cpu_info *uci, u8 *mc, unsigned int size)
{
/* Synchronization during CPU hotplug. */
static DEFINE_MUTEX(x86_cpu_microcode_mutex);
mutex_lock(&x86_cpu_microcode_mutex);
- save_microcode_patch(mc, size);
+ save_microcode_patch(uci, mc, size);
show_saved_mc();
mutex_unlock(&x86_cpu_microcode_mutex);
* permanent memory. So it will be loaded early when a CPU is hot added
* or resumes.
*/
- save_mc_for_early(new_mc, new_mc_size);
+ save_mc_for_early(uci, new_mc, new_mc_size);
pr_debug("CPU%d found a matching microcode update with version 0x%x (current=0x%x)\n",
cpu, new_rev, uci->cpu_sig.rev);
return ret ?: nbytes;
}
+/**
+ * rdtgroup_remove - the helper to remove resource group safely
+ * @rdtgrp: resource group to remove
+ *
+ * On resource group creation via a mkdir, an extra kernfs_node reference is
+ * taken to ensure that the rdtgroup structure remains accessible for the
+ * rdtgroup_kn_unlock() calls where it is removed.
+ *
+ * Drop the extra reference here, then free the rdtgroup structure.
+ *
+ * Return: void
+ */
+static void rdtgroup_remove(struct rdtgroup *rdtgrp)
+{
+ kernfs_put(rdtgrp->kn);
+ kfree(rdtgrp);
+}
+
struct task_move_callback {
struct callback_head work;
struct rdtgroup *rdtgrp;
(rdtgrp->flags & RDT_DELETED)) {
current->closid = 0;
current->rmid = 0;
- kfree(rdtgrp);
+ rdtgroup_remove(rdtgrp);
}
if (unlikely(current->flags & PF_EXITING))
if (IS_ERR(kn_subdir))
return PTR_ERR(kn_subdir);
- kernfs_get(kn_subdir);
ret = rdtgroup_kn_set_ugid(kn_subdir);
if (ret)
return ret;
kn_info = kernfs_create_dir(parent_kn, "info", parent_kn->mode, NULL);
if (IS_ERR(kn_info))
return PTR_ERR(kn_info);
- kernfs_get(kn_info);
ret = rdtgroup_add_files(kn_info, RF_TOP_INFO);
if (ret)
goto out_destroy;
}
- /*
- * This extra ref will be put in kernfs_remove() and guarantees
- * that @rdtgrp->kn is always accessible.
- */
- kernfs_get(kn_info);
-
ret = rdtgroup_kn_set_ugid(kn_info);
if (ret)
goto out_destroy;
if (dest_kn)
*dest_kn = kn;
- /*
- * This extra ref will be put in kernfs_remove() and guarantees
- * that @rdtgrp->kn is always accessible.
- */
- kernfs_get(kn);
-
ret = rdtgroup_kn_set_ugid(kn);
if (ret)
goto out_destroy;
rdtgrp->mode == RDT_MODE_PSEUDO_LOCKED)
rdtgroup_pseudo_lock_remove(rdtgrp);
kernfs_unbreak_active_protection(kn);
- kernfs_put(rdtgrp->kn);
- kfree(rdtgrp);
+ rdtgroup_remove(rdtgrp);
} else {
kernfs_unbreak_active_protection(kn);
}
&kn_mongrp);
if (ret < 0)
goto out_info;
- kernfs_get(kn_mongrp);
ret = mkdir_mondata_all(rdtgroup_default.kn,
&rdtgroup_default, &kn_mondata);
if (ret < 0)
goto out_mongrp;
- kernfs_get(kn_mondata);
rdtgroup_default.mon.mon_data_kn = kn_mondata;
}
if (atomic_read(&sentry->waitcount) != 0)
sentry->flags = RDT_DELETED;
else
- kfree(sentry);
+ rdtgroup_remove(sentry);
}
}
if (atomic_read(&rdtgrp->waitcount) != 0)
rdtgrp->flags = RDT_DELETED;
else
- kfree(rdtgrp);
+ rdtgroup_remove(rdtgrp);
}
/* Notify online CPUs to update per cpu storage and PQR_ASSOC MSR */
update_closid_rmid(cpu_online_mask, &rdtgroup_default);
if (IS_ERR(kn))
return PTR_ERR(kn);
- /*
- * This extra ref will be put in kernfs_remove() and guarantees
- * that kn is always accessible.
- */
- kernfs_get(kn);
ret = rdtgroup_kn_set_ugid(kn);
if (ret)
goto out_destroy;
/*
* kernfs_remove() will drop the reference count on "kn" which
* will free it. But we still need it to stick around for the
- * rdtgroup_kn_unlock(kn} call below. Take one extra reference
- * here, which will be dropped inside rdtgroup_kn_unlock().
+ * rdtgroup_kn_unlock(kn) call. Take one extra reference here,
+ * which will be dropped by kernfs_put() in rdtgroup_remove().
*/
kernfs_get(kn);
out_idfree:
free_rmid(rdtgrp->mon.rmid);
out_destroy:
+ kernfs_put(rdtgrp->kn);
kernfs_remove(rdtgrp->kn);
out_free_rgrp:
kfree(rdtgrp);
{
kernfs_remove(rgrp->kn);
free_rmid(rgrp->mon.rmid);
- kfree(rgrp);
+ rdtgroup_remove(rgrp);
}
/*
WARN_ON(list_empty(&prdtgrp->mon.crdtgrp_list));
list_del(&rdtgrp->mon.crdtgrp_list);
- /*
- * one extra hold on this, will drop when we kfree(rdtgrp)
- * in rdtgroup_kn_unlock()
- */
- kernfs_get(kn);
kernfs_remove(rdtgrp->kn);
return 0;
rdtgrp->flags = RDT_DELETED;
list_del(&rdtgrp->rdtgroup_list);
- /*
- * one extra hold on this, will drop when we kfree(rdtgrp)
- * in rdtgroup_kn_unlock()
- */
- kernfs_get(kn);
kernfs_remove(rdtgrp->kn);
return 0;
}
if (!user_mode(regs))
return copy_from_kernel_nofault(buf, (u8 *)src, nbytes);
+ /* The user space code from other tasks cannot be accessed. */
+ if (regs != task_pt_regs(current))
+ return -EPERM;
/*
* Make sure userspace isn't trying to trick us into dumping kernel
* memory by pointing the userspace instruction pointer at it.
if (__chk_range_not_ok(src, nbytes, TASK_SIZE_MAX))
return -EINVAL;
+ /*
+ * Even if named copy_from_user_nmi() this can be invoked from
+ * other contexts and will not try to resolve a pagefault, which is
+ * the correct thing to do here as this code can be called from any
+ * context.
+ */
return copy_from_user_nmi(buf, (void __user *)src, nbytes);
}
u8 opcodes[OPCODE_BUFSIZE];
unsigned long prologue = regs->ip - PROLOGUE_SIZE;
- if (copy_code(regs, opcodes, prologue, sizeof(opcodes))) {
- printk("%sCode: Unable to access opcode bytes at RIP 0x%lx.\n",
- loglvl, prologue);
- } else {
+ switch (copy_code(regs, opcodes, prologue, sizeof(opcodes))) {
+ case 0:
printk("%sCode: %" __stringify(PROLOGUE_SIZE) "ph <%02x> %"
__stringify(EPILOGUE_SIZE) "ph\n", loglvl, opcodes,
opcodes[PROLOGUE_SIZE], opcodes + PROLOGUE_SIZE + 1);
+ break;
+ case -EPERM:
+ /* No access to the user space stack of other tasks. Ignore. */
+ break;
+ default:
+ printk("%sCode: Unable to access opcode bytes at RIP 0x%lx.\n",
+ loglvl, prologue);
+ break;
}
}
}
void perf_get_regs_user(struct perf_regs *regs_user,
- struct pt_regs *regs,
- struct pt_regs *regs_user_copy)
+ struct pt_regs *regs)
{
regs_user->regs = task_pt_regs(current);
regs_user->abi = perf_reg_abi(current);
return PERF_SAMPLE_REGS_ABI_64;
}
+static DEFINE_PER_CPU(struct pt_regs, nmi_user_regs);
+
void perf_get_regs_user(struct perf_regs *regs_user,
- struct pt_regs *regs,
- struct pt_regs *regs_user_copy)
+ struct pt_regs *regs)
{
+ struct pt_regs *regs_user_copy = this_cpu_ptr(&nmi_user_regs);
struct pt_regs *user_regs = task_pt_regs(current);
+ if (!in_nmi()) {
+ regs_user->regs = user_regs;
+ regs_user->abi = perf_reg_abi(current);
+ return;
+ }
+
/*
* If we're in an NMI that interrupted task_pt_regs setup, then
* we can't sample user regs at all. This check isn't really
*/
void __cpuidle default_idle(void)
{
- safe_halt();
+ raw_safe_halt();
}
#if defined(CONFIG_APM_MODULE) || defined(CONFIG_HALTPOLL_CPUIDLE_MODULE)
EXPORT_SYMBOL(default_idle);
/*
* AMD Erratum 400 aware idle routine. We handle it the same way as C3 power
* states (local apic timer and TSC stop).
+ *
+ * XXX this function is completely buggered vs RCU and tracing.
*/
static void amd_e400_idle(void)
{
* The switch back from broadcast mode needs to be called with
* interrupts disabled.
*/
- local_irq_disable();
+ raw_local_irq_disable();
tick_broadcast_exit();
- local_irq_enable();
+ raw_local_irq_enable();
}
/*
if (!need_resched())
__sti_mwait(0, 0);
else
- local_irq_enable();
+ raw_local_irq_enable();
} else {
- local_irq_enable();
+ raw_local_irq_enable();
}
__current_clr_polling();
}
if (!tboot_enabled())
return 0;
- if (intel_iommu_tboot_noforce)
- return 1;
-
- if (no_iommu || swiotlb || dmar_disabled)
+ if (no_iommu || dmar_disabled)
pr_warn("Forcing Intel-IOMMU to enabled\n");
dmar_disabled = 0;
-#ifdef CONFIG_SWIOTLB
- swiotlb = 0;
-#endif
no_iommu = 0;
return 1;
vcpu->arch.cr4_guest_rsvd_bits =
__cr4_reserved_bits(guest_cpuid_has, vcpu);
+ vcpu->arch.cr3_lm_rsvd_bits = rsvd_bits(cpuid_maxphyaddr(vcpu), 63);
+
/* Invoke the vendor callback only after the above state is updated. */
kvm_x86_ops.vcpu_after_set_cpuid(vcpu);
}
edx.split.num_counters_fixed = min(cap.num_counters_fixed, MAX_FIXED_COUNTERS);
edx.split.bit_width_fixed = cap.bit_width_fixed;
- edx.split.reserved = 0;
+ edx.split.anythread_deprecated = 1;
+ edx.split.reserved1 = 0;
+ edx.split.reserved2 = 0;
entry->eax = eax.full;
entry->ebx = cap.events_mask;
return X86EMUL_CONTINUE;
}
+static int em_clflushopt(struct x86_emulate_ctxt *ctxt)
+{
+ /* emulating clflushopt regardless of cpuid */
+ return X86EMUL_CONTINUE;
+}
+
static int em_movsxd(struct x86_emulate_ctxt *ctxt)
{
ctxt->dst.val = (s32) ctxt->src.val;
};
static const struct gprefix pfx_0f_ae_7 = {
- I(SrcMem | ByteOp, em_clflush), N, N, N,
+ I(SrcMem | ByteOp, em_clflush), I(SrcMem | ByteOp, em_clflushopt), N, N,
};
static const struct group_dual group15 = { {
* check if there is pending interrupt from
* non-APIC source without intack.
*/
-static int kvm_cpu_has_extint(struct kvm_vcpu *v)
-{
- u8 accept = kvm_apic_accept_pic_intr(v);
-
- if (accept) {
- if (irqchip_split(v->kvm))
- return pending_userspace_extint(v);
- else
- return v->kvm->arch.vpic->output;
- } else
- return 0;
-}
-
-/*
- * check if there is injectable interrupt:
- * when virtual interrupt delivery enabled,
- * interrupt from apic will handled by hardware,
- * we don't need to check it here.
- */
-int kvm_cpu_has_injectable_intr(struct kvm_vcpu *v)
+int kvm_cpu_has_extint(struct kvm_vcpu *v)
{
/*
- * FIXME: interrupt.injected represents an interrupt that it's
+ * FIXME: interrupt.injected represents an interrupt whose
* side-effects have already been applied (e.g. bit from IRR
* already moved to ISR). Therefore, it is incorrect to rely
* on interrupt.injected to know if there is a pending
if (!lapic_in_kernel(v))
return v->arch.interrupt.injected;
+ if (!kvm_apic_accept_pic_intr(v))
+ return 0;
+
+ if (irqchip_split(v->kvm))
+ return pending_userspace_extint(v);
+ else
+ return v->kvm->arch.vpic->output;
+}
+
+/*
+ * check if there is injectable interrupt:
+ * when virtual interrupt delivery enabled,
+ * interrupt from apic will handled by hardware,
+ * we don't need to check it here.
+ */
+int kvm_cpu_has_injectable_intr(struct kvm_vcpu *v)
+{
if (kvm_cpu_has_extint(v))
return 1;
*/
int kvm_cpu_has_interrupt(struct kvm_vcpu *v)
{
- /*
- * FIXME: interrupt.injected represents an interrupt that it's
- * side-effects have already been applied (e.g. bit from IRR
- * already moved to ISR). Therefore, it is incorrect to rely
- * on interrupt.injected to know if there is a pending
- * interrupt in the user-mode LAPIC.
- * This leads to nVMX/nSVM not be able to distinguish
- * if it should exit from L2 to L1 on EXTERNAL_INTERRUPT on
- * pending interrupt or should re-inject an injected
- * interrupt.
- */
- if (!lapic_in_kernel(v))
- return v->arch.interrupt.injected;
-
if (kvm_cpu_has_extint(v))
return 1;
*/
static int kvm_cpu_get_extint(struct kvm_vcpu *v)
{
- if (kvm_cpu_has_extint(v)) {
- if (irqchip_split(v->kvm)) {
- int vector = v->arch.pending_external_vector;
-
- v->arch.pending_external_vector = -1;
- return vector;
- } else
- return kvm_pic_read_irq(v->kvm); /* PIC */
- } else
+ if (!kvm_cpu_has_extint(v)) {
+ WARN_ON(!lapic_in_kernel(v));
return -1;
+ }
+
+ if (!lapic_in_kernel(v))
+ return v->arch.interrupt.nr;
+
+ if (irqchip_split(v->kvm)) {
+ int vector = v->arch.pending_external_vector;
+
+ v->arch.pending_external_vector = -1;
+ return vector;
+ } else
+ return kvm_pic_read_irq(v->kvm); /* PIC */
}
/*
*/
int kvm_cpu_get_interrupt(struct kvm_vcpu *v)
{
- int vector;
-
- if (!lapic_in_kernel(v))
- return v->arch.interrupt.nr;
-
- vector = kvm_cpu_get_extint(v);
-
+ int vector = kvm_cpu_get_extint(v);
if (vector != -1)
return vector; /* PIC */
struct kvm_lapic *apic = vcpu->arch.apic;
u32 ppr;
- if (!kvm_apic_hw_enabled(apic))
+ if (!kvm_apic_present(vcpu))
return -1;
__apic_update_ppr(apic, &ppr);
{
u64 sptes[PT64_ROOT_MAX_LEVEL];
struct rsvd_bits_validate *rsvd_check;
- int root = vcpu->arch.mmu->root_level;
+ int root = vcpu->arch.mmu->shadow_root_level;
int leaf;
int level;
bool reserved = false;
{
struct kvm_mmu_page *sp;
+ if (!kvm->arch.tdp_mmu_enabled)
+ return false;
+ if (WARN_ON(!VALID_PAGE(hpa)))
+ return false;
+
sp = to_shadow_page(hpa);
+ if (WARN_ON(!sp))
+ return false;
return sp->tdp_mmu_page && sp->root_count;
}
* Its safe to read more than we are asked, caller should ensure that
* destination has enough space.
*/
- src_paddr = round_down(src_paddr, 16);
offset = src_paddr & 15;
+ src_paddr = round_down(src_paddr, 16);
sz = round_up(sz + offset, 16);
return __sev_issue_dbg_cmd(kvm, src_paddr, dst_paddr, sz, err, false);
svm->avic_is_running = true;
svm->msrpm = svm_vcpu_alloc_msrpm();
- if (!svm->msrpm)
+ if (!svm->msrpm) {
+ err = -ENOMEM;
goto error_free_vmcb_page;
+ }
svm_vcpu_init_msrpm(vcpu, svm->msrpm);
static void svm_vcpu_after_set_cpuid(struct kvm_vcpu *vcpu)
{
struct vcpu_svm *svm = to_svm(vcpu);
+ struct kvm_cpuid_entry2 *best;
vcpu->arch.xsaves_enabled = guest_cpuid_has(vcpu, X86_FEATURE_XSAVE) &&
boot_cpu_has(X86_FEATURE_XSAVE) &&
/* Check again if INVPCID interception if required */
svm_check_invpcid(svm);
+ /* For sev guests, the memory encryption bit is not reserved in CR3. */
+ if (sev_guest(vcpu->kvm)) {
+ best = kvm_find_cpuid_entry(vcpu, 0x8000001F, 0);
+ if (best)
+ vcpu->arch.cr3_lm_rsvd_bits &= ~(1UL << (best->ebx & 0x3f));
+ }
+
if (!kvm_vcpu_apicv_active(vcpu))
return;
}
if (is_long_mode(vcpu) &&
- (cr3 & rsvd_bits(cpuid_maxphyaddr(vcpu), 63)))
+ (cr3 & vcpu->arch.cr3_lm_rsvd_bits))
return 1;
else if (is_pae_paging(vcpu) &&
!load_pdptrs(vcpu, vcpu->arch.walk_mmu, cr3))
static int kvm_cpu_accept_dm_intr(struct kvm_vcpu *vcpu)
{
+ /*
+ * We can accept userspace's request for interrupt injection
+ * as long as we have a place to store the interrupt number.
+ * The actual injection will happen when the CPU is able to
+ * deliver the interrupt.
+ */
+ if (kvm_cpu_has_extint(vcpu))
+ return false;
+
+ /* Acknowledging ExtINT does not happen if LINT0 is masked. */
return (!lapic_in_kernel(vcpu) ||
kvm_apic_accept_pic_intr(vcpu));
}
-/*
- * if userspace requested an interrupt window, check that the
- * interrupt window is open.
- *
- * No need to exit to userspace if we already have an interrupt queued.
- */
static int kvm_vcpu_ready_for_interrupt_injection(struct kvm_vcpu *vcpu)
{
return kvm_arch_interrupt_allowed(vcpu) &&
- !kvm_cpu_has_interrupt(vcpu) &&
- !kvm_event_needs_reinjection(vcpu) &&
kvm_cpu_accept_dm_intr(vcpu);
}
return meminfo_to_nid(&numa_reserved_meminfo, start);
}
+EXPORT_SYMBOL_GPL(phys_to_target_node);
int memory_add_physaddr_to_nid(u64 start)
{
nid = numa_meminfo.blk[0].nid;
return nid;
}
+EXPORT_SYMBOL_GPL(memory_add_physaddr_to_nid);
#endif
gfp_mask = GFP_KERNEL | __GFP_ZERO;
efi_pgd = (pgd_t *)__get_free_pages(gfp_mask, PGD_ALLOCATION_ORDER);
if (!efi_pgd)
- return -ENOMEM;
+ goto fail;
pgd = efi_pgd + pgd_index(EFI_VA_END);
p4d = p4d_alloc(&init_mm, pgd, EFI_VA_END);
- if (!p4d) {
- free_page((unsigned long)efi_pgd);
- return -ENOMEM;
- }
+ if (!p4d)
+ goto free_pgd;
pud = pud_alloc(&init_mm, p4d, EFI_VA_END);
- if (!pud) {
- if (pgtable_l5_enabled())
- free_page((unsigned long) pgd_page_vaddr(*pgd));
- free_pages((unsigned long)efi_pgd, PGD_ALLOCATION_ORDER);
- return -ENOMEM;
- }
+ if (!pud)
+ goto free_p4d;
efi_mm.pgd = efi_pgd;
mm_init_cpumask(&efi_mm);
init_new_context(NULL, &efi_mm);
return 0;
+
+free_p4d:
+ if (pgtable_l5_enabled())
+ free_page((unsigned long)pgd_page_vaddr(*pgd));
+free_pgd:
+ free_pages((unsigned long)efi_pgd, PGD_ALLOCATION_ORDER);
+fail:
+ return -ENOMEM;
}
/*
void xen_uninit_lock_cpu(int cpu)
{
+ int irq;
+
if (!xen_pvspin)
return;
- unbind_from_irqhandler(per_cpu(lock_kicker_irq, cpu), NULL);
+ /*
+ * When booting the kernel with 'mitigations=auto,nosmt', the secondary
+ * CPUs are not activated, and lock_kicker_irq is not initialized.
+ */
+ irq = per_cpu(lock_kicker_irq, cpu);
+ if (irq == -1)
+ return;
+
+ unbind_from_irqhandler(irq, NULL);
per_cpu(lock_kicker_irq, cpu) = -1;
kfree(per_cpu(irq_name, cpu));
per_cpu(irq_name, cpu) = NULL;
*/
#define VMALLOC_START (XCHAL_KSEG_CACHED_VADDR - 0x10000000)
#define VMALLOC_END (VMALLOC_START + 0x07FEFFFF)
-#define TLBTEMP_BASE_1 (VMALLOC_END + 1)
+#define TLBTEMP_BASE_1 (VMALLOC_START + 0x08000000)
#define TLBTEMP_BASE_2 (TLBTEMP_BASE_1 + DCACHE_WAY_SIZE)
#if 2 * DCACHE_WAY_SIZE > ICACHE_WAY_SIZE
#define TLBTEMP_SIZE (2 * DCACHE_WAY_SIZE)
return -EFAULT;
}
#else
-long strncpy_from_user(char *dst, const char *src, long count);
+long strncpy_from_user(char *dst, const char __user *src, long count);
#endif
/*
kvaddr = TLBTEMP_BASE_1 +
(page_to_phys(page) & DCACHE_ALIAS_MASK);
+ preempt_disable();
__invalidate_dcache_page_alias(kvaddr,
page_to_phys(page));
+ preempt_enable();
}
}
}
if (!alias && !mapping)
return;
+ preempt_disable();
virt = TLBTEMP_BASE_1 + (phys & DCACHE_ALIAS_MASK);
__flush_invalidate_dcache_page_alias(virt, phys);
if (mapping)
__invalidate_icache_page_alias(virt, phys);
+ preempt_enable();
}
/* There shouldn't be an entry in the cache for this page anymore. */
unsigned long phys = page_to_phys(pfn_to_page(pfn));
unsigned long virt = TLBTEMP_BASE_1 + (address & DCACHE_ALIAS_MASK);
+ preempt_disable();
__flush_invalidate_dcache_page_alias(virt, phys);
__invalidate_icache_page_alias(virt, phys);
+ preempt_enable();
}
EXPORT_SYMBOL(local_flush_cache_page);
unsigned long phys = page_to_phys(page);
unsigned long tmp;
+ preempt_disable();
tmp = TLBTEMP_BASE_1 + (phys & DCACHE_ALIAS_MASK);
__flush_invalidate_dcache_page_alias(tmp, phys);
tmp = TLBTEMP_BASE_1 + (addr & DCACHE_ALIAS_MASK);
__flush_invalidate_dcache_page_alias(tmp, phys);
__invalidate_icache_page_alias(tmp, phys);
+ preempt_enable();
clear_bit(PG_arch_1, &page->flags);
}
if (alias) {
unsigned long t = TLBTEMP_BASE_1 + (vaddr & DCACHE_ALIAS_MASK);
+ preempt_disable();
__flush_invalidate_dcache_page_alias(t, phys);
+ preempt_enable();
}
/* Copy data */
if (alias) {
unsigned long t = TLBTEMP_BASE_1 + (vaddr & DCACHE_ALIAS_MASK);
+ preempt_disable();
__flush_invalidate_dcache_range((unsigned long) dst, len);
if ((vma->vm_flags & VM_EXEC) != 0)
__invalidate_icache_page_alias(t, phys);
+ preempt_enable();
} else if ((vma->vm_flags & VM_EXEC) != 0) {
__flush_dcache_range((unsigned long)dst,len);
if (alias) {
unsigned long t = TLBTEMP_BASE_1 + (vaddr & DCACHE_ALIAS_MASK);
+ preempt_disable();
__flush_invalidate_dcache_page_alias(t, phys);
+ preempt_enable();
}
memcpy(dst, src, len);
blkg_iostat_set(&blkg->iostat.cur, &tmp);
u64_stats_update_end(&blkg->iostat.sync);
}
+ disk_put_part(part);
}
}
/* release the tag's ownership to the req cloned from */
spin_lock_irqsave(&fq->mq_flush_lock, flags);
- WRITE_ONCE(flush_rq->state, MQ_RQ_IDLE);
if (!refcount_dec_and_test(&flush_rq->ref)) {
fq->rq_status = error;
spin_unlock_irqrestore(&fq->mq_flush_lock, flags);
return;
}
+ /*
+ * Flush request has to be marked as IDLE when it is really ended
+ * because its .end_io() is called from timeout code path too for
+ * avoiding use-after-free.
+ */
+ WRITE_ONCE(flush_rq->state, MQ_RQ_IDLE);
if (fq->rq_status != BLK_STS_OK)
error = fq->rq_status;
* Set disk capacity and notify if the size is not currently
* zero and will not be set to zero
*/
-void set_capacity_revalidate_and_notify(struct gendisk *disk, sector_t size,
+bool set_capacity_revalidate_and_notify(struct gendisk *disk, sector_t size,
bool update_bdev)
{
sector_t capacity = get_capacity(disk);
char *envp[] = { "RESIZE=1", NULL };
kobject_uevent_env(&disk_to_dev(disk)->kobj, KOBJ_CHANGE, envp);
+ return true;
}
+
+ return false;
}
EXPORT_SYMBOL_GPL(set_capacity_revalidate_and_notify);
spin_lock_init(&ksm->idle_slots_lock);
slot_hashtable_size = roundup_pow_of_two(num_slots);
+ /*
+ * hash_ptr() assumes bits != 0, so ensure the hash table has at least 2
+ * buckets. This only makes a difference when there is only 1 keyslot.
+ */
+ if (slot_hashtable_size < 2)
+ slot_hashtable_size = 2;
+
ksm->log_slot_ht_size = ilog2(slot_hashtable_size);
ksm->slot_hashtable = kvmalloc_array(slot_hashtable_size,
sizeof(ksm->slot_hashtable[0]),
obj-$(CONFIG_SSB) += ssb/
obj-$(CONFIG_BCMA) += bcma/
obj-$(CONFIG_VHOST_RING) += vhost/
+obj-$(CONFIG_VHOST_IOTLB) += vhost/
obj-$(CONFIG_VHOST) += vhost/
obj-$(CONFIG_VLYNQ) += vlynq/
obj-$(CONFIG_GREYBUS) += greybus/
mark_cut_flag = 0;
synth_printf("%s\n", spk_msg_get(MSG_CUT));
- speakup_clear_selection();
ret = speakup_set_selection(tty);
switch (ret) {
struct tty_struct *tty;
};
-void speakup_clear_selection(void)
-{
- console_lock();
- clear_selection();
- console_unlock();
-}
-
static void __speakup_set_selection(struct work_struct *work)
{
struct speakup_selection_work *ssw =
goto unref;
}
+ console_lock();
+ clear_selection();
+ console_unlock();
+
set_selection_kernel(&sel, tty);
unref:
void speakup_start_ttys(void);
void synth_buffer_add(u16 ch);
void synth_buffer_clear(void);
-void speakup_clear_selection(void);
int speakup_set_selection(struct tty_struct *tty);
void speakup_cancel_selection(void);
int speakup_paste_selection(struct tty_struct *tty);
if (!tty->ops->write)
return -EOPNOTSUPP;
+
+ mutex_lock(&speakup_tty_mutex);
+ if (speakup_tty) {
+ mutex_unlock(&speakup_tty_mutex);
+ return -EBUSY;
+ }
speakup_tty = tty;
ldisc_data = kmalloc(sizeof(*ldisc_data), GFP_KERNEL);
- if (!ldisc_data)
+ if (!ldisc_data) {
+ speakup_tty = NULL;
+ mutex_unlock(&speakup_tty_mutex);
return -ENOMEM;
+ }
init_completion(&ldisc_data->completion);
ldisc_data->buf_free = true;
speakup_tty->disc_data = ldisc_data;
+ mutex_unlock(&speakup_tty_mutex);
return 0;
}
struct spk_ldisc_data *ldisc_data = speakup_tty->disc_data;
char rv;
- if (wait_for_completion_timeout(&ldisc_data->completion,
+ if (!timeout) {
+ if (!try_wait_for_completion(&ldisc_data->completion))
+ return 0xff;
+ } else if (wait_for_completion_timeout(&ldisc_data->completion,
usecs_to_jiffies(timeout)) == 0) {
- if (timeout)
- pr_warn("spk_ttyio: timeout (%d) while waiting for input\n",
- timeout);
+ pr_warn("spk_ttyio: timeout (%d) while waiting for input\n",
+ timeout);
return 0xff;
}
E_NEW_DEFAULT,
};
+/*
+ * Note: add new members at the end, speakupmap.h depends on the values of the
+ * enum starting from SPELL_DELAY (see inc_dec_var)
+ */
enum var_id_t {
VERSION = 0, SYNTH, SILENT, SYNTH_DIRECT,
KEYMAP, CHARS,
SAY_CONTROL, SAY_WORD_CTL, NO_INTERRUPT, KEY_ECHO,
SPELL_DELAY, PUNC_LEVEL, READING_PUNC,
ATTRIB_BLEEP, BLEEPS,
- RATE, PITCH, INFLECTION, VOL, TONE, PUNCT, VOICE, FREQUENCY, LANG,
+ RATE, PITCH, VOL, TONE, PUNCT, VOICE, FREQUENCY, LANG,
DIRECT, PAUSE,
- CAPS_START, CAPS_STOP, CHARTAB,
+ CAPS_START, CAPS_STOP, CHARTAB, INFLECTION,
MAXVARS
};
if (rc)
return rc;
+ /* IO space doesn't need mapping */
+ if (reg->space_id == ACPI_ADR_SPACE_SYSTEM_IO)
+ return 0;
+
if (!acpi_os_map_generic_address(reg))
return -ENXIO;
* iort_set_fwnode() - Create iort_fwnode and use it to register
* iommu data in the iort_fwnode_list
*
- * @node: IORT table node associated with the IOMMU
+ * @iort_node: IORT table node associated with the IOMMU
* @fwnode: fwnode associated with the IORT node
*
* Returns: 0 on success
/**
* iort_get_device_domain() - Find MSI domain related to a device
* @dev: The device.
- * @req_id: Requester ID for the device.
+ * @id: Requester ID for the device.
+ * @bus_token: irq domain bus token.
*
* Returns: the MSI domain for this device, NULL otherwise
*/
*
* @dev: device to configure
* @dma_addr: device DMA address result pointer
- * @size: DMA range size result pointer
+ * @dma_size: DMA range size result pointer
*/
void iort_dma_setup(struct device *dev, u64 *dma_addr, u64 *dma_size)
{
/**
* iort_add_platform_device() - Allocate a platform device for IORT node
* @node: Pointer to device ACPI IORT node
+ * @ops: Pointer to IORT device config struct
*
* Returns: 0 on success, <0 failure
*/
struct acpi_fan_fps *fps = &fan->fps[i];
snprintf(fps->name, ACPI_FPS_NAME_LEN, "state%d", i);
+ sysfs_attr_init(&fps->dev_attr.attr);
fps->dev_attr.show = show_state;
fps->dev_attr.store = NULL;
fps->dev_attr.attr.name = fps->name;
struct sk_buff *skb;
unsigned long flags, timeout;
int read, write, lastread = -1;
- APRINTK(!in_interrupt(),
- "lanai_shutdown_tx_vci called w/o process context!\n");
+
if (lvcc->vbase == NULL) /* We were never bound to a VCI */
return;
/* 15.2.1 - wait for queue to drain */
static void ns_close(struct atm_vcc *vcc);
static void fill_tst(ns_dev * card, int n, vc_map * vc);
static int ns_send(struct atm_vcc *vcc, struct sk_buff *skb);
+static int ns_send_bh(struct atm_vcc *vcc, struct sk_buff *skb);
static int push_scqe(ns_dev * card, vc_map * vc, scq_info * scq, ns_scqe * tbd,
- struct sk_buff *skb);
+ struct sk_buff *skb, bool may_sleep);
static void process_tsq(ns_dev * card);
static void drain_scq(ns_dev * card, scq_info * scq, int pos);
static void process_rsq(ns_dev * card);
.close = ns_close,
.ioctl = ns_ioctl,
.send = ns_send,
+ .send_bh = ns_send_bh,
.phy_put = ns_phy_put,
.phy_get = ns_phy_get,
.proc_read = ns_proc_read,
card->tst_addr = new_tst;
}
-static int ns_send(struct atm_vcc *vcc, struct sk_buff *skb)
+static int _ns_send(struct atm_vcc *vcc, struct sk_buff *skb, bool may_sleep)
{
ns_dev *card;
vc_map *vc;
scq = card->scq0;
}
- if (push_scqe(card, vc, scq, &scqe, skb) != 0) {
+ if (push_scqe(card, vc, scq, &scqe, skb, may_sleep) != 0) {
atomic_inc(&vcc->stats->tx_err);
+ dma_unmap_single(&card->pcidev->dev, NS_PRV_DMA(skb), skb->len,
+ DMA_TO_DEVICE);
dev_kfree_skb_any(skb);
return -EIO;
}
return 0;
}
+static int ns_send(struct atm_vcc *vcc, struct sk_buff *skb)
+{
+ return _ns_send(vcc, skb, true);
+}
+
+static int ns_send_bh(struct atm_vcc *vcc, struct sk_buff *skb)
+{
+ return _ns_send(vcc, skb, false);
+}
+
static int push_scqe(ns_dev * card, vc_map * vc, scq_info * scq, ns_scqe * tbd,
- struct sk_buff *skb)
+ struct sk_buff *skb, bool may_sleep)
{
unsigned long flags;
ns_scqe tsr;
spin_lock_irqsave(&scq->lock, flags);
while (scq->tail == scq->next) {
- if (in_interrupt()) {
+ if (!may_sleep) {
spin_unlock_irqrestore(&scq->lock, flags);
printk("nicstar%d: Error pushing TBD.\n", card->index);
return 1;
int has_run = 0;
while (scq->tail == scq->next) {
- if (in_interrupt()) {
+ if (!may_sleep) {
data = scq_virt_to_bus(scq, scq->next);
ns_write_sram(card, scq->scd, &data, 1);
spin_unlock_irqrestore(&scq->lock, flags);
bd_set_nr_sectors(bdev, size);
- set_capacity_revalidate_and_notify(lo->lo_disk, size, false);
+ if (!set_capacity_revalidate_and_notify(lo->lo_disk, size, false))
+ kobject_uevent(&disk_to_dev(bdev->bd_disk)->kobj, KOBJ_CHANGE);
}
static inline int
if (test_bit(NBD_RT_DISCONNECT_ON_CLOSE, &nbd->config->runtime_flags) &&
bdev->bd_openers == 0)
nbd_disconnect_and_put(nbd);
+ bdput(bdev);
nbd_config_put(nbd);
nbd_put(nbd);
u32 sysc_mask, syss_done, rstval;
int syss_offset, error = 0;
+ if (ddata->cap->regbits->srst_shift < 0)
+ return 0;
+
syss_offset = ddata->offsets[SYSC_SYSSTATUS];
sysc_mask = BIT(ddata->cap->regbits->srst_shift);
return error;
}
}
- error = sysc_wait_softreset(ddata);
- if (error)
- dev_warn(ddata->dev, "OCP softreset timed out\n");
+ /*
+ * Some modules like i2c and hdq1w have unusable reset status unless
+ * the module reset quirk is enabled. Skip status check on enable.
+ */
+ if (!(ddata->cfg.quirks & SYSC_MODULE_QUIRK_ENA_RESETDONE)) {
+ error = sysc_wait_softreset(ddata);
+ if (error)
+ dev_warn(ddata->dev, "OCP softreset timed out\n");
+ }
if (ddata->cfg.quirks & SYSC_QUIRK_OPT_CLKS_IN_RESET)
sysc_disable_opt_clocks(ddata);
SYSC_QUIRK("hdmi", 0, 0, 0x10, -ENODEV, 0x50030200, 0xffffffff,
SYSC_QUIRK_OPT_CLKS_NEEDED),
SYSC_QUIRK("hdq1w", 0, 0, 0x14, 0x18, 0x00000006, 0xffffffff,
- SYSC_MODULE_QUIRK_HDQ1W),
+ SYSC_MODULE_QUIRK_HDQ1W | SYSC_MODULE_QUIRK_ENA_RESETDONE),
SYSC_QUIRK("hdq1w", 0, 0, 0x14, 0x18, 0x0000000a, 0xffffffff,
- SYSC_MODULE_QUIRK_HDQ1W),
+ SYSC_MODULE_QUIRK_HDQ1W | SYSC_MODULE_QUIRK_ENA_RESETDONE),
SYSC_QUIRK("i2c", 0, 0, 0x20, 0x10, 0x00000036, 0x000000ff,
- SYSC_MODULE_QUIRK_I2C),
+ SYSC_MODULE_QUIRK_I2C | SYSC_MODULE_QUIRK_ENA_RESETDONE),
SYSC_QUIRK("i2c", 0, 0, 0x20, 0x10, 0x0000003c, 0x000000ff,
- SYSC_MODULE_QUIRK_I2C),
+ SYSC_MODULE_QUIRK_I2C | SYSC_MODULE_QUIRK_ENA_RESETDONE),
SYSC_QUIRK("i2c", 0, 0, 0x20, 0x10, 0x00000040, 0x000000ff,
- SYSC_MODULE_QUIRK_I2C),
+ SYSC_MODULE_QUIRK_I2C | SYSC_MODULE_QUIRK_ENA_RESETDONE),
SYSC_QUIRK("i2c", 0, 0, 0x10, 0x90, 0x5040000a, 0xfffff0f0,
- SYSC_MODULE_QUIRK_I2C),
+ SYSC_MODULE_QUIRK_I2C | SYSC_MODULE_QUIRK_ENA_RESETDONE),
SYSC_QUIRK("gpu", 0x50000000, 0x14, -ENODEV, -ENODEV, 0x00010201, 0xffffffff, 0),
SYSC_QUIRK("gpu", 0x50000000, 0xfe00, 0xfe10, -ENODEV, 0x40000000 , 0xffffffff,
SYSC_MODULE_QUIRK_SGX),
if ((ddata->cfg.quirks & SYSC_QUIRK_NO_RESET_ON_INIT) &&
(ddata->cfg.quirks & SYSC_QUIRK_NO_IDLE))
- return -EBUSY;
+ return -ENXIO;
return 0;
}
/*
* Allocate DMA memory from ancestor. When a virtio
* device is created by remoteproc, the DMA memory is
- * associated with the grandparent device:
- * vdev => rproc => platform-dev.
+ * associated with the parent device:
+ * virtioY => remoteprocX#vdevYbuffer.
*/
- if (!vdev->dev.parent || !vdev->dev.parent->parent)
+ buf->dev = vdev->dev.parent;
+ if (!buf->dev)
goto free_buf;
- buf->dev = vdev->dev.parent->parent;
/* Increase device refcnt to avoid freeing it */
get_device(buf->dev);
hws[IMX8MM_CLK_A53_CORE] = imx_clk_hw_mux2("arm_a53_core", base + 0x9880, 24, 1, imx8mm_a53_core_sels, ARRAY_SIZE(imx8mm_a53_core_sels));
/* BUS */
- hws[IMX8MM_CLK_MAIN_AXI] = imx8m_clk_hw_composite_critical("main_axi", imx8mm_main_axi_sels, base + 0x8800);
+ hws[IMX8MM_CLK_MAIN_AXI] = imx8m_clk_hw_composite_bus_critical("main_axi", imx8mm_main_axi_sels, base + 0x8800);
hws[IMX8MM_CLK_ENET_AXI] = imx8m_clk_hw_composite_bus("enet_axi", imx8mm_enet_axi_sels, base + 0x8880);
- hws[IMX8MM_CLK_NAND_USDHC_BUS] = imx8m_clk_hw_composite_critical("nand_usdhc_bus", imx8mm_nand_usdhc_sels, base + 0x8900);
+ hws[IMX8MM_CLK_NAND_USDHC_BUS] = imx8m_clk_hw_composite_bus_critical("nand_usdhc_bus", imx8mm_nand_usdhc_sels, base + 0x8900);
hws[IMX8MM_CLK_VPU_BUS] = imx8m_clk_hw_composite_bus("vpu_bus", imx8mm_vpu_bus_sels, base + 0x8980);
hws[IMX8MM_CLK_DISP_AXI] = imx8m_clk_hw_composite_bus("disp_axi", imx8mm_disp_axi_sels, base + 0x8a00);
hws[IMX8MM_CLK_DISP_APB] = imx8m_clk_hw_composite_bus("disp_apb", imx8mm_disp_apb_sels, base + 0x8a80);
hws[IMX8MM_CLK_USB_BUS] = imx8m_clk_hw_composite_bus("usb_bus", imx8mm_usb_bus_sels, base + 0x8b80);
hws[IMX8MM_CLK_GPU_AXI] = imx8m_clk_hw_composite_bus("gpu_axi", imx8mm_gpu_axi_sels, base + 0x8c00);
hws[IMX8MM_CLK_GPU_AHB] = imx8m_clk_hw_composite_bus("gpu_ahb", imx8mm_gpu_ahb_sels, base + 0x8c80);
- hws[IMX8MM_CLK_NOC] = imx8m_clk_hw_composite_critical("noc", imx8mm_noc_sels, base + 0x8d00);
- hws[IMX8MM_CLK_NOC_APB] = imx8m_clk_hw_composite_critical("noc_apb", imx8mm_noc_apb_sels, base + 0x8d80);
+ hws[IMX8MM_CLK_NOC] = imx8m_clk_hw_composite_bus_critical("noc", imx8mm_noc_sels, base + 0x8d00);
+ hws[IMX8MM_CLK_NOC_APB] = imx8m_clk_hw_composite_bus_critical("noc_apb", imx8mm_noc_apb_sels, base + 0x8d80);
/* AHB */
- hws[IMX8MM_CLK_AHB] = imx8m_clk_hw_composite_critical("ahb", imx8mm_ahb_sels, base + 0x9000);
+ hws[IMX8MM_CLK_AHB] = imx8m_clk_hw_composite_bus_critical("ahb", imx8mm_ahb_sels, base + 0x9000);
hws[IMX8MM_CLK_AUDIO_AHB] = imx8m_clk_hw_composite_bus("audio_ahb", imx8mm_audio_ahb_sels, base + 0x9100);
/* IPG */
hws[IMX8MN_CLK_A53_CORE] = imx_clk_hw_mux2("arm_a53_core", base + 0x9880, 24, 1, imx8mn_a53_core_sels, ARRAY_SIZE(imx8mn_a53_core_sels));
/* BUS */
- hws[IMX8MN_CLK_MAIN_AXI] = imx8m_clk_hw_composite_critical("main_axi", imx8mn_main_axi_sels, base + 0x8800);
+ hws[IMX8MN_CLK_MAIN_AXI] = imx8m_clk_hw_composite_bus_critical("main_axi", imx8mn_main_axi_sels, base + 0x8800);
hws[IMX8MN_CLK_ENET_AXI] = imx8m_clk_hw_composite_bus("enet_axi", imx8mn_enet_axi_sels, base + 0x8880);
hws[IMX8MN_CLK_NAND_USDHC_BUS] = imx8m_clk_hw_composite_bus("nand_usdhc_bus", imx8mn_nand_usdhc_sels, base + 0x8900);
hws[IMX8MN_CLK_DISP_AXI] = imx8m_clk_hw_composite_bus("disp_axi", imx8mn_disp_axi_sels, base + 0x8a00);
hws[IMX8MN_CLK_USB_BUS] = imx8m_clk_hw_composite_bus("usb_bus", imx8mn_usb_bus_sels, base + 0x8b80);
hws[IMX8MN_CLK_GPU_AXI] = imx8m_clk_hw_composite_bus("gpu_axi", imx8mn_gpu_axi_sels, base + 0x8c00);
hws[IMX8MN_CLK_GPU_AHB] = imx8m_clk_hw_composite_bus("gpu_ahb", imx8mn_gpu_ahb_sels, base + 0x8c80);
- hws[IMX8MN_CLK_NOC] = imx8m_clk_hw_composite_critical("noc", imx8mn_noc_sels, base + 0x8d00);
+ hws[IMX8MN_CLK_NOC] = imx8m_clk_hw_composite_bus_critical("noc", imx8mn_noc_sels, base + 0x8d00);
- hws[IMX8MN_CLK_AHB] = imx8m_clk_hw_composite_critical("ahb", imx8mn_ahb_sels, base + 0x9000);
+ hws[IMX8MN_CLK_AHB] = imx8m_clk_hw_composite_bus_critical("ahb", imx8mn_ahb_sels, base + 0x9000);
hws[IMX8MN_CLK_AUDIO_AHB] = imx8m_clk_hw_composite_bus("audio_ahb", imx8mn_audio_ahb_sels, base + 0x9100);
hws[IMX8MN_CLK_IPG_ROOT] = imx_clk_hw_divider2("ipg_root", "ahb", base + 0x9080, 0, 1);
hws[IMX8MN_CLK_IPG_AUDIO_ROOT] = imx_clk_hw_divider2("ipg_audio_root", "audio_ahb", base + 0x9180, 0, 1);
/* CORE SEL */
hws[IMX8MP_CLK_A53_CORE] = imx_clk_hw_mux2("arm_a53_core", ccm_base + 0x9880, 24, 1, imx8mp_a53_core_sels, ARRAY_SIZE(imx8mp_a53_core_sels));
- hws[IMX8MP_CLK_MAIN_AXI] = imx8m_clk_hw_composite_critical("main_axi", imx8mp_main_axi_sels, ccm_base + 0x8800);
+ hws[IMX8MP_CLK_MAIN_AXI] = imx8m_clk_hw_composite_bus_critical("main_axi", imx8mp_main_axi_sels, ccm_base + 0x8800);
hws[IMX8MP_CLK_ENET_AXI] = imx8m_clk_hw_composite_bus("enet_axi", imx8mp_enet_axi_sels, ccm_base + 0x8880);
- hws[IMX8MP_CLK_NAND_USDHC_BUS] = imx8m_clk_hw_composite_critical("nand_usdhc_bus", imx8mp_nand_usdhc_sels, ccm_base + 0x8900);
+ hws[IMX8MP_CLK_NAND_USDHC_BUS] = imx8m_clk_hw_composite_bus_critical("nand_usdhc_bus", imx8mp_nand_usdhc_sels, ccm_base + 0x8900);
hws[IMX8MP_CLK_VPU_BUS] = imx8m_clk_hw_composite_bus("vpu_bus", imx8mp_vpu_bus_sels, ccm_base + 0x8980);
hws[IMX8MP_CLK_MEDIA_AXI] = imx8m_clk_hw_composite_bus("media_axi", imx8mp_media_axi_sels, ccm_base + 0x8a00);
hws[IMX8MP_CLK_MEDIA_APB] = imx8m_clk_hw_composite_bus("media_apb", imx8mp_media_apb_sels, ccm_base + 0x8a80);
hws[IMX8MP_CLK_HDMI_AXI] = imx8m_clk_hw_composite_bus("hdmi_axi", imx8mp_media_axi_sels, ccm_base + 0x8b80);
hws[IMX8MP_CLK_GPU_AXI] = imx8m_clk_hw_composite_bus("gpu_axi", imx8mp_gpu_axi_sels, ccm_base + 0x8c00);
hws[IMX8MP_CLK_GPU_AHB] = imx8m_clk_hw_composite_bus("gpu_ahb", imx8mp_gpu_ahb_sels, ccm_base + 0x8c80);
- hws[IMX8MP_CLK_NOC] = imx8m_clk_hw_composite_critical("noc", imx8mp_noc_sels, ccm_base + 0x8d00);
- hws[IMX8MP_CLK_NOC_IO] = imx8m_clk_hw_composite_critical("noc_io", imx8mp_noc_io_sels, ccm_base + 0x8d80);
+ hws[IMX8MP_CLK_NOC] = imx8m_clk_hw_composite_bus_critical("noc", imx8mp_noc_sels, ccm_base + 0x8d00);
+ hws[IMX8MP_CLK_NOC_IO] = imx8m_clk_hw_composite_bus_critical("noc_io", imx8mp_noc_io_sels, ccm_base + 0x8d80);
hws[IMX8MP_CLK_ML_AXI] = imx8m_clk_hw_composite_bus("ml_axi", imx8mp_ml_axi_sels, ccm_base + 0x8e00);
hws[IMX8MP_CLK_ML_AHB] = imx8m_clk_hw_composite_bus("ml_ahb", imx8mp_ml_ahb_sels, ccm_base + 0x8e80);
- hws[IMX8MP_CLK_AHB] = imx8m_clk_hw_composite_critical("ahb_root", imx8mp_ahb_sels, ccm_base + 0x9000);
+ hws[IMX8MP_CLK_AHB] = imx8m_clk_hw_composite_bus_critical("ahb_root", imx8mp_ahb_sels, ccm_base + 0x9000);
hws[IMX8MP_CLK_AUDIO_AHB] = imx8m_clk_hw_composite_bus("audio_ahb", imx8mp_audio_ahb_sels, ccm_base + 0x9100);
hws[IMX8MP_CLK_MIPI_DSI_ESC_RX] = imx8m_clk_hw_composite_bus("mipi_dsi_esc_rx", imx8mp_mipi_dsi_esc_rx_sels, ccm_base + 0x9200);
hws[IMX8MQ_CLK_A53_CORE] = imx_clk_hw_mux2("arm_a53_core", base + 0x9880, 24, 1, imx8mq_a53_core_sels, ARRAY_SIZE(imx8mq_a53_core_sels));
/* BUS */
- hws[IMX8MQ_CLK_MAIN_AXI] = imx8m_clk_hw_composite_critical("main_axi", imx8mq_main_axi_sels, base + 0x8800);
+ hws[IMX8MQ_CLK_MAIN_AXI] = imx8m_clk_hw_composite_bus_critical("main_axi", imx8mq_main_axi_sels, base + 0x8800);
hws[IMX8MQ_CLK_ENET_AXI] = imx8m_clk_hw_composite_bus("enet_axi", imx8mq_enet_axi_sels, base + 0x8880);
hws[IMX8MQ_CLK_NAND_USDHC_BUS] = imx8m_clk_hw_composite_bus("nand_usdhc_bus", imx8mq_nand_usdhc_sels, base + 0x8900);
hws[IMX8MQ_CLK_VPU_BUS] = imx8m_clk_hw_composite_bus("vpu_bus", imx8mq_vpu_bus_sels, base + 0x8980);
hws[IMX8MQ_CLK_USB_BUS] = imx8m_clk_hw_composite_bus("usb_bus", imx8mq_usb_bus_sels, base + 0x8b80);
hws[IMX8MQ_CLK_GPU_AXI] = imx8m_clk_hw_composite_bus("gpu_axi", imx8mq_gpu_axi_sels, base + 0x8c00);
hws[IMX8MQ_CLK_GPU_AHB] = imx8m_clk_hw_composite_bus("gpu_ahb", imx8mq_gpu_ahb_sels, base + 0x8c80);
- hws[IMX8MQ_CLK_NOC] = imx8m_clk_hw_composite_critical("noc", imx8mq_noc_sels, base + 0x8d00);
- hws[IMX8MQ_CLK_NOC_APB] = imx8m_clk_hw_composite_critical("noc_apb", imx8mq_noc_apb_sels, base + 0x8d80);
+ hws[IMX8MQ_CLK_NOC] = imx8m_clk_hw_composite_bus_critical("noc", imx8mq_noc_sels, base + 0x8d00);
+ hws[IMX8MQ_CLK_NOC_APB] = imx8m_clk_hw_composite_bus_critical("noc_apb", imx8mq_noc_apb_sels, base + 0x8d80);
/* AHB */
/* AHB clock is used by the AHB bus therefore marked as critical */
- hws[IMX8MQ_CLK_AHB] = imx8m_clk_hw_composite_critical("ahb", imx8mq_ahb_sels, base + 0x9000);
+ hws[IMX8MQ_CLK_AHB] = imx8m_clk_hw_composite_bus_critical("ahb", imx8mq_ahb_sels, base + 0x9000);
hws[IMX8MQ_CLK_AUDIO_AHB] = imx8m_clk_hw_composite_bus("audio_ahb", imx8mq_audio_ahb_sels, base + 0x9100);
/* IPG */
IMX_COMPOSITE_BUS, \
CLK_SET_RATE_NO_REPARENT | CLK_OPS_PARENT_ENABLE)
+#define imx8m_clk_hw_composite_bus_critical(name, parent_names, reg) \
+ imx8m_clk_hw_composite_flags(name, parent_names, ARRAY_SIZE(parent_names), reg, \
+ IMX_COMPOSITE_BUS, \
+ CLK_SET_RATE_NO_REPARENT | CLK_OPS_PARENT_ENABLE | CLK_IS_CRITICAL)
+
#define imx8m_clk_hw_composite_core(name, parent_names, reg) \
imx8m_clk_hw_composite_flags(name, parent_names, \
ARRAY_SIZE(parent_names), reg, \
void *data;
};
-#define to_clk_regmap(_hw) container_of(_hw, struct clk_regmap, hw)
+static inline struct clk_regmap *to_clk_regmap(struct clk_hw *hw)
+{
+ return container_of(hw, struct clk_regmap, hw);
+}
/**
* struct clk_regmap_gate_data - regmap backed gate specific data
unsigned int enable_mask;
bool enable_is_inverted;
};
-#define to_clk_regmap(_hw) container_of(_hw, struct clk_regmap, hw)
+
+static inline struct clk_regmap *to_clk_regmap(struct clk_hw *hw)
+{
+ return container_of(hw, struct clk_regmap, hw);
+}
int clk_is_enabled_regmap(struct clk_hw *hw);
int clk_enable_regmap(struct clk_hw *hw);
.reg_bits = 32,
.val_bits = 32,
.reg_stride = 4,
- .max_register = 0x24,
+ .max_register = QUPRD,
};
static const struct regmap_config ti_eqep_regmap16_config = {
.reg_bits = 16,
.val_bits = 16,
.reg_stride = 2,
- .max_register = 0x1e,
+ .max_register = QCPRDLAT,
};
static int ti_eqep_probe(struct platform_device *pdev)
#define pr_fmt(fmt) KBUILD_MODNAME ": " fmt
+#include <linux/clk-provider.h>
#include <linux/cpu.h>
#include <linux/cpufreq.h>
#include <linux/cpumask.h>
static int scmi_cpufreq_probe(struct scmi_device *sdev)
{
int ret;
+ struct device *dev = &sdev->dev;
handle = sdev->handle;
if (!handle || !handle->perf_ops)
return -ENODEV;
+#ifdef CONFIG_COMMON_CLK
+ /* dummy clock provider as needed by OPP if clocks property is used */
+ if (of_find_property(dev->of_node, "#clock-cells", NULL))
+ devm_of_clk_add_hw_provider(dev, of_clk_hw_simple_get, NULL);
+#endif
+
ret = cpufreq_register_driver(&scmi_cpufreq_driver);
if (ret) {
- dev_err(&sdev->dev, "%s: registering cpufreq failed, err: %d\n",
+ dev_err(dev, "%s: registering cpufreq failed, err: %d\n",
__func__, ret);
}
struct tegra186_cpufreq_cluster {
const struct tegra186_cpufreq_cluster_info *info;
struct cpufreq_frequency_table *table;
+ u32 ref_clk_khz;
+ u32 div;
};
struct tegra186_cpufreq_data {
static unsigned int tegra186_cpufreq_get(unsigned int cpu)
{
- struct cpufreq_frequency_table *tbl;
+ struct tegra186_cpufreq_data *data = cpufreq_get_driver_data();
struct cpufreq_policy *policy;
void __iomem *edvd_reg;
unsigned int i, freq = 0;
if (!policy)
return 0;
- tbl = policy->freq_table;
edvd_reg = policy->driver_data;
ndiv = readl(edvd_reg) & EDVD_CORE_VOLT_FREQ_F_MASK;
- for (i = 0; tbl[i].frequency != CPUFREQ_TABLE_END; i++) {
- if ((tbl[i].driver_data & EDVD_CORE_VOLT_FREQ_F_MASK) == ndiv) {
- freq = tbl[i].frequency;
- break;
+ for (i = 0; i < data->num_clusters; i++) {
+ struct tegra186_cpufreq_cluster *cluster = &data->clusters[i];
+ int core;
+
+ for (core = 0; core < ARRAY_SIZE(cluster->info->cpus); core++) {
+ if (cluster->info->cpus[core] != policy->cpu)
+ continue;
+
+ freq = (cluster->ref_clk_khz * ndiv) / cluster->div;
+ goto out;
}
}
+out:
cpufreq_cpu_put(policy);
return freq;
static struct cpufreq_frequency_table *init_vhint_table(
struct platform_device *pdev, struct tegra_bpmp *bpmp,
- unsigned int cluster_id)
+ struct tegra186_cpufreq_cluster *cluster)
{
struct cpufreq_frequency_table *table;
struct mrq_cpu_vhint_request req;
memset(&req, 0, sizeof(req));
req.addr = phys;
- req.cluster_id = cluster_id;
+ req.cluster_id = cluster->info->bpmp_cluster_id;
memset(&msg, 0, sizeof(msg));
msg.mrq = MRQ_CPU_VHINT;
goto free;
}
+ cluster->ref_clk_khz = data->ref_clk_hz / 1000;
+ cluster->div = data->pdiv * data->mdiv;
+
for (i = data->vfloor, j = 0; i <= data->vceil; i++) {
struct cpufreq_frequency_table *point;
u16 ndiv = data->ndiv[i];
point = &table[j++];
point->driver_data = edvd_val;
- point->frequency = data->ref_clk_hz * ndiv / data->pdiv /
- data->mdiv / 1000;
+ point->frequency = (cluster->ref_clk_khz * ndiv) / cluster->div;
}
table[j].frequency = CPUFREQ_TABLE_END;
struct tegra186_cpufreq_cluster *cluster = &data->clusters[i];
cluster->info = &tegra186_clusters[i];
- cluster->table = init_vhint_table(
- pdev, bpmp, cluster->info->bpmp_cluster_id);
+ cluster->table = init_vhint_table(pdev, bpmp, cluster);
if (IS_ERR(cluster->table)) {
err = PTR_ERR(cluster->table);
goto put_bpmp;
}
local_fiq_disable();
- tegra_pm_set_cpu_in_lp2();
+ RCU_NONIDLE(tegra_pm_set_cpu_in_lp2());
cpu_pm_enter();
switch (index) {
}
cpu_pm_exit();
- tegra_pm_clear_cpu_in_lp2();
+ RCU_NONIDLE(tegra_pm_clear_cpu_in_lp2());
local_fiq_enable();
return err ?: index;
Say M if unsure.
config DEV_DAX_HMEM_DEVICES
- depends on NUMA_KEEP_MEMINFO # for phys_to_target_node()
depends on DEV_DAX_HMEM && DAX=y
def_bool y
static int __dma_async_device_channel_register(struct dma_device *device,
struct dma_chan *chan)
{
- int rc = 0;
+ int rc;
chan->local = alloc_percpu(typeof(*chan->local));
if (!chan->local)
- goto err_out;
+ return -ENOMEM;
chan->dev = kzalloc(sizeof(*chan->dev), GFP_KERNEL);
if (!chan->dev) {
- free_percpu(chan->local);
- chan->local = NULL;
- goto err_out;
+ rc = -ENOMEM;
+ goto err_free_local;
}
/*
if (chan->chan_id < 0) {
pr_err("%s: unable to alloc ida for chan: %d\n",
__func__, chan->chan_id);
- goto err_out;
+ rc = chan->chan_id;
+ goto err_free_dev;
}
chan->dev->device.class = &dma_devclass;
mutex_lock(&device->chan_mutex);
ida_free(&device->chan_ida, chan->chan_id);
mutex_unlock(&device->chan_mutex);
- err_out:
- free_percpu(chan->local);
+ err_free_dev:
kfree(chan->dev);
+ err_free_local:
+ free_percpu(chan->local);
return rc;
}
resource_size_t start;
start = pci_resource_start(pdev, IDXD_WQ_BAR);
- start = start + wq->id * IDXD_PORTAL_SIZE;
+ start += idxd_get_wq_portal_full_offset(wq->id, IDXD_PORTAL_LIMITED);
wq->dportal = devm_ioremap(dev, start, IDXD_PORTAL_SIZE);
if (!wq->dportal)
int i, wq_offset;
lockdep_assert_held(&idxd->dev_lock);
- memset(&wq->wqcfg, 0, sizeof(wq->wqcfg));
+ memset(wq->wqcfg, 0, idxd->wqcfg_size);
wq->type = IDXD_WQT_NONE;
wq->size = 0;
wq->group = NULL;
clear_bit(WQ_FLAG_DEDICATED, &wq->flags);
memset(wq->name, 0, WQ_NAME_SIZE);
- for (i = 0; i < 8; i++) {
- wq_offset = idxd->wqcfg_offset + wq->id * 32 + i * sizeof(u32);
+ for (i = 0; i < WQCFG_STRIDES(idxd); i++) {
+ wq_offset = WQCFG_OFFSET(idxd, wq->id, i);
iowrite32(0, idxd->reg_base + wq_offset);
dev_dbg(dev, "WQ[%d][%d][%#x]: %#x\n",
wq->id, i, wq_offset,
if (!wq->group)
return 0;
- memset(&wq->wqcfg, 0, sizeof(union wqcfg));
+ memset(wq->wqcfg, 0, idxd->wqcfg_size);
/* byte 0-3 */
- wq->wqcfg.wq_size = wq->size;
+ wq->wqcfg->wq_size = wq->size;
if (wq->size == 0) {
dev_warn(dev, "Incorrect work queue size: 0\n");
}
/* bytes 4-7 */
- wq->wqcfg.wq_thresh = wq->threshold;
+ wq->wqcfg->wq_thresh = wq->threshold;
/* byte 8-11 */
- wq->wqcfg.priv = !!(wq->type == IDXD_WQT_KERNEL);
- wq->wqcfg.mode = 1;
-
- wq->wqcfg.priority = wq->priority;
+ wq->wqcfg->priv = !!(wq->type == IDXD_WQT_KERNEL);
+ wq->wqcfg->mode = 1;
+ wq->wqcfg->priority = wq->priority;
/* bytes 12-15 */
- wq->wqcfg.max_xfer_shift = ilog2(wq->max_xfer_bytes);
- wq->wqcfg.max_batch_shift = ilog2(wq->max_batch_size);
+ wq->wqcfg->max_xfer_shift = ilog2(wq->max_xfer_bytes);
+ wq->wqcfg->max_batch_shift = ilog2(wq->max_batch_size);
dev_dbg(dev, "WQ %d CFGs\n", wq->id);
- for (i = 0; i < 8; i++) {
- wq_offset = idxd->wqcfg_offset + wq->id * 32 + i * sizeof(u32);
- iowrite32(wq->wqcfg.bits[i], idxd->reg_base + wq_offset);
+ for (i = 0; i < WQCFG_STRIDES(idxd); i++) {
+ wq_offset = WQCFG_OFFSET(idxd, wq->id, i);
+ iowrite32(wq->wqcfg->bits[i], idxd->reg_base + wq_offset);
dev_dbg(dev, "WQ[%d][%d][%#x]: %#x\n",
wq->id, i, wq_offset,
ioread32(idxd->reg_base + wq_offset));
u32 priority;
enum idxd_wq_state state;
unsigned long flags;
- union wqcfg wqcfg;
+ union wqcfg *wqcfg;
u32 vec_ptr; /* interrupt steering */
struct dsa_hw_desc **hw_descs;
int num_descs;
int max_wq_size;
int token_limit;
int nr_tokens; /* non-reserved tokens */
+ unsigned int wqcfg_size;
union sw_err_reg sw_err;
wait_queue_head_t cmd_waitq;
wq->idxd_cdev.minor = -1;
wq->max_xfer_bytes = idxd->max_xfer_bytes;
wq->max_batch_size = idxd->max_batch_size;
+ wq->wqcfg = devm_kzalloc(dev, idxd->wqcfg_size, GFP_KERNEL);
+ if (!wq->wqcfg)
+ return -ENOMEM;
}
for (i = 0; i < idxd->max_engines; i++) {
dev_dbg(dev, "total workqueue size: %u\n", idxd->max_wq_size);
idxd->max_wqs = idxd->hw.wq_cap.num_wqs;
dev_dbg(dev, "max workqueues: %u\n", idxd->max_wqs);
+ idxd->wqcfg_size = 1 << (idxd->hw.wq_cap.wqcfg_size + IDXD_WQCFG_MIN);
+ dev_dbg(dev, "wqcfg size: %u\n", idxd->wqcfg_size);
/* reading operation capabilities */
for (i = 0; i < 4; i++) {
#define IDXD_MMIO_BAR 0
#define IDXD_WQ_BAR 2
-#define IDXD_PORTAL_SIZE 0x4000
+#define IDXD_PORTAL_SIZE PAGE_SIZE
/* MMIO Device BAR0 Registers */
#define IDXD_VER_OFFSET 0x00
struct {
u64 total_wq_size:16;
u64 num_wqs:8;
- u64 rsvd:24;
+ u64 wqcfg_size:4;
+ u64 rsvd:20;
u64 shared_mode:1;
u64 dedicated_mode:1;
u64 rsvd2:1;
u64 bits;
} __packed;
#define IDXD_WQCAP_OFFSET 0x20
+#define IDXD_WQCFG_MIN 5
union group_cap_reg {
struct {
};
u32 bits[8];
} __packed;
+
+/*
+ * This macro calculates the offset into the WQCFG register
+ * idxd - struct idxd *
+ * n - wq id
+ * ofs - the index of the 32b dword for the config register
+ *
+ * The WQCFG register block is divided into groups per each wq. The n index
+ * allows us to move to the register group that's for that particular wq.
+ * Each register is 32bits. The ofs gives us the number of register to access.
+ */
+#define WQCFG_OFFSET(_idxd_dev, n, ofs) \
+({\
+ typeof(_idxd_dev) __idxd_dev = (_idxd_dev); \
+ (__idxd_dev)->wqcfg_offset + (n) * (__idxd_dev)->wqcfg_size + sizeof(u32) * (ofs); \
+})
+
+#define WQCFG_STRIDES(_idxd_dev) ((_idxd_dev)->wqcfg_size / sizeof(u32))
+
#endif
if (idxd->state != IDXD_DEV_ENABLED)
return -EIO;
- portal = wq->dportal + idxd_get_wq_portal_offset(IDXD_PORTAL_UNLIMITED);
+ portal = wq->dportal;
/*
* The wmb() flushes writes to coherent DMA data before possibly
* triggering a DMA read. The wmb() is necessary even on UP because
#define DCA2_TAG_MAP_BYTE3 0x82
#define DCA2_TAG_MAP_BYTE4 0x82
-/* verify if tag map matches expected values */
-static inline int dca2_tag_map_valid(u8 *tag_map)
-{
- return ((tag_map[0] == DCA2_TAG_MAP_BYTE0) &&
- (tag_map[1] == DCA2_TAG_MAP_BYTE1) &&
- (tag_map[2] == DCA2_TAG_MAP_BYTE2) &&
- (tag_map[3] == DCA2_TAG_MAP_BYTE3) &&
- (tag_map[4] == DCA2_TAG_MAP_BYTE4));
-}
-
/*
* "Legacy" DCA systems do not implement the DCA register set in the
* I/OAT device. Software needs direct support for their tag mappings.
* If burst size is smaller than bus width then make sure we only
* transfer one at a time to avoid a burst stradling an MFIFO entry.
*/
- if (desc->rqcfg.brst_size * 8 < pl330->pcfg.data_bus_width)
+ if (burst * 8 < pl330->pcfg.data_bus_width)
desc->rqcfg.brst_len = 1;
desc->bytes_requested = len;
#define XUDMA_GET_PUT_RESOURCE(res) \
struct udma_##res *xudma_##res##_get(struct udma_dev *ud, int id) \
{ \
- return __udma_reserve_##res(ud, false, id); \
+ return __udma_reserve_##res(ud, UDMA_TP_NORMAL, id); \
} \
EXPORT_SYMBOL(xudma_##res##_get); \
\
}
}
+/* Currently used by omap2 & 3 to block deeper SoC idle states */
+static bool omap_dma_busy(struct omap_dmadev *od)
+{
+ struct omap_chan *c;
+ int lch = -1;
+
+ while (1) {
+ lch = find_next_bit(od->lch_bitmap, od->lch_count, lch + 1);
+ if (lch >= od->lch_count)
+ break;
+ c = od->lch_map[lch];
+ if (!c)
+ continue;
+ if (omap_dma_chan_read(c, CCR) & CCR_ENABLE)
+ return true;
+ }
+
+ return false;
+}
+
/* Currently only used for omap2. For omap1, also a check for lcd_dma is needed */
static int omap_dma_busy_notifier(struct notifier_block *nb,
unsigned long cmd, void *v)
{
struct omap_dmadev *od;
- struct omap_chan *c;
- int lch = -1;
od = container_of(nb, struct omap_dmadev, nb);
switch (cmd) {
case CPU_CLUSTER_PM_ENTER:
- while (1) {
- lch = find_next_bit(od->lch_bitmap, od->lch_count,
- lch + 1);
- if (lch >= od->lch_count)
- break;
- c = od->lch_map[lch];
- if (!c)
- continue;
- if (omap_dma_chan_read(c, CCR) & CCR_ENABLE)
- return NOTIFY_BAD;
- }
+ if (omap_dma_busy(od))
+ return NOTIFY_BAD;
break;
case CPU_CLUSTER_PM_ENTER_FAILED:
case CPU_CLUSTER_PM_EXIT:
switch (cmd) {
case CPU_CLUSTER_PM_ENTER:
+ if (omap_dma_busy(od))
+ return NOTIFY_BAD;
omap_dma_context_save(od);
break;
case CPU_CLUSTER_PM_ENTER_FAILED:
#define to_dma_tx_descriptor(tx) \
container_of(tx, struct xilinx_dma_tx_descriptor, async_tx)
#define xilinx_dma_poll_timeout(chan, reg, val, cond, delay_us, timeout_us) \
- readl_poll_timeout(chan->xdev->regs + chan->ctrl_offset + reg, val, \
- cond, delay_us, timeout_us)
+ readl_poll_timeout_atomic(chan->xdev->regs + chan->ctrl_offset + reg, \
+ val, cond, delay_us, timeout_us)
/* IO accessors */
static inline u32 dma_read(struct xilinx_dma_chan *chan, u32 reg)
{
struct xilinx_cdma_tx_segment *cdma_seg;
struct xilinx_axidma_tx_segment *axidma_seg;
+ struct xilinx_aximcdma_tx_segment *aximcdma_seg;
struct xilinx_cdma_desc_hw *cdma_hw;
struct xilinx_axidma_desc_hw *axidma_hw;
+ struct xilinx_aximcdma_desc_hw *aximcdma_hw;
struct list_head *entry;
u32 residue = 0;
cdma_hw = &cdma_seg->hw;
residue += (cdma_hw->control - cdma_hw->status) &
chan->xdev->max_buffer_len;
- } else {
+ } else if (chan->xdev->dma_config->dmatype ==
+ XDMA_TYPE_AXIDMA) {
axidma_seg = list_entry(entry,
struct xilinx_axidma_tx_segment,
node);
axidma_hw = &axidma_seg->hw;
residue += (axidma_hw->control - axidma_hw->status) &
chan->xdev->max_buffer_len;
+ } else {
+ aximcdma_seg =
+ list_entry(entry,
+ struct xilinx_aximcdma_tx_segment,
+ node);
+ aximcdma_hw = &aximcdma_seg->hw;
+ residue +=
+ (aximcdma_hw->control - aximcdma_hw->status) &
+ chan->xdev->max_buffer_len;
}
}
upper_32_bits(chan->seg_p + sizeof(*chan->seg_mv) *
((i + 1) % XILINX_DMA_NUM_DESCS));
chan->seg_mv[i].phys = chan->seg_p +
- sizeof(*chan->seg_v) * i;
+ sizeof(*chan->seg_mv) * i;
list_add_tail(&chan->seg_mv[i].node,
&chan->free_seg_list);
}
static void xilinx_mcdma_start_transfer(struct xilinx_dma_chan *chan)
{
struct xilinx_dma_tx_descriptor *head_desc, *tail_desc;
- struct xilinx_axidma_tx_segment *tail_segment;
+ struct xilinx_aximcdma_tx_segment *tail_segment;
u32 reg;
/*
tail_desc = list_last_entry(&chan->pending_list,
struct xilinx_dma_tx_descriptor, node);
tail_segment = list_last_entry(&tail_desc->segments,
- struct xilinx_axidma_tx_segment, node);
+ struct xilinx_aximcdma_tx_segment, node);
reg = dma_ctrl_read(chan, XILINX_MCDMA_CHAN_CR_OFFSET(chan->tdest));
struct xilinx_vdma_tx_segment *tail_segment;
struct xilinx_dma_tx_descriptor *tail_desc;
struct xilinx_axidma_tx_segment *axidma_tail_segment;
+ struct xilinx_aximcdma_tx_segment *aximcdma_tail_segment;
struct xilinx_cdma_tx_segment *cdma_tail_segment;
if (list_empty(&chan->pending_list))
struct xilinx_cdma_tx_segment,
node);
cdma_tail_segment->hw.next_desc = (u32)desc->async_tx.phys;
- } else {
+ } else if (chan->xdev->dma_config->dmatype == XDMA_TYPE_AXIDMA) {
axidma_tail_segment = list_last_entry(&tail_desc->segments,
struct xilinx_axidma_tx_segment,
node);
axidma_tail_segment->hw.next_desc = (u32)desc->async_tx.phys;
+ } else {
+ aximcdma_tail_segment =
+ list_last_entry(&tail_desc->segments,
+ struct xilinx_aximcdma_tx_segment,
+ node);
+ aximcdma_tail_segment->hw.next_desc = (u32)desc->async_tx.phys;
}
/*
chan->stop_transfer = xilinx_dma_stop_transfer;
}
- /* check if SG is enabled (only for AXIDMA and CDMA) */
+ /* check if SG is enabled (only for AXIDMA, AXIMCDMA, and CDMA) */
if (xdev->dma_config->dmatype != XDMA_TYPE_VDMA) {
- if (dma_ctrl_read(chan, XILINX_DMA_REG_DMASR) &
- XILINX_DMA_DMASR_SG_MASK)
+ if (xdev->dma_config->dmatype == XDMA_TYPE_AXIMCDMA ||
+ dma_ctrl_read(chan, XILINX_DMA_REG_DMASR) &
+ XILINX_DMA_DMASR_SG_MASK)
chan->has_sg = true;
dev_dbg(chan->dev, "ch %d: SG %s\n", chan->id,
chan->has_sg ? "enabled" : "disabled");
config EFI_EARLYCON
def_bool y
- depends on SERIAL_EARLYCON && !ARM && !IA64
+ depends on EFI && SERIAL_EARLYCON && !ARM && !IA64
select FONT_SUPPORT
select ARCH_USE_MEMREMAP_PROT
if (efi_rt_services_supported(EFI_RT_SUPPORTED_GET_VARIABLE |
EFI_RT_SUPPORTED_GET_NEXT_VARIABLE_NAME)) {
- efivar_ssdt_load();
error = generic_ops_register();
if (error)
goto err_put;
+ efivar_ssdt_load();
platform_device_register_simple("efivars", 0, NULL, 0);
}
#include <linux/of_platform.h>
#include <linux/slab.h>
#include <linux/uaccess.h>
+#include <linux/hashtable.h>
#include <linux/firmware/xlnx-zynqmp.h>
#include "zynqmp-debug.h"
+/* Max HashMap Order for PM API feature check (1<<7 = 128) */
+#define PM_API_FEATURE_CHECK_MAX_ORDER 7
+
static bool feature_check_enabled;
-static u32 zynqmp_pm_features[PM_API_MAX];
+DEFINE_HASHTABLE(pm_api_features_map, PM_API_FEATURE_CHECK_MAX_ORDER);
+
+/**
+ * struct pm_api_feature_data - PM API Feature data
+ * @pm_api_id: PM API Id, used as key to index into hashmap
+ * @feature_status: status of PM API feature: valid, invalid
+ * @hentry: hlist_node that hooks this entry into hashtable
+ */
+struct pm_api_feature_data {
+ u32 pm_api_id;
+ int feature_status;
+ struct hlist_node hentry;
+};
static const struct mfd_cell firmware_devs[] = {
{
int ret;
u32 ret_payload[PAYLOAD_ARG_CNT];
u64 smc_arg[2];
+ struct pm_api_feature_data *feature_data;
if (!feature_check_enabled)
return 0;
- /* Return value if feature is already checked */
- if (zynqmp_pm_features[api_id] != PM_FEATURE_UNCHECKED)
- return zynqmp_pm_features[api_id];
+ /* Check for existing entry in hash table for given api */
+ hash_for_each_possible(pm_api_features_map, feature_data, hentry,
+ api_id) {
+ if (feature_data->pm_api_id == api_id)
+ return feature_data->feature_status;
+ }
+
+ /* Add new entry if not present */
+ feature_data = kmalloc(sizeof(*feature_data), GFP_KERNEL);
+ if (!feature_data)
+ return -ENOMEM;
+ feature_data->pm_api_id = api_id;
smc_arg[0] = PM_SIP_SVC | PM_FEATURE_CHECK;
smc_arg[1] = api_id;
ret = do_fw_call(smc_arg[0], smc_arg[1], 0, ret_payload);
- if (ret) {
- zynqmp_pm_features[api_id] = PM_FEATURE_INVALID;
- return PM_FEATURE_INVALID;
- }
+ if (ret)
+ ret = -EOPNOTSUPP;
+ else
+ ret = ret_payload[1];
- zynqmp_pm_features[api_id] = ret_payload[1];
+ feature_data->feature_status = ret;
+ hash_add(pm_api_features_map, &feature_data->hentry, api_id);
- return zynqmp_pm_features[api_id];
+ return ret;
}
/**
* Make sure to stay in x0 register
*/
u64 smc_arg[4];
+ int ret;
- if (zynqmp_pm_feature(pm_api_id) == PM_FEATURE_INVALID)
- return -ENOTSUPP;
+ /* Check if feature is supported or not */
+ ret = zynqmp_pm_feature(pm_api_id);
+ if (ret < 0)
+ return ret;
smc_arg[0] = PM_SIP_SVC | pm_api_id;
smc_arg[1] = ((u64)arg1 << 32) | arg0;
*/
int zynqmp_pm_sd_dll_reset(u32 node_id, u32 type)
{
- return zynqmp_pm_invoke_fn(PM_IOCTL, node_id, IOCTL_SET_SD_TAPDELAY,
+ return zynqmp_pm_invoke_fn(PM_IOCTL, node_id, IOCTL_SD_DLL_RESET,
type, 0, NULL);
}
EXPORT_SYMBOL_GPL(zynqmp_pm_sd_dll_reset);
static int zynqmp_firmware_remove(struct platform_device *pdev)
{
+ struct pm_api_feature_data *feature_data;
+ int i;
+
mfd_remove_devices(&pdev->dev);
zynqmp_pm_api_debugfs_exit();
+ hash_for_each(pm_api_features_map, i, feature_data, hentry) {
+ hash_del(&feature_data->hentry);
+ kfree(feature_data);
+ }
+
return 0;
}
static const struct aspeed_bank_props ast2600_bank_props[] = {
/* input output */
+ {4, 0xffffffff, 0x00ffffff}, /* Q/R/S/T */
{5, 0xffffffff, 0xffffff00}, /* U/V/W/X */
{6, 0x0000ffff, 0x0000ffff}, /* Y/Z */
{ },
#ifdef CONFIG_PM_SLEEP
static int dwapb_irq_set_wake(struct irq_data *d, unsigned int enable)
{
- struct irq_chip_generic *igc = irq_data_get_irq_chip_data(d);
- struct dwapb_gpio *gpio = igc->private;
+ struct gpio_chip *gc = irq_data_get_irq_chip_data(d);
+ struct dwapb_gpio *gpio = to_dwapb_gpio(gc);
struct dwapb_context *ctx = gpio->ports[0].ctx;
irq_hw_number_t bit = irqd_to_hwirq(d);
{
struct device *dev = bank->chip.parent;
void __iomem *base = bank->base;
- u32 nowake;
+ u32 mask, nowake;
bank->saved_datain = readl_relaxed(base + bank->regs->datain);
if (!bank->enabled_non_wakeup_gpios)
goto update_gpio_context_count;
+ /* Check for pending EDGE_FALLING, ignore EDGE_BOTH */
+ mask = bank->enabled_non_wakeup_gpios & bank->context.fallingdetect;
+ mask &= ~bank->context.risingdetect;
+ bank->saved_datain |= mask;
+
+ /* Check for pending EDGE_RISING, ignore EDGE_BOTH */
+ mask = bank->enabled_non_wakeup_gpios & bank->context.risingdetect;
+ mask &= ~bank->context.fallingdetect;
+ bank->saved_datain &= ~mask;
+
if (!may_lose_context)
goto update_gpio_context_count;
#include <linux/spinlock.h>
#include <linux/types.h>
+/*
+ * PLX PEX8311 PCI LCS_INTCSR Interrupt Control/Status
+ *
+ * Bit: Description
+ * 0: Enable Interrupt Sources (Bit 0)
+ * 1: Enable Interrupt Sources (Bit 1)
+ * 2: Generate Internal PCI Bus Internal SERR# Interrupt
+ * 3: Mailbox Interrupt Enable
+ * 4: Power Management Interrupt Enable
+ * 5: Power Management Interrupt
+ * 6: Slave Read Local Data Parity Check Error Enable
+ * 7: Slave Read Local Data Parity Check Error Status
+ * 8: Internal PCI Wire Interrupt Enable
+ * 9: PCI Express Doorbell Interrupt Enable
+ * 10: PCI Abort Interrupt Enable
+ * 11: Local Interrupt Input Enable
+ * 12: Retry Abort Enable
+ * 13: PCI Express Doorbell Interrupt Active
+ * 14: PCI Abort Interrupt Active
+ * 15: Local Interrupt Input Active
+ * 16: Local Interrupt Output Enable
+ * 17: Local Doorbell Interrupt Enable
+ * 18: DMA Channel 0 Interrupt Enable
+ * 19: DMA Channel 1 Interrupt Enable
+ * 20: Local Doorbell Interrupt Active
+ * 21: DMA Channel 0 Interrupt Active
+ * 22: DMA Channel 1 Interrupt Active
+ * 23: Built-In Self-Test (BIST) Interrupt Active
+ * 24: Direct Master was the Bus Master during a Master or Target Abort
+ * 25: DMA Channel 0 was the Bus Master during a Master or Target Abort
+ * 26: DMA Channel 1 was the Bus Master during a Master or Target Abort
+ * 27: Target Abort after internal 256 consecutive Master Retrys
+ * 28: PCI Bus wrote data to LCS_MBOX0
+ * 29: PCI Bus wrote data to LCS_MBOX1
+ * 30: PCI Bus wrote data to LCS_MBOX2
+ * 31: PCI Bus wrote data to LCS_MBOX3
+ */
+#define PLX_PEX8311_PCI_LCS_INTCSR 0x68
+#define INTCSR_INTERNAL_PCI_WIRE BIT(8)
+#define INTCSR_LOCAL_INPUT BIT(11)
+
/**
* struct idio_24_gpio_reg - GPIO device registers structure
* @out0_7: Read: FET Outputs 0-7
struct idio_24_gpio {
struct gpio_chip chip;
raw_spinlock_t lock;
+ __u8 __iomem *plx;
struct idio_24_gpio_reg __iomem *reg;
unsigned long irq_mask;
};
unsigned long flags;
const unsigned long bit_offset = irqd_to_hwirq(data) - 24;
unsigned char new_irq_mask;
- const unsigned long bank_offset = bit_offset/8 * 8;
+ const unsigned long bank_offset = bit_offset / 8;
unsigned char cos_enable_state;
raw_spin_lock_irqsave(&idio24gpio->lock, flags);
- idio24gpio->irq_mask &= BIT(bit_offset);
- new_irq_mask = idio24gpio->irq_mask >> bank_offset;
+ idio24gpio->irq_mask &= ~BIT(bit_offset);
+ new_irq_mask = idio24gpio->irq_mask >> bank_offset * 8;
if (!new_irq_mask) {
cos_enable_state = ioread8(&idio24gpio->reg->cos_enable);
unsigned long flags;
unsigned char prev_irq_mask;
const unsigned long bit_offset = irqd_to_hwirq(data) - 24;
- const unsigned long bank_offset = bit_offset/8 * 8;
+ const unsigned long bank_offset = bit_offset / 8;
unsigned char cos_enable_state;
raw_spin_lock_irqsave(&idio24gpio->lock, flags);
- prev_irq_mask = idio24gpio->irq_mask >> bank_offset;
+ prev_irq_mask = idio24gpio->irq_mask >> bank_offset * 8;
idio24gpio->irq_mask |= BIT(bit_offset);
if (!prev_irq_mask) {
struct device *const dev = &pdev->dev;
struct idio_24_gpio *idio24gpio;
int err;
+ const size_t pci_plx_bar_index = 1;
const size_t pci_bar_index = 2;
const char *const name = pci_name(pdev);
struct gpio_irq_chip *girq;
return err;
}
- err = pcim_iomap_regions(pdev, BIT(pci_bar_index), name);
+ err = pcim_iomap_regions(pdev, BIT(pci_plx_bar_index) | BIT(pci_bar_index), name);
if (err) {
dev_err(dev, "Unable to map PCI I/O addresses (%d)\n", err);
return err;
}
+ idio24gpio->plx = pcim_iomap_table(pdev)[pci_plx_bar_index];
idio24gpio->reg = pcim_iomap_table(pdev)[pci_bar_index];
idio24gpio->chip.label = name;
/* Software board reset */
iowrite8(0, &idio24gpio->reg->soft_reset);
+ /*
+ * enable PLX PEX8311 internal PCI wire interrupt and local interrupt
+ * input
+ */
+ iowrite8((INTCSR_INTERNAL_PCI_WIRE | INTCSR_LOCAL_INPUT) >> 8,
+ idio24gpio->plx + PLX_PEX8311_PCI_LCS_INTCSR + 1);
err = devm_gpiochip_add_data(dev, &idio24gpio->chip, idio24gpio);
if (err) {
return PTR_ERR(chip->regs);
ngpio = of_irq_count(node);
- if (ngpio >= SIFIVE_GPIO_MAX) {
+ if (ngpio > SIFIVE_GPIO_MAX) {
dev_err(dev, "Too many GPIO interrupts (max=%d)\n",
SIFIVE_GPIO_MAX);
return -ENXIO;
struct gpio_device;
-#ifdef CONFIG_GPIO_CDEV
-
int gpiolib_cdev_register(struct gpio_device *gdev, dev_t devt);
void gpiolib_cdev_unregister(struct gpio_device *gdev);
-#else
-
-static inline int gpiolib_cdev_register(struct gpio_device *gdev, dev_t devt)
-{
- return 0;
-}
-
-static inline void gpiolib_cdev_unregister(struct gpio_device *gdev)
-{
-}
-
-#endif /* CONFIG_GPIO_CDEV */
-
#endif /* GPIOLIB_CDEV_H */
kfree(gdev);
}
+#ifdef CONFIG_GPIO_CDEV
+#define gcdev_register(gdev, devt) gpiolib_cdev_register((gdev), (devt))
+#define gcdev_unregister(gdev) gpiolib_cdev_unregister((gdev))
+#else
+/*
+ * gpiolib_cdev_register() indirectly calls device_add(), which is still
+ * required even when cdev is not selected.
+ */
+#define gcdev_register(gdev, devt) device_add(&(gdev)->dev)
+#define gcdev_unregister(gdev) device_del(&(gdev)->dev)
+#endif
+
static int gpiochip_setup_dev(struct gpio_device *gdev)
{
int ret;
- ret = gpiolib_cdev_register(gdev, gpio_devt);
+ ret = gcdev_register(gdev, gpio_devt);
if (ret)
return ret;
return 0;
err_remove_device:
- gpiolib_cdev_unregister(gdev);
+ gcdev_unregister(gdev);
return ret;
}
* be removed, else it will be dangling until the last user is
* gone.
*/
- gpiolib_cdev_unregister(gdev);
+ gcdev_unregister(gdev);
put_device(&gdev->dev);
}
EXPORT_SYMBOL_GPL(gpiochip_remove);
if (!amdgpu_device_supports_baco(adev_to_drm(adev)))
return -ENOTSUPP;
- if (ras && ras->supported)
+ if (ras && ras->supported && adev->nbio.funcs->enable_doorbell_interrupt)
adev->nbio.funcs->enable_doorbell_interrupt(adev, false);
return amdgpu_dpm_baco_enter(adev);
if (ret)
return ret;
- if (ras && ras->supported)
+ if (ras && ras->supported && adev->nbio.funcs->enable_doorbell_interrupt)
adev->nbio.funcs->enable_doorbell_interrupt(adev, true);
return 0;
{0x1002, 0x15dd, PCI_ANY_ID, PCI_ANY_ID, 0, 0, CHIP_RAVEN|AMD_IS_APU},
{0x1002, 0x15d8, PCI_ANY_ID, PCI_ANY_ID, 0, 0, CHIP_RAVEN|AMD_IS_APU},
/* Arcturus */
- {0x1002, 0x738C, PCI_ANY_ID, PCI_ANY_ID, 0, 0, CHIP_ARCTURUS|AMD_EXP_HW_SUPPORT},
- {0x1002, 0x7388, PCI_ANY_ID, PCI_ANY_ID, 0, 0, CHIP_ARCTURUS|AMD_EXP_HW_SUPPORT},
- {0x1002, 0x738E, PCI_ANY_ID, PCI_ANY_ID, 0, 0, CHIP_ARCTURUS|AMD_EXP_HW_SUPPORT},
- {0x1002, 0x7390, PCI_ANY_ID, PCI_ANY_ID, 0, 0, CHIP_ARCTURUS|AMD_EXP_HW_SUPPORT},
+ {0x1002, 0x738C, PCI_ANY_ID, PCI_ANY_ID, 0, 0, CHIP_ARCTURUS},
+ {0x1002, 0x7388, PCI_ANY_ID, PCI_ANY_ID, 0, 0, CHIP_ARCTURUS},
+ {0x1002, 0x738E, PCI_ANY_ID, PCI_ANY_ID, 0, 0, CHIP_ARCTURUS},
+ {0x1002, 0x7390, PCI_ANY_ID, PCI_ANY_ID, 0, 0, CHIP_ARCTURUS},
/* Navi10 */
{0x1002, 0x7310, PCI_ANY_ID, PCI_ANY_ID, 0, 0, CHIP_NAVI10},
{0x1002, 0x7312, PCI_ANY_ID, PCI_ANY_ID, 0, 0, CHIP_NAVI10},
static int amdgpu_ttm_init_on_chip(struct amdgpu_device *adev,
unsigned int type,
- uint64_t size)
+ uint64_t size_in_page)
{
return ttm_range_man_init(&adev->mman.bdev, type,
- false, size >> PAGE_SHIFT);
+ false, size_in_page);
}
/**
unsigned harvest_config;
/* store image width to adjust nb memory state */
unsigned decode_image_width;
+ uint32_t keyselect;
};
int amdgpu_uvd_sw_init(struct amdgpu_device *adev);
SOC15_REG_GOLDEN_VALUE(GC, 0, mmDB_DEBUG3, 0xffffffff, 0x00000280),
SOC15_REG_GOLDEN_VALUE(GC, 0, mmDB_DEBUG4, 0xffffffff, 0x00800000),
SOC15_REG_GOLDEN_VALUE(GC, 0, mmDB_EXCEPTION_CONTROL, 0x7fff0f1f, 0x00b80000),
+ SOC15_REG_GOLDEN_VALUE(GC, 0 ,mmGCEA_SDP_TAG_RESERVE0, 0xffffffff, 0x10100100),
+ SOC15_REG_GOLDEN_VALUE(GC, 0, mmGCEA_SDP_TAG_RESERVE1, 0xffffffff, 0x17000088),
SOC15_REG_GOLDEN_VALUE(GC, 0, mmGCR_GENERAL_CNTL_Sienna_Cichlid, 0x1ff1ffff, 0x00000500),
SOC15_REG_GOLDEN_VALUE(GC, 0, mmGE_PC_CNTL, 0x003fffff, 0x00280400),
SOC15_REG_GOLDEN_VALUE(GC, 0, mmGL2A_ADDR_MATCH_MASK, 0xffffffff, 0xffffffcf),
if (adev->enable_virtual_display || amdgpu_sriov_vf(adev))
amdgpu_device_ip_block_add(adev, &dce_virtual_ip_block);
#if defined(CONFIG_DRM_AMD_DC)
- else if (amdgpu_device_has_dc_support(adev) &&
- !nv_is_headless_sku(adev->pdev))
+ else if (amdgpu_device_has_dc_support(adev))
amdgpu_device_ip_block_add(adev, &dm_ip_block);
#endif
amdgpu_device_ip_block_add(adev, &gfx_v10_0_ip_block);
MODULE_FIRMWARE("amdgpu/renoir_asd.bin");
MODULE_FIRMWARE("amdgpu/renoir_ta.bin");
MODULE_FIRMWARE("amdgpu/green_sardine_asd.bin");
+MODULE_FIRMWARE("amdgpu/green_sardine_ta.bin");
/* address block */
#define smnMP1_FIRMWARE_FLAGS 0x3010024
*/
static int uvd_v3_1_fw_validate(struct amdgpu_device *adev)
{
- void *ptr;
- uint32_t ucode_len, i;
- uint32_t keysel;
-
- ptr = adev->uvd.inst[0].cpu_addr;
- ptr += 192 + 16;
- memcpy(&ucode_len, ptr, 4);
- ptr += ucode_len;
- memcpy(&keysel, ptr, 4);
+ int i;
+ uint32_t keysel = adev->uvd.keyselect;
WREG32(mmUVD_FW_START, keysel);
struct amdgpu_ring *ring;
struct amdgpu_device *adev = (struct amdgpu_device *)handle;
int r;
+ void *ptr;
+ uint32_t ucode_len;
/* UVD TRAP */
r = amdgpu_irq_add_id(adev, AMDGPU_IRQ_CLIENTID_LEGACY, 124, &adev->uvd.inst->irq);
if (r)
return r;
+ /* Retrieval firmware validate key */
+ ptr = adev->uvd.inst[0].cpu_addr;
+ ptr += 192 + 16;
+ memcpy(&ucode_len, ptr, 4);
+ ptr += ucode_len;
+ memcpy(&adev->uvd.keyselect, ptr, 4);
+
r = amdgpu_uvd_entity_init(adev);
return r;
amdgpu_dm_init_color_mod();
#ifdef CONFIG_DRM_AMD_DC_HDCP
- if (adev->asic_type >= CHIP_RAVEN) {
+ if (adev->dm.dc->caps.max_links > 0 && adev->asic_type >= CHIP_RAVEN) {
adev->dm.hdcp_workqueue = hdcp_create_workqueue(adev, &init_params.cp_psp, adev->dm.dc);
if (!adev->dm.hdcp_workqueue)
bool mode_set_reset_required = false;
drm_atomic_helper_update_legacy_modeset_state(dev, state);
- drm_atomic_helper_calc_timestamping_constants(state);
dm_state = dm_atomic_get_new_state(state);
if (dm_state && dm_state->context) {
}
}
+ drm_atomic_helper_calc_timestamping_constants(state);
+
/* update changed items */
for_each_oldnew_crtc_in_state(state, crtc, old_crtc_state, new_crtc_state, i) {
struct amdgpu_crtc *acrtc = to_amdgpu_crtc(crtc);
pflip_int_entry(1),
pflip_int_entry(2),
pflip_int_entry(3),
- [DC_IRQ_SOURCE_PFLIP5] = dummy_irq_entry(),
- [DC_IRQ_SOURCE_PFLIP6] = dummy_irq_entry(),
+ pflip_int_entry(4),
+ pflip_int_entry(5),
[DC_IRQ_SOURCE_PFLIP_UNDERLAY0] = dummy_irq_entry(),
gpio_pad_int_entry(0),
gpio_pad_int_entry(1),
pflip_int_entry(1),
pflip_int_entry(2),
pflip_int_entry(3),
- [DC_IRQ_SOURCE_PFLIP5] = dummy_irq_entry(),
- [DC_IRQ_SOURCE_PFLIP6] = dummy_irq_entry(),
+ pflip_int_entry(4),
+ pflip_int_entry(5),
[DC_IRQ_SOURCE_PFLIP_UNDERLAY0] = dummy_irq_entry(),
gpio_pad_int_entry(0),
gpio_pad_int_entry(1),
case DRM_MODE_DPMS_SUSPEND:
if (ast->tx_chip_type == AST_TX_DP501)
ast_set_dp501_video_output(crtc->dev, 1);
- ast_crtc_load_lut(ast, crtc);
break;
case DRM_MODE_DPMS_OFF:
if (ast->tx_chip_type == AST_TX_DP501)
return 0;
}
+static void
+ast_crtc_helper_atomic_flush(struct drm_crtc *crtc, struct drm_crtc_state *old_crtc_state)
+{
+ struct ast_private *ast = to_ast_private(crtc->dev);
+ struct ast_crtc_state *ast_crtc_state = to_ast_crtc_state(crtc->state);
+ struct ast_crtc_state *old_ast_crtc_state = to_ast_crtc_state(old_crtc_state);
+
+ /*
+ * The gamma LUT has to be reloaded after changing the primary
+ * plane's color format.
+ */
+ if (old_ast_crtc_state->format != ast_crtc_state->format)
+ ast_crtc_load_lut(ast, crtc);
+}
+
static void
ast_crtc_helper_atomic_enable(struct drm_crtc *crtc,
struct drm_crtc_state *old_crtc_state)
static const struct drm_crtc_helper_funcs ast_crtc_helper_funcs = {
.atomic_check = ast_crtc_helper_atomic_check,
+ .atomic_flush = ast_crtc_helper_atomic_flush,
.atomic_enable = ast_crtc_helper_atomic_enable,
.atomic_disable = ast_crtc_helper_atomic_disable,
};
if DRM_CDNS_MHDP8546
config DRM_CDNS_MHDP8546_J721E
- depends on ARCH_K3_J721E_SOC || COMPILE_TEST
+ depends on ARCH_K3 || COMPILE_TEST
bool "J721E Cadence DPI/DP wrapper support"
default y
help
{
enum drm_connector_status result;
- mutex_lock(&hdmi->mutex);
- hdmi->force = DRM_FORCE_UNSPECIFIED;
- dw_hdmi_update_power(hdmi);
- dw_hdmi_update_phy_mask(hdmi);
- mutex_unlock(&hdmi->mutex);
-
result = hdmi->phy.ops->read_hpd(hdmi, hdmi->phy.data);
mutex_lock(&hdmi->mutex);
unsigned int c = 0;
if (pl_flag & DRM_GEM_VRAM_PL_FLAG_TOPDOWN)
- pl_flag = TTM_PL_FLAG_TOPDOWN;
+ invariant_flags = TTM_PL_FLAG_TOPDOWN;
gbo->placement.placement = gbo->placements;
gbo->placement.busy_placement = gbo->placements;
# SPDX-License-Identifier: GPL-2.0-only
config DRM_EXYNOS
tristate "DRM Support for Samsung SoC Exynos Series"
- depends on OF && DRM && (ARCH_S3C64XX || ARCH_S5PV210 || ARCH_EXYNOS || ARCH_MULTIPLATFORM || COMPILE_TEST)
+ depends on OF && DRM && COMMON_CLK
+ depends on ARCH_S3C64XX || ARCH_S5PV210 || ARCH_EXYNOS || ARCH_MULTIPLATFORM || COMPILE_TEST
depends on MMU
select DRM_KMS_HELPER
select VIDEOMODE_HELPERS
{
struct drm_psb_private *dev_priv = dev->dev_private;
unsigned long irqflags;
+ unsigned int i;
spin_lock_irqsave(&dev_priv->irqmask_lock, irqflags);
PSB_WVDC32(dev_priv->vdc_irq_mask, PSB_INT_ENABLE_R);
PSB_WVDC32(0xFFFFFFFF, PSB_HWSTAM);
- if (dev->vblank[0].enabled)
- psb_enable_pipestat(dev_priv, 0, PIPE_VBLANK_INTERRUPT_ENABLE);
- else
- psb_disable_pipestat(dev_priv, 0, PIPE_VBLANK_INTERRUPT_ENABLE);
-
- if (dev->vblank[1].enabled)
- psb_enable_pipestat(dev_priv, 1, PIPE_VBLANK_INTERRUPT_ENABLE);
- else
- psb_disable_pipestat(dev_priv, 1, PIPE_VBLANK_INTERRUPT_ENABLE);
-
- if (dev->vblank[2].enabled)
- psb_enable_pipestat(dev_priv, 2, PIPE_VBLANK_INTERRUPT_ENABLE);
- else
- psb_disable_pipestat(dev_priv, 2, PIPE_VBLANK_INTERRUPT_ENABLE);
+ for (i = 0; i < dev->num_crtcs; ++i) {
+ if (dev->vblank[i].enabled)
+ psb_enable_pipestat(dev_priv, i, PIPE_VBLANK_INTERRUPT_ENABLE);
+ else
+ psb_disable_pipestat(dev_priv, i, PIPE_VBLANK_INTERRUPT_ENABLE);
+ }
if (dev_priv->ops->hotplug_enable)
dev_priv->ops->hotplug_enable(dev, true);
{
struct drm_psb_private *dev_priv = dev->dev_private;
unsigned long irqflags;
+ unsigned int i;
spin_lock_irqsave(&dev_priv->irqmask_lock, irqflags);
PSB_WVDC32(0xFFFFFFFF, PSB_HWSTAM);
- if (dev->vblank[0].enabled)
- psb_disable_pipestat(dev_priv, 0, PIPE_VBLANK_INTERRUPT_ENABLE);
-
- if (dev->vblank[1].enabled)
- psb_disable_pipestat(dev_priv, 1, PIPE_VBLANK_INTERRUPT_ENABLE);
-
- if (dev->vblank[2].enabled)
- psb_disable_pipestat(dev_priv, 2, PIPE_VBLANK_INTERRUPT_ENABLE);
+ for (i = 0; i < dev->num_crtcs; ++i) {
+ if (dev->vblank[i].enabled)
+ psb_disable_pipestat(dev_priv, i, PIPE_VBLANK_INTERRUPT_ENABLE);
+ }
dev_priv->vdc_irq_mask &= _PSB_IRQ_SGX_FLAG |
_PSB_IRQ_MSVDX_FLAG |
case 10 ... 11:
bpp = 10 * 3;
break;
- case 12:
+ case 12 ... 16:
bpp = 12 * 3;
break;
default:
+ MISSING_CASE(conn_state->max_bpc);
return -EINVAL;
}
void (*truncate)(struct drm_i915_gem_object *obj);
void (*writeback)(struct drm_i915_gem_object *obj);
+ int (*pread)(struct drm_i915_gem_object *obj,
+ const struct drm_i915_gem_pread *arg);
int (*pwrite)(struct drm_i915_gem_object *obj,
const struct drm_i915_gem_pwrite *arg);
vaddr, dma);
}
+static int
+phys_pwrite(struct drm_i915_gem_object *obj,
+ const struct drm_i915_gem_pwrite *args)
+{
+ void *vaddr = sg_page(obj->mm.pages->sgl) + args->offset;
+ char __user *user_data = u64_to_user_ptr(args->data_ptr);
+ int err;
+
+ err = i915_gem_object_wait(obj,
+ I915_WAIT_INTERRUPTIBLE |
+ I915_WAIT_ALL,
+ MAX_SCHEDULE_TIMEOUT);
+ if (err)
+ return err;
+
+ /*
+ * We manually control the domain here and pretend that it
+ * remains coherent i.e. in the GTT domain, like shmem_pwrite.
+ */
+ i915_gem_object_invalidate_frontbuffer(obj, ORIGIN_CPU);
+
+ if (copy_from_user(vaddr, user_data, args->size))
+ return -EFAULT;
+
+ drm_clflush_virt_range(vaddr, args->size);
+ intel_gt_chipset_flush(&to_i915(obj->base.dev)->gt);
+
+ i915_gem_object_flush_frontbuffer(obj, ORIGIN_CPU);
+ return 0;
+}
+
+static int
+phys_pread(struct drm_i915_gem_object *obj,
+ const struct drm_i915_gem_pread *args)
+{
+ void *vaddr = sg_page(obj->mm.pages->sgl) + args->offset;
+ char __user *user_data = u64_to_user_ptr(args->data_ptr);
+ int err;
+
+ err = i915_gem_object_wait(obj,
+ I915_WAIT_INTERRUPTIBLE,
+ MAX_SCHEDULE_TIMEOUT);
+ if (err)
+ return err;
+
+ drm_clflush_virt_range(vaddr, args->size);
+ if (copy_to_user(user_data, vaddr, args->size))
+ return -EFAULT;
+
+ return 0;
+}
+
static void phys_release(struct drm_i915_gem_object *obj)
{
fput(obj->base.filp);
.get_pages = i915_gem_object_get_pages_phys,
.put_pages = i915_gem_object_put_pages_phys,
+ .pread = phys_pread,
+ .pwrite = phys_pwrite,
+
.release = phys_release,
};
#include "i915_trace.h"
#include "intel_breadcrumbs.h"
#include "intel_context.h"
+#include "intel_engine_pm.h"
#include "intel_gt_pm.h"
#include "intel_gt_requests.h"
-static void irq_enable(struct intel_engine_cs *engine)
+static bool irq_enable(struct intel_engine_cs *engine)
{
if (!engine->irq_enable)
- return;
+ return false;
/* Caller disables interrupts */
spin_lock(&engine->gt->irq_lock);
engine->irq_enable(engine);
spin_unlock(&engine->gt->irq_lock);
+
+ return true;
}
static void irq_disable(struct intel_engine_cs *engine)
static void __intel_breadcrumbs_arm_irq(struct intel_breadcrumbs *b)
{
- lockdep_assert_held(&b->irq_lock);
-
- if (!b->irq_engine || b->irq_armed)
- return;
-
- if (!intel_gt_pm_get_if_awake(b->irq_engine->gt))
+ /*
+ * Since we are waiting on a request, the GPU should be busy
+ * and should have its own rpm reference.
+ */
+ if (GEM_WARN_ON(!intel_gt_pm_get_if_awake(b->irq_engine->gt)))
return;
/*
*/
WRITE_ONCE(b->irq_armed, true);
- /*
- * Since we are waiting on a request, the GPU should be busy
- * and should have its own rpm reference. This is tracked
- * by i915->gt.awake, we can forgo holding our own wakref
- * for the interrupt as before i915->gt.awake is released (when
- * the driver is idle) we disarm the breadcrumbs.
- */
-
- if (!b->irq_enabled++)
- irq_enable(b->irq_engine);
+ /* Requests may have completed before we could enable the interrupt. */
+ if (!b->irq_enabled++ && irq_enable(b->irq_engine))
+ irq_work_queue(&b->irq_work);
}
-static void __intel_breadcrumbs_disarm_irq(struct intel_breadcrumbs *b)
+static void intel_breadcrumbs_arm_irq(struct intel_breadcrumbs *b)
{
- lockdep_assert_held(&b->irq_lock);
-
- if (!b->irq_engine || !b->irq_armed)
+ if (!b->irq_engine)
return;
+ spin_lock(&b->irq_lock);
+ if (!b->irq_armed)
+ __intel_breadcrumbs_arm_irq(b);
+ spin_unlock(&b->irq_lock);
+}
+
+static void __intel_breadcrumbs_disarm_irq(struct intel_breadcrumbs *b)
+{
GEM_BUG_ON(!b->irq_enabled);
if (!--b->irq_enabled)
irq_disable(b->irq_engine);
{
intel_context_get(ce);
list_add_tail(&ce->signal_link, &b->signalers);
- if (list_is_first(&ce->signal_link, &b->signalers))
- __intel_breadcrumbs_arm_irq(b);
}
static void remove_signaling_context(struct intel_breadcrumbs *b,
intel_engine_add_retire(b->irq_engine, tl);
}
-static bool __signal_request(struct i915_request *rq, struct list_head *signals)
+static bool __signal_request(struct i915_request *rq)
{
- clear_bit(I915_FENCE_FLAG_SIGNAL, &rq->fence.flags);
-
if (!__dma_fence_signal(&rq->fence)) {
i915_request_put(rq);
return false;
}
- list_add_tail(&rq->signal_link, signals);
return true;
}
+static struct llist_node *
+slist_add(struct llist_node *node, struct llist_node *head)
+{
+ node->next = head;
+ return node;
+}
+
static void signal_irq_work(struct irq_work *work)
{
struct intel_breadcrumbs *b = container_of(work, typeof(*b), irq_work);
const ktime_t timestamp = ktime_get();
+ struct llist_node *signal, *sn;
struct intel_context *ce, *cn;
struct list_head *pos, *next;
- LIST_HEAD(signal);
+
+ signal = NULL;
+ if (unlikely(!llist_empty(&b->signaled_requests)))
+ signal = llist_del_all(&b->signaled_requests);
spin_lock(&b->irq_lock);
- if (list_empty(&b->signalers))
+ /*
+ * Keep the irq armed until the interrupt after all listeners are gone.
+ *
+ * Enabling/disabling the interrupt is rather costly, roughly a couple
+ * of hundred microseconds. If we are proactive and enable/disable
+ * the interrupt around every request that wants a breadcrumb, we
+ * quickly drown in the extra orders of magnitude of latency imposed
+ * on request submission.
+ *
+ * So we try to be lazy, and keep the interrupts enabled until no
+ * more listeners appear within a breadcrumb interrupt interval (that
+ * is until a request completes that no one cares about). The
+ * observation is that listeners come in batches, and will often
+ * listen to a bunch of requests in succession. Though note on icl+,
+ * interrupts are always enabled due to concerns with rc6 being
+ * dysfunctional with per-engine interrupt masking.
+ *
+ * We also try to avoid raising too many interrupts, as they may
+ * be generated by userspace batches and it is unfortunately rather
+ * too easy to drown the CPU under a flood of GPU interrupts. Thus
+ * whenever no one appears to be listening, we turn off the interrupts.
+ * Fewer interrupts should conserve power -- at the very least, fewer
+ * interrupt draw less ire from other users of the system and tools
+ * like powertop.
+ */
+ if (!signal && b->irq_armed && list_empty(&b->signalers))
__intel_breadcrumbs_disarm_irq(b);
- list_splice_init(&b->signaled_requests, &signal);
-
list_for_each_entry_safe(ce, cn, &b->signalers, signal_link) {
GEM_BUG_ON(list_empty(&ce->signals));
* spinlock as the callback chain may end up adding
* more signalers to the same context or engine.
*/
- __signal_request(rq, &signal);
+ clear_bit(I915_FENCE_FLAG_SIGNAL, &rq->fence.flags);
+ if (__signal_request(rq))
+ /* We own signal_node now, xfer to local list */
+ signal = slist_add(&rq->signal_node, signal);
}
/*
spin_unlock(&b->irq_lock);
- list_for_each_safe(pos, next, &signal) {
+ llist_for_each_safe(signal, sn, signal) {
struct i915_request *rq =
- list_entry(pos, typeof(*rq), signal_link);
+ llist_entry(signal, typeof(*rq), signal_node);
struct list_head cb_list;
spin_lock(&rq->lock);
i915_request_put(rq);
}
+
+ if (!READ_ONCE(b->irq_armed) && !list_empty(&b->signalers))
+ intel_breadcrumbs_arm_irq(b);
}
struct intel_breadcrumbs *
spin_lock_init(&b->irq_lock);
INIT_LIST_HEAD(&b->signalers);
- INIT_LIST_HEAD(&b->signaled_requests);
+ init_llist_head(&b->signaled_requests);
init_irq_work(&b->irq_work, signal_irq_work);
void intel_breadcrumbs_park(struct intel_breadcrumbs *b)
{
- unsigned long flags;
-
- if (!READ_ONCE(b->irq_armed))
- return;
-
- spin_lock_irqsave(&b->irq_lock, flags);
- __intel_breadcrumbs_disarm_irq(b);
- spin_unlock_irqrestore(&b->irq_lock, flags);
-
- if (!list_empty(&b->signalers))
- irq_work_queue(&b->irq_work);
+ /* Kick the work once more to drain the signalers */
+ irq_work_sync(&b->irq_work);
+ while (unlikely(READ_ONCE(b->irq_armed))) {
+ local_irq_disable();
+ signal_irq_work(&b->irq_work);
+ local_irq_enable();
+ cond_resched();
+ }
+ GEM_BUG_ON(!list_empty(&b->signalers));
}
void intel_breadcrumbs_free(struct intel_breadcrumbs *b)
{
+ irq_work_sync(&b->irq_work);
+ GEM_BUG_ON(!list_empty(&b->signalers));
+ GEM_BUG_ON(b->irq_armed);
kfree(b);
}
* its signal completion.
*/
if (__request_completed(rq)) {
- if (__signal_request(rq, &b->signaled_requests))
+ if (__signal_request(rq) &&
+ llist_add(&rq->signal_node, &b->signaled_requests))
irq_work_queue(&b->irq_work);
return;
}
GEM_BUG_ON(!check_signal_order(ce, rq));
set_bit(I915_FENCE_FLAG_SIGNAL, &rq->fence.flags);
- /* Check after attaching to irq, interrupt may have already fired. */
- if (__request_completed(rq))
- irq_work_queue(&b->irq_work);
+ /*
+ * Defer enabling the interrupt to after HW submission and recheck
+ * the request as it may have completed and raised the interrupt as
+ * we were attaching it into the lists.
+ */
+ irq_work_queue(&b->irq_work);
}
bool i915_request_enable_breadcrumb(struct i915_request *rq)
struct intel_engine_cs *irq_engine;
struct list_head signalers;
- struct list_head signaled_requests;
+ struct llist_head signaled_requests;
struct irq_work irq_work; /* for use from inside irq_lock */
* instances.
*/
if ((INTEL_GEN(i915) >= 11 &&
- engine->gt->info.vdbox_sfc_access & engine->mask) ||
+ (engine->gt->info.vdbox_sfc_access &
+ BIT(engine->instance))) ||
(INTEL_GEN(i915) >= 9 && engine->instance == 0))
engine->uabi_capabilities |=
I915_VIDEO_AND_ENHANCE_CLASS_CAPABILITY_SFC;
struct virtual_engine {
struct intel_engine_cs base;
struct intel_context context;
+ struct rcu_work rcu;
/*
* We allow only a single request through the virtual engine at a time
return &ve->base.execlists.default_priolist.requests[0];
}
-static void virtual_context_destroy(struct kref *kref)
+static void rcu_virtual_context_destroy(struct work_struct *wrk)
{
struct virtual_engine *ve =
- container_of(kref, typeof(*ve), context.ref);
+ container_of(wrk, typeof(*ve), rcu.work);
unsigned int n;
- GEM_BUG_ON(!list_empty(virtual_queue(ve)));
- GEM_BUG_ON(ve->request);
GEM_BUG_ON(ve->context.inflight);
+ /* Preempt-to-busy may leave a stale request behind. */
+ if (unlikely(ve->request)) {
+ struct i915_request *old;
+
+ spin_lock_irq(&ve->base.active.lock);
+
+ old = fetch_and_zero(&ve->request);
+ if (old) {
+ GEM_BUG_ON(!i915_request_completed(old));
+ __i915_request_submit(old);
+ i915_request_put(old);
+ }
+
+ spin_unlock_irq(&ve->base.active.lock);
+ }
+
+ /*
+ * Flush the tasklet in case it is still running on another core.
+ *
+ * This needs to be done before we remove ourselves from the siblings'
+ * rbtrees as in the case it is running in parallel, it may reinsert
+ * the rb_node into a sibling.
+ */
+ tasklet_kill(&ve->base.execlists.tasklet);
+
+ /* Decouple ourselves from the siblings, no more access allowed. */
for (n = 0; n < ve->num_siblings; n++) {
struct intel_engine_cs *sibling = ve->siblings[n];
struct rb_node *node = &ve->nodes[sibling->id].rb;
- unsigned long flags;
if (RB_EMPTY_NODE(node))
continue;
- spin_lock_irqsave(&sibling->active.lock, flags);
+ spin_lock_irq(&sibling->active.lock);
/* Detachment is lazily performed in the execlists tasklet */
if (!RB_EMPTY_NODE(node))
rb_erase_cached(node, &sibling->execlists.virtual);
- spin_unlock_irqrestore(&sibling->active.lock, flags);
+ spin_unlock_irq(&sibling->active.lock);
}
GEM_BUG_ON(__tasklet_is_scheduled(&ve->base.execlists.tasklet));
+ GEM_BUG_ON(!list_empty(virtual_queue(ve)));
if (ve->context.state)
__execlists_context_fini(&ve->context);
intel_context_fini(&ve->context);
+ intel_breadcrumbs_free(ve->base.breadcrumbs);
intel_engine_free_request_pool(&ve->base);
kfree(ve->bonds);
kfree(ve);
}
+static void virtual_context_destroy(struct kref *kref)
+{
+ struct virtual_engine *ve =
+ container_of(kref, typeof(*ve), context.ref);
+
+ GEM_BUG_ON(!list_empty(&ve->context.signals));
+
+ /*
+ * When destroying the virtual engine, we have to be aware that
+ * it may still be in use from an hardirq/softirq context causing
+ * the resubmission of a completed request (background completion
+ * due to preempt-to-busy). Before we can free the engine, we need
+ * to flush the submission code and tasklets that are still potentially
+ * accessing the engine. Flushing the tasklets requires process context,
+ * and since we can guard the resubmit onto the engine with an RCU read
+ * lock, we can delegate the free of the engine to an RCU worker.
+ */
+ INIT_RCU_WORK(&ve->rcu, rcu_virtual_context_destroy);
+ queue_rcu_work(system_wq, &ve->rcu);
+}
+
static void virtual_engine_initial_hint(struct virtual_engine *ve)
{
int swp;
* only, __init_mocs_table() take care to program unused index with
* this entry.
*/
- MOCS_ENTRY(1, LE_3_WB | LE_TC_1_LLC | LE_LRUM(3),
- L3_3_WB),
+ MOCS_ENTRY(I915_MOCS_PTE,
+ LE_0_PAGETABLE | LE_TC_0_PAGETABLE,
+ L3_1_UC),
GEN11_MOCS_ENTRIES,
/* Implicitly enable L1 - HDC:L1 + L3 + LLC */
static void gen11_rc6_enable(struct intel_rc6 *rc6)
{
- struct intel_uncore *uncore = rc6_to_uncore(rc6);
+ struct intel_gt *gt = rc6_to_gt(rc6);
+ struct intel_uncore *uncore = gt->uncore;
struct intel_engine_cs *engine;
enum intel_engine_id id;
+ u32 pg_enable;
+ int i;
/* 2b: Program RC6 thresholds.*/
set(uncore, GEN6_RC6_WAKE_RATE_LIMIT, 54 << 16 | 85);
GEN6_RC_CTL_RC6_ENABLE |
GEN6_RC_CTL_EI_MODE(1);
- set(uncore, GEN9_PG_ENABLE,
- GEN9_RENDER_PG_ENABLE |
- GEN9_MEDIA_PG_ENABLE |
- GEN11_MEDIA_SAMPLER_PG_ENABLE);
+ pg_enable =
+ GEN9_RENDER_PG_ENABLE |
+ GEN9_MEDIA_PG_ENABLE |
+ GEN11_MEDIA_SAMPLER_PG_ENABLE;
+
+ if (INTEL_GEN(gt->i915) >= 12) {
+ for (i = 0; i < I915_MAX_VCS; i++)
+ if (HAS_ENGINE(gt, _VCS(i)))
+ pg_enable |= (VDN_HCP_POWERGATE_ENABLE(i) |
+ VDN_MFX_POWERGATE_ENABLE(i));
+ }
+
+ set(uncore, GEN9_PG_ENABLE, pg_enable);
}
static void gen9_rc6_enable(struct intel_rc6 *rc6)
return;
}
- if (wal->list)
+ if (wal->list) {
memcpy(list, wal->list, sizeof(*wa) * wal->count);
+ kfree(wal->list);
+ }
wal->list = list;
}
/* let the virtual display supports DP1.2 */
static u8 dpcd_fix_data[DPCD_HEADER_SIZE] = {
- 0x12, 0x014, 0x04, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00
+ 0x12, 0x014, 0x84, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00
};
static void emulate_monitor_status_change(struct intel_vgpu *vgpu)
#define F_CMD_ACCESS (1 << 3)
/* This reg has been accessed by a VM */
#define F_ACCESSED (1 << 4)
-/* This reg has been accessed through GPU commands */
+/* This reg could be accessed by unaligned address */
#define F_UNALIGN (1 << 6)
/* This reg is in GVT's mmio save-restor list and in hardware
* logical context image
/* Take a module reference as mdev core doesn't take
* a reference for vendor driver.
*/
- if (!try_module_get(THIS_MODULE))
+ if (!try_module_get(THIS_MODULE)) {
+ ret = -ENODEV;
goto undo_group;
+ }
ret = kvmgt_guest_init(mdev);
if (ret)
if (IS_BROADWELL(dev_priv))
ret = intel_gvt_hypervisor_set_edid(vgpu, PORT_B);
- else
+ /* FixMe: Re-enable APL/BXT once vfio_edid enabled */
+ else if (!IS_BROXTON(dev_priv))
ret = intel_gvt_hypervisor_set_edid(vgpu, PORT_D);
if (ret)
goto out_clean_sched_policy;
return ret;
}
-static int
-i915_gem_phys_pwrite(struct drm_i915_gem_object *obj,
- struct drm_i915_gem_pwrite *args,
- struct drm_file *file)
-{
- void *vaddr = sg_page(obj->mm.pages->sgl) + args->offset;
- char __user *user_data = u64_to_user_ptr(args->data_ptr);
-
- /*
- * We manually control the domain here and pretend that it
- * remains coherent i.e. in the GTT domain, like shmem_pwrite.
- */
- i915_gem_object_invalidate_frontbuffer(obj, ORIGIN_CPU);
-
- if (copy_from_user(vaddr, user_data, args->size))
- return -EFAULT;
-
- drm_clflush_virt_range(vaddr, args->size);
- intel_gt_chipset_flush(&to_i915(obj->base.dev)->gt);
-
- i915_gem_object_flush_frontbuffer(obj, ORIGIN_CPU);
- return 0;
-}
-
static int
i915_gem_create(struct drm_file *file,
struct intel_memory_region *mr,
trace_i915_gem_object_pread(obj, args->offset, args->size);
+ ret = -ENODEV;
+ if (obj->ops->pread)
+ ret = obj->ops->pread(obj, args);
+ if (ret != -ENODEV)
+ goto out;
+
ret = i915_gem_object_wait(obj,
I915_WAIT_INTERRUPTIBLE,
MAX_SCHEDULE_TIMEOUT);
if (ret == -EFAULT || ret == -ENOSPC) {
if (i915_gem_object_has_struct_page(obj))
ret = i915_gem_shmem_pwrite(obj, args);
- else
- ret = i915_gem_phys_pwrite(obj, args, file);
}
i915_gem_object_unpin_pages(obj);
DRM_I915_PERF_RECORD_OA_REPORT_LOST);
if (ret)
return ret;
- intel_uncore_write(uncore, oastatus_reg,
- oastatus & ~GEN8_OASTATUS_REPORT_LOST);
+
+ intel_uncore_rmw(uncore, oastatus_reg,
+ GEN8_OASTATUS_COUNTER_OVERFLOW |
+ GEN8_OASTATUS_REPORT_LOST,
+ IS_GEN_RANGE(uncore->i915, 8, 10) ?
+ (GEN8_OASTATUS_HEAD_POINTER_WRAP |
+ GEN8_OASTATUS_TAIL_POINTER_WRAP) : 0);
}
return gen8_append_oa_reports(stream, buf, count, offset);
#define GEN7_OASTATUS2_MEM_SELECT_GGTT (1 << 0) /* 0: PPGTT, 1: GGTT */
#define GEN8_OASTATUS _MMIO(0x2b08)
+#define GEN8_OASTATUS_TAIL_POINTER_WRAP (1 << 17)
+#define GEN8_OASTATUS_HEAD_POINTER_WRAP (1 << 16)
#define GEN8_OASTATUS_OVERRUN_STATUS (1 << 3)
#define GEN8_OASTATUS_COUNTER_OVERFLOW (1 << 2)
#define GEN8_OASTATUS_OABUFFER_OVERFLOW (1 << 1)
#define GEN9_PWRGT_MEDIA_STATUS_MASK (1 << 0)
#define GEN9_PWRGT_RENDER_STATUS_MASK (1 << 1)
-#define POWERGATE_ENABLE _MMIO(0xa210)
-#define VDN_HCP_POWERGATE_ENABLE(n) BIT(((n) * 2) + 3)
-#define VDN_MFX_POWERGATE_ENABLE(n) BIT(((n) * 2) + 4)
-
#define GTFIFODBG _MMIO(0x120000)
#define GT_FIFO_SBDEDICATE_FREE_ENTRY_CHV (0x1f << 20)
#define GT_FIFO_FREE_ENTRIES_CHV (0x7f << 13)
#define GEN9_MEDIA_PG_IDLE_HYSTERESIS _MMIO(0xA0C4)
#define GEN9_RENDER_PG_IDLE_HYSTERESIS _MMIO(0xA0C8)
#define GEN9_PG_ENABLE _MMIO(0xA210)
-#define GEN9_RENDER_PG_ENABLE REG_BIT(0)
-#define GEN9_MEDIA_PG_ENABLE REG_BIT(1)
-#define GEN11_MEDIA_SAMPLER_PG_ENABLE REG_BIT(2)
+#define GEN9_RENDER_PG_ENABLE REG_BIT(0)
+#define GEN9_MEDIA_PG_ENABLE REG_BIT(1)
+#define GEN11_MEDIA_SAMPLER_PG_ENABLE REG_BIT(2)
+#define VDN_HCP_POWERGATE_ENABLE(n) REG_BIT(3 + 2 * (n))
+#define VDN_MFX_POWERGATE_ENABLE(n) REG_BIT(4 + 2 * (n))
#define GEN8_PUSHBUS_CONTROL _MMIO(0xA248)
#define GEN8_PUSHBUS_ENABLE _MMIO(0xA250)
#define GEN8_PUSHBUS_SHIFT _MMIO(0xA25C)
struct intel_context *context;
struct intel_ring *ring;
struct intel_timeline __rcu *timeline;
- struct list_head signal_link;
+
+ union {
+ struct list_head signal_link;
+ struct llist_node signal_node;
+ };
/*
* The rcu epoch of when this request was allocated. Used to judiciously
static void tgl_init_clock_gating(struct drm_i915_private *dev_priv)
{
- u32 vd_pg_enable = 0;
- unsigned int i;
-
/* Wa_1409120013:tgl */
I915_WRITE(ILK_DPFC_CHICKEN,
ILK_DPFC_CHICKEN_COMP_DUMMY_PIXEL);
- /* This is not a WA. Enable VD HCP & MFX_ENC powergate */
- for (i = 0; i < I915_MAX_VCS; i++) {
- if (HAS_ENGINE(&dev_priv->gt, _VCS(i)))
- vd_pg_enable |= VDN_HCP_POWERGATE_ENABLE(i) |
- VDN_MFX_POWERGATE_ENABLE(i);
- }
-
- I915_WRITE(POWERGATE_ENABLE,
- I915_READ(POWERGATE_ENABLE) | vd_pg_enable);
-
/* Wa_1409825376:tgl (pre-prod)*/
if (IS_TGL_DISP_REVID(dev_priv, TGL_REVID_A0, TGL_REVID_B1))
I915_WRITE(GEN9_CLKGATE_DIS_3, I915_READ(GEN9_CLKGATE_DIS_3) |
struct intel_context *ce;
ce = intel_context_create(engine);
- if (IS_ERR(ce))
+ if (IS_ERR(ce)) {
+ err = PTR_ERR(ce);
goto out;
+ }
err = intel_context_pin(ce);
if (err) {
struct intel_context *ce;
ce = intel_context_create(engine);
- if (IS_ERR(ce))
+ if (IS_ERR(ce)) {
+ err = PTR_ERR(ce);
goto out;
+ }
err = intel_context_pin(ce);
if (err) {
match);
if (ret) {
dev_err(dev, "failed to add component master\n");
- goto clk_disable;
+ /*
+ * The EPOD regulator is already disabled at this point so some
+ * special errorpath code is needed
+ */
+ clk_disable_unprepare(mcde->mcde_clk);
+ regulator_disable(mcde->vana);
+ return ret;
}
return 0;
return 0;
}
-static void mtk_dpi_encoder_destroy(struct drm_encoder *encoder)
-{
- drm_encoder_cleanup(encoder);
-}
-
-static const struct drm_encoder_funcs mtk_dpi_encoder_funcs = {
- .destroy = mtk_dpi_encoder_destroy,
-};
-
static int mtk_dpi_bridge_attach(struct drm_bridge *bridge,
enum drm_bridge_attach_flags flags)
{
u32 horizontal_sync_active_byte;
u32 horizontal_backporch_byte;
u32 horizontal_frontporch_byte;
+ u32 horizontal_front_back_byte;
+ u32 data_phy_cycles_byte;
u32 dsi_tmp_buf_bpp, data_phy_cycles;
+ u32 delta;
struct mtk_phy_timing *timing = &dsi->phy_timing;
struct videomode *vm = &dsi->vm;
horizontal_sync_active_byte = (vm->hsync_len * dsi_tmp_buf_bpp - 10);
if (dsi->mode_flags & MIPI_DSI_MODE_VIDEO_SYNC_PULSE)
- horizontal_backporch_byte = vm->hback_porch * dsi_tmp_buf_bpp;
+ horizontal_backporch_byte = vm->hback_porch * dsi_tmp_buf_bpp - 10;
else
horizontal_backporch_byte = (vm->hback_porch + vm->hsync_len) *
- dsi_tmp_buf_bpp;
+ dsi_tmp_buf_bpp - 10;
data_phy_cycles = timing->lpx + timing->da_hs_prepare +
- timing->da_hs_zero + timing->da_hs_exit;
-
- if (dsi->mode_flags & MIPI_DSI_MODE_VIDEO_BURST) {
- if ((vm->hfront_porch + vm->hback_porch) * dsi_tmp_buf_bpp >
- data_phy_cycles * dsi->lanes + 18) {
- horizontal_frontporch_byte =
- vm->hfront_porch * dsi_tmp_buf_bpp -
- (data_phy_cycles * dsi->lanes + 18) *
- vm->hfront_porch /
- (vm->hfront_porch + vm->hback_porch);
-
- horizontal_backporch_byte =
- horizontal_backporch_byte -
- (data_phy_cycles * dsi->lanes + 18) *
- vm->hback_porch /
- (vm->hfront_porch + vm->hback_porch);
- } else {
- DRM_WARN("HFP less than d-phy, FPS will under 60Hz\n");
- horizontal_frontporch_byte = vm->hfront_porch *
- dsi_tmp_buf_bpp;
- }
+ timing->da_hs_zero + timing->da_hs_exit + 3;
+
+ delta = dsi->mode_flags & MIPI_DSI_MODE_VIDEO_BURST ? 18 : 12;
+
+ horizontal_frontporch_byte = vm->hfront_porch * dsi_tmp_buf_bpp;
+ horizontal_front_back_byte = horizontal_frontporch_byte + horizontal_backporch_byte;
+ data_phy_cycles_byte = data_phy_cycles * dsi->lanes + delta;
+
+ if (horizontal_front_back_byte > data_phy_cycles_byte) {
+ horizontal_frontporch_byte -= data_phy_cycles_byte *
+ horizontal_frontporch_byte /
+ horizontal_front_back_byte;
+
+ horizontal_backporch_byte -= data_phy_cycles_byte *
+ horizontal_backporch_byte /
+ horizontal_front_back_byte;
} else {
- if ((vm->hfront_porch + vm->hback_porch) * dsi_tmp_buf_bpp >
- data_phy_cycles * dsi->lanes + 12) {
- horizontal_frontporch_byte =
- vm->hfront_porch * dsi_tmp_buf_bpp -
- (data_phy_cycles * dsi->lanes + 12) *
- vm->hfront_porch /
- (vm->hfront_porch + vm->hback_porch);
- horizontal_backporch_byte = horizontal_backporch_byte -
- (data_phy_cycles * dsi->lanes + 12) *
- vm->hback_porch /
- (vm->hfront_porch + vm->hback_porch);
- } else {
- DRM_WARN("HFP less than d-phy, FPS will under 60Hz\n");
- horizontal_frontporch_byte = vm->hfront_porch *
- dsi_tmp_buf_bpp;
- }
+ DRM_WARN("HFP + HBP less than d-phy, FPS will under 60Hz\n");
}
writel(horizontal_sync_active_byte, dsi->regs + DSI_HSA_WC);
* DAC
*****************************************************************************/
static void
-nv50_dac_disable(struct drm_encoder *encoder)
+nv50_dac_disable(struct drm_encoder *encoder, struct drm_atomic_state *state)
{
struct nouveau_encoder *nv_encoder = nouveau_encoder(encoder);
struct nv50_core *core = nv50_disp(encoder->dev)->core;
}
static void
-nv50_dac_enable(struct drm_encoder *encoder)
+nv50_dac_enable(struct drm_encoder *encoder, struct drm_atomic_state *state)
{
struct nouveau_encoder *nv_encoder = nouveau_encoder(encoder);
struct nouveau_crtc *nv_crtc = nouveau_crtc(encoder->crtc);
static const struct drm_encoder_helper_funcs
nv50_dac_help = {
.atomic_check = nv50_outp_atomic_check,
- .enable = nv50_dac_enable,
- .disable = nv50_dac_disable,
+ .atomic_enable = nv50_dac_enable,
+ .atomic_disable = nv50_dac_disable,
.detect = nv50_dac_detect
};
}
static void
-nv50_msto_enable(struct drm_encoder *encoder)
+nv50_msto_enable(struct drm_encoder *encoder, struct drm_atomic_state *state)
{
struct nv50_head *head = nv50_head(encoder->crtc);
struct nv50_head_atom *armh = nv50_head_atom(head->base.base.state);
}
static void
-nv50_msto_disable(struct drm_encoder *encoder)
+nv50_msto_disable(struct drm_encoder *encoder, struct drm_atomic_state *state)
{
struct nv50_msto *msto = nv50_msto(encoder);
struct nv50_mstc *mstc = msto->mstc;
static const struct drm_encoder_helper_funcs
nv50_msto_help = {
- .disable = nv50_msto_disable,
- .enable = nv50_msto_enable,
+ .atomic_disable = nv50_msto_disable,
+ .atomic_enable = nv50_msto_enable,
.atomic_check = nv50_msto_atomic_check,
};
}
static void
-nv50_sor_enable(struct drm_encoder *encoder,
- struct drm_atomic_state *state)
+nv50_sor_enable(struct drm_encoder *encoder, struct drm_atomic_state *state)
{
struct nouveau_encoder *nv_encoder = nouveau_encoder(encoder);
struct nouveau_crtc *nv_crtc = nouveau_crtc(encoder->crtc);
}
static void
-nv50_pior_disable(struct drm_encoder *encoder)
+nv50_pior_disable(struct drm_encoder *encoder, struct drm_atomic_state *state)
{
struct nouveau_encoder *nv_encoder = nouveau_encoder(encoder);
struct nv50_core *core = nv50_disp(encoder->dev)->core;
}
static void
-nv50_pior_enable(struct drm_encoder *encoder)
+nv50_pior_enable(struct drm_encoder *encoder, struct drm_atomic_state *state)
{
struct nouveau_encoder *nv_encoder = nouveau_encoder(encoder);
struct nouveau_crtc *nv_crtc = nouveau_crtc(encoder->crtc);
}
core->func->pior->ctrl(core, nv_encoder->or, ctrl, asyh);
- nv_encoder->crtc = encoder->crtc;
+ nv_encoder->crtc = &nv_crtc->base;
}
static const struct drm_encoder_helper_funcs
nv50_pior_help = {
.atomic_check = nv50_pior_atomic_check,
- .enable = nv50_pior_enable,
- .disable = nv50_pior_disable,
+ .atomic_enable = nv50_pior_enable,
+ .atomic_disable = nv50_pior_disable,
};
static void
if (domain & NOUVEAU_GEM_DOMAIN_VRAM) {
struct nvif_mmu *mmu = &drm->client.mmu;
- const u8 type = mmu->type[drm->ttm.type_vram].type;
pl[*n].mem_type = TTM_PL_VRAM;
pl[*n].flags = flags & ~TTM_PL_FLAG_CACHED;
/* Some BARs do not support being ioremapped WC */
if (drm->client.device.info.family >= NV_DEVICE_INFO_V0_TESLA &&
- type & NVIF_MEM_UNCACHED)
+ mmu->type[drm->ttm.type_vram].type & NVIF_MEM_UNCACHED)
pl[*n].flags &= ~TTM_PL_FLAG_WC;
(*n)++;
nouveau_connector_set_edid(struct nouveau_connector *nv_connector,
struct edid *edid)
{
- struct edid *old_edid = nv_connector->edid;
+ if (nv_connector->edid != edid) {
+ struct edid *old_edid = nv_connector->edid;
- drm_connector_update_edid_property(&nv_connector->base, edid);
- kfree(old_edid);
- nv_connector->edid = edid;
+ drm_connector_update_edid_property(&nv_connector->base, edid);
+ kfree(old_edid);
+ nv_connector->edid = edid;
+ }
}
static enum drm_connector_status
/* Try retrieving EDID via DDC */
if (!drm->vbios.fp_no_ddc) {
status = nouveau_connector_detect(connector, force);
- if (status == connector_status_connected)
+ if (status == connector_status_connected) {
+ edid = nv_connector->edid;
goto out;
+ }
}
/* On some laptops (Sony, i'm looking at you) there appears to
NV_PRINTK(err, cli, "validating bo list\n");
validate_fini(op, chan, NULL, NULL);
return ret;
+ } else if (ret > 0) {
+ *apply_relocs = true;
}
- *apply_relocs = ret;
+
return 0;
}
nouveau_bo_wr32(nvbo, r->reloc_bo_offset >> 2, data);
}
- u_free(reloc);
return ret;
}
break;
}
}
- u_free(reloc);
}
out_prevalid:
+ if (!IS_ERR(reloc))
+ u_free(reloc);
u_free(bo);
u_free(push);
*
* XXX(hch): this has no business in a driver and needs to move
* to the device tree.
+ *
+ * If we have two subsequent calls to dma_direct_set_offset
+ * returns -EINVAL. Unfortunately, this happens when we have two
+ * backends in the system, and will result in the driver
+ * reporting an error while it has been setup properly before.
+ * Ignore EINVAL, but it should really be removed eventually.
*/
ret = dma_direct_set_offset(drm->dev, PHYS_OFFSET, 0, SZ_4G);
- if (ret)
+ if (ret && ret != -EINVAL)
return ret;
}
phy_node = of_parse_phandle(dev->of_node, "phys", 0);
if (!phy_node) {
dev_err(dev, "Can't found PHY phandle\n");
+ ret = -EINVAL;
goto err_disable_clk_tmds;
}
struct drm_modeset_lock ctm_state_lock;
struct drm_private_obj ctm_manager;
+ struct drm_private_obj hvs_channels;
struct drm_private_obj load_tracker;
/* List of vc4_debugfs_info_entry for adding to debugfs once
unsigned int top;
unsigned int bottom;
} margins;
+
+ /* Transitional state below, only valid during atomic commits */
+ bool update_muxing;
};
#define VC4_HVS_CHANNEL_DISABLED ((unsigned int)-1)
{
}
+#define WIFI_2_4GHz_CH1_MIN_FREQ 2400000000ULL
+#define WIFI_2_4GHz_CH1_MAX_FREQ 2422000000ULL
+
+static int vc4_hdmi_encoder_atomic_check(struct drm_encoder *encoder,
+ struct drm_crtc_state *crtc_state,
+ struct drm_connector_state *conn_state)
+{
+ struct drm_display_mode *mode = &crtc_state->adjusted_mode;
+ struct vc4_hdmi *vc4_hdmi = encoder_to_vc4_hdmi(encoder);
+ unsigned long long pixel_rate = mode->clock * 1000;
+ unsigned long long tmds_rate;
+
+ if (vc4_hdmi->variant->unsupported_odd_h_timings &&
+ ((mode->hdisplay % 2) || (mode->hsync_start % 2) ||
+ (mode->hsync_end % 2) || (mode->htotal % 2)))
+ return -EINVAL;
+
+ /*
+ * The 1440p@60 pixel rate is in the same range than the first
+ * WiFi channel (between 2.4GHz and 2.422GHz with 22MHz
+ * bandwidth). Slightly lower the frequency to bring it out of
+ * the WiFi range.
+ */
+ tmds_rate = pixel_rate * 10;
+ if (vc4_hdmi->disable_wifi_frequencies &&
+ (tmds_rate >= WIFI_2_4GHz_CH1_MIN_FREQ &&
+ tmds_rate <= WIFI_2_4GHz_CH1_MAX_FREQ)) {
+ mode->clock = 238560;
+ pixel_rate = mode->clock * 1000;
+ }
+
+ if (pixel_rate > vc4_hdmi->variant->max_pixel_clock)
+ return -EINVAL;
+
+ return 0;
+}
+
static enum drm_mode_status
vc4_hdmi_encoder_mode_valid(struct drm_encoder *encoder,
const struct drm_display_mode *mode)
{
struct vc4_hdmi *vc4_hdmi = encoder_to_vc4_hdmi(encoder);
+ if (vc4_hdmi->variant->unsupported_odd_h_timings &&
+ ((mode->hdisplay % 2) || (mode->hsync_start % 2) ||
+ (mode->hsync_end % 2) || (mode->htotal % 2)))
+ return MODE_H_ILLEGAL;
+
if ((mode->clock * 1000) > vc4_hdmi->variant->max_pixel_clock)
return MODE_CLOCK_HIGH;
}
static const struct drm_encoder_helper_funcs vc4_hdmi_encoder_helper_funcs = {
+ .atomic_check = vc4_hdmi_encoder_atomic_check,
.mode_valid = vc4_hdmi_encoder_mode_valid,
.disable = vc4_hdmi_encoder_disable,
.enable = vc4_hdmi_encoder_enable,
vc4_hdmi->hpd_active_low = hpd_gpio_flags & OF_GPIO_ACTIVE_LOW;
}
+ vc4_hdmi->disable_wifi_frequencies =
+ of_property_read_bool(dev->of_node, "wifi-2.4ghz-coexistence");
+
pm_runtime_enable(dev);
drm_simple_encoder_init(drm, encoder, DRM_MODE_ENCODER_TMDS);
PHY_LANE_2,
PHY_LANE_CK,
},
+ .unsupported_odd_h_timings = true,
.init_resources = vc5_hdmi_init_resources,
.csc_setup = vc5_hdmi_csc_setup,
PHY_LANE_CK,
PHY_LANE_2,
},
+ .unsupported_odd_h_timings = true,
.init_resources = vc5_hdmi_init_resources,
.csc_setup = vc5_hdmi_csc_setup,
*/
enum vc4_hdmi_phy_channel phy_lane_mapping[4];
+ /* The BCM2711 cannot deal with odd horizontal pixel timings */
+ bool unsupported_odd_h_timings;
+
/* Callback to get the resources (memory region, interrupts,
* clocks, etc) for that variant.
*/
int hpd_gpio;
bool hpd_active_low;
+ /*
+ * On some systems (like the RPi4), some modes are in the same
+ * frequency range than the WiFi channels (1440p@60Hz for
+ * example). Should we take evasive actions because that system
+ * has a wifi adapter?
+ */
+ bool disable_wifi_frequencies;
+
struct cec_adapter *cec_adap;
struct cec_msg cec_rx_msg;
bool cec_tx_ok;
#include "vc4_drv.h"
#include "vc4_regs.h"
+#define HVS_NUM_CHANNELS 3
+
struct vc4_ctm_state {
struct drm_private_state base;
struct drm_color_ctm *ctm;
return container_of(priv, struct vc4_ctm_state, base);
}
+struct vc4_hvs_state {
+ struct drm_private_state base;
+ unsigned int unassigned_channels;
+};
+
+static struct vc4_hvs_state *
+to_vc4_hvs_state(struct drm_private_state *priv)
+{
+ return container_of(priv, struct vc4_hvs_state, base);
+}
+
struct vc4_load_tracker_state {
struct drm_private_state base;
u64 hvs_load;
drm_atomic_private_obj_init(&vc4->base, &vc4->ctm_manager, &ctm_state->base,
&vc4_ctm_state_funcs);
- return drmm_add_action(&vc4->base, vc4_ctm_obj_fini, NULL);
+ return drmm_add_action_or_reset(&vc4->base, vc4_ctm_obj_fini, NULL);
}
/* Converts a DRM S31.32 value to the HW S0.9 format. */
VC4_SET_FIELD(ctm_state->fifo, SCALER_OLEDOFFS_DISPFIFO));
}
+static struct vc4_hvs_state *
+vc4_hvs_get_global_state(struct drm_atomic_state *state)
+{
+ struct vc4_dev *vc4 = to_vc4_dev(state->dev);
+ struct drm_private_state *priv_state;
+
+ priv_state = drm_atomic_get_private_obj_state(state, &vc4->hvs_channels);
+ if (IS_ERR(priv_state))
+ return ERR_CAST(priv_state);
+
+ return to_vc4_hvs_state(priv_state);
+}
+
static void vc4_hvs_pv_muxing_commit(struct vc4_dev *vc4,
struct drm_atomic_state *state)
{
{
struct drm_crtc_state *crtc_state;
struct drm_crtc *crtc;
- unsigned char dsp2_mux = 0;
- unsigned char dsp3_mux = 3;
- unsigned char dsp4_mux = 3;
- unsigned char dsp5_mux = 3;
+ unsigned char mux;
unsigned int i;
u32 reg;
struct vc4_crtc_state *vc4_state = to_vc4_crtc_state(crtc_state);
struct vc4_crtc *vc4_crtc = to_vc4_crtc(crtc);
- if (!crtc_state->active)
+ if (!vc4_state->update_muxing)
continue;
switch (vc4_crtc->data->hvs_output) {
case 2:
- dsp2_mux = (vc4_state->assigned_channel == 2) ? 0 : 1;
+ mux = (vc4_state->assigned_channel == 2) ? 0 : 1;
+ reg = HVS_READ(SCALER_DISPECTRL);
+ HVS_WRITE(SCALER_DISPECTRL,
+ (reg & ~SCALER_DISPECTRL_DSP2_MUX_MASK) |
+ VC4_SET_FIELD(mux, SCALER_DISPECTRL_DSP2_MUX));
break;
case 3:
- dsp3_mux = vc4_state->assigned_channel;
+ if (vc4_state->assigned_channel == VC4_HVS_CHANNEL_DISABLED)
+ mux = 3;
+ else
+ mux = vc4_state->assigned_channel;
+
+ reg = HVS_READ(SCALER_DISPCTRL);
+ HVS_WRITE(SCALER_DISPCTRL,
+ (reg & ~SCALER_DISPCTRL_DSP3_MUX_MASK) |
+ VC4_SET_FIELD(mux, SCALER_DISPCTRL_DSP3_MUX));
break;
case 4:
- dsp4_mux = vc4_state->assigned_channel;
+ if (vc4_state->assigned_channel == VC4_HVS_CHANNEL_DISABLED)
+ mux = 3;
+ else
+ mux = vc4_state->assigned_channel;
+
+ reg = HVS_READ(SCALER_DISPEOLN);
+ HVS_WRITE(SCALER_DISPEOLN,
+ (reg & ~SCALER_DISPEOLN_DSP4_MUX_MASK) |
+ VC4_SET_FIELD(mux, SCALER_DISPEOLN_DSP4_MUX));
+
break;
case 5:
- dsp5_mux = vc4_state->assigned_channel;
+ if (vc4_state->assigned_channel == VC4_HVS_CHANNEL_DISABLED)
+ mux = 3;
+ else
+ mux = vc4_state->assigned_channel;
+
+ reg = HVS_READ(SCALER_DISPDITHER);
+ HVS_WRITE(SCALER_DISPDITHER,
+ (reg & ~SCALER_DISPDITHER_DSP5_MUX_MASK) |
+ VC4_SET_FIELD(mux, SCALER_DISPDITHER_DSP5_MUX));
break;
default:
break;
}
}
-
- reg = HVS_READ(SCALER_DISPECTRL);
- HVS_WRITE(SCALER_DISPECTRL,
- (reg & ~SCALER_DISPECTRL_DSP2_MUX_MASK) |
- VC4_SET_FIELD(dsp2_mux, SCALER_DISPECTRL_DSP2_MUX));
-
- reg = HVS_READ(SCALER_DISPCTRL);
- HVS_WRITE(SCALER_DISPCTRL,
- (reg & ~SCALER_DISPCTRL_DSP3_MUX_MASK) |
- VC4_SET_FIELD(dsp3_mux, SCALER_DISPCTRL_DSP3_MUX));
-
- reg = HVS_READ(SCALER_DISPEOLN);
- HVS_WRITE(SCALER_DISPEOLN,
- (reg & ~SCALER_DISPEOLN_DSP4_MUX_MASK) |
- VC4_SET_FIELD(dsp4_mux, SCALER_DISPEOLN_DSP4_MUX));
-
- reg = HVS_READ(SCALER_DISPDITHER);
- HVS_WRITE(SCALER_DISPDITHER,
- (reg & ~SCALER_DISPDITHER_DSP5_MUX_MASK) |
- VC4_SET_FIELD(dsp5_mux, SCALER_DISPDITHER_DSP5_MUX));
}
static void
&load_state->base,
&vc4_load_tracker_state_funcs);
- return drmm_add_action(&vc4->base, vc4_load_tracker_obj_fini, NULL);
+ return drmm_add_action_or_reset(&vc4->base, vc4_load_tracker_obj_fini, NULL);
}
-#define NUM_OUTPUTS 6
-#define NUM_CHANNELS 3
+static struct drm_private_state *
+vc4_hvs_channels_duplicate_state(struct drm_private_obj *obj)
+{
+ struct vc4_hvs_state *old_state = to_vc4_hvs_state(obj->state);
+ struct vc4_hvs_state *state;
+
+ state = kzalloc(sizeof(*state), GFP_KERNEL);
+ if (!state)
+ return NULL;
-static int
-vc4_atomic_check(struct drm_device *dev, struct drm_atomic_state *state)
+ __drm_atomic_helper_private_obj_duplicate_state(obj, &state->base);
+
+ state->unassigned_channels = old_state->unassigned_channels;
+
+ return &state->base;
+}
+
+static void vc4_hvs_channels_destroy_state(struct drm_private_obj *obj,
+ struct drm_private_state *state)
{
- unsigned long unassigned_channels = GENMASK(NUM_CHANNELS - 1, 0);
- struct drm_crtc_state *old_crtc_state, *new_crtc_state;
- struct drm_crtc *crtc;
- int i, ret;
+ struct vc4_hvs_state *hvs_state = to_vc4_hvs_state(state);
- /*
- * Since the HVS FIFOs are shared across all the pixelvalves and
- * the TXP (and thus all the CRTCs), we need to pull the current
- * state of all the enabled CRTCs so that an update to a single
- * CRTC still keeps the previous FIFOs enabled and assigned to
- * the same CRTCs, instead of evaluating only the CRTC being
- * modified.
- */
- list_for_each_entry(crtc, &dev->mode_config.crtc_list, head) {
- struct drm_crtc_state *crtc_state;
+ kfree(hvs_state);
+}
- if (!crtc->state->enable)
- continue;
+static const struct drm_private_state_funcs vc4_hvs_state_funcs = {
+ .atomic_duplicate_state = vc4_hvs_channels_duplicate_state,
+ .atomic_destroy_state = vc4_hvs_channels_destroy_state,
+};
- crtc_state = drm_atomic_get_crtc_state(state, crtc);
- if (IS_ERR(crtc_state))
- return PTR_ERR(crtc_state);
- }
+static void vc4_hvs_channels_obj_fini(struct drm_device *dev, void *unused)
+{
+ struct vc4_dev *vc4 = to_vc4_dev(dev);
+
+ drm_atomic_private_obj_fini(&vc4->hvs_channels);
+}
+
+static int vc4_hvs_channels_obj_init(struct vc4_dev *vc4)
+{
+ struct vc4_hvs_state *state;
+
+ state = kzalloc(sizeof(*state), GFP_KERNEL);
+ if (!state)
+ return -ENOMEM;
+
+ state->unassigned_channels = GENMASK(HVS_NUM_CHANNELS - 1, 0);
+ drm_atomic_private_obj_init(&vc4->base, &vc4->hvs_channels,
+ &state->base,
+ &vc4_hvs_state_funcs);
+
+ return drmm_add_action_or_reset(&vc4->base, vc4_hvs_channels_obj_fini, NULL);
+}
+
+/*
+ * The BCM2711 HVS has up to 7 outputs connected to the pixelvalves and
+ * the TXP (and therefore all the CRTCs found on that platform).
+ *
+ * The naive (and our initial) implementation would just iterate over
+ * all the active CRTCs, try to find a suitable FIFO, and then remove it
+ * from the pool of available FIFOs. However, there are a few corner
+ * cases that need to be considered:
+ *
+ * - When running in a dual-display setup (so with two CRTCs involved),
+ * we can update the state of a single CRTC (for example by changing
+ * its mode using xrandr under X11) without affecting the other. In
+ * this case, the other CRTC wouldn't be in the state at all, so we
+ * need to consider all the running CRTCs in the DRM device to assign
+ * a FIFO, not just the one in the state.
+ *
+ * - To fix the above, we can't use drm_atomic_get_crtc_state on all
+ * enabled CRTCs to pull their CRTC state into the global state, since
+ * a page flip would start considering their vblank to complete. Since
+ * we don't have a guarantee that they are actually active, that
+ * vblank might never happen, and shouldn't even be considered if we
+ * want to do a page flip on a single CRTC. That can be tested by
+ * doing a modetest -v first on HDMI1 and then on HDMI0.
+ *
+ * - Since we need the pixelvalve to be disabled and enabled back when
+ * the FIFO is changed, we should keep the FIFO assigned for as long
+ * as the CRTC is enabled, only considering it free again once that
+ * CRTC has been disabled. This can be tested by booting X11 on a
+ * single display, and changing the resolution down and then back up.
+ */
+static int vc4_pv_muxing_atomic_check(struct drm_device *dev,
+ struct drm_atomic_state *state)
+{
+ struct vc4_hvs_state *hvs_new_state;
+ struct drm_crtc_state *old_crtc_state, *new_crtc_state;
+ struct drm_crtc *crtc;
+ unsigned int i;
+
+ hvs_new_state = vc4_hvs_get_global_state(state);
+ if (!hvs_new_state)
+ return -EINVAL;
for_each_oldnew_crtc_in_state(state, crtc, old_crtc_state, new_crtc_state, i) {
+ struct vc4_crtc_state *old_vc4_crtc_state =
+ to_vc4_crtc_state(old_crtc_state);
struct vc4_crtc_state *new_vc4_crtc_state =
to_vc4_crtc_state(new_crtc_state);
struct vc4_crtc *vc4_crtc = to_vc4_crtc(crtc);
unsigned int matching_channels;
- if (old_crtc_state->enable && !new_crtc_state->enable)
- new_vc4_crtc_state->assigned_channel = VC4_HVS_CHANNEL_DISABLED;
-
- if (!new_crtc_state->enable)
+ /* Nothing to do here, let's skip it */
+ if (old_crtc_state->enable == new_crtc_state->enable)
continue;
- if (new_vc4_crtc_state->assigned_channel != VC4_HVS_CHANNEL_DISABLED) {
- unassigned_channels &= ~BIT(new_vc4_crtc_state->assigned_channel);
+ /* Muxing will need to be modified, mark it as such */
+ new_vc4_crtc_state->update_muxing = true;
+
+ /* If we're disabling our CRTC, we put back our channel */
+ if (!new_crtc_state->enable) {
+ hvs_new_state->unassigned_channels |= BIT(old_vc4_crtc_state->assigned_channel);
+ new_vc4_crtc_state->assigned_channel = VC4_HVS_CHANNEL_DISABLED;
continue;
}
* the future, we will need to have something smarter,
* but it works so far.
*/
- matching_channels = unassigned_channels & vc4_crtc->data->hvs_available_channels;
+ matching_channels = hvs_new_state->unassigned_channels & vc4_crtc->data->hvs_available_channels;
if (matching_channels) {
unsigned int channel = ffs(matching_channels) - 1;
new_vc4_crtc_state->assigned_channel = channel;
- unassigned_channels &= ~BIT(channel);
+ hvs_new_state->unassigned_channels &= ~BIT(channel);
} else {
return -EINVAL;
}
}
+ return 0;
+}
+
+static int
+vc4_atomic_check(struct drm_device *dev, struct drm_atomic_state *state)
+{
+ int ret;
+
+ ret = vc4_pv_muxing_atomic_check(dev, state);
+ if (ret)
+ return ret;
+
ret = vc4_ctm_atomic_check(dev, state);
if (ret < 0)
return ret;
if (ret)
return ret;
+ ret = vc4_hvs_channels_obj_init(vc4);
+ if (ret)
+ return ret;
+
drm_mode_config_reset(dev);
drm_kms_helper_poll_init(dev);
#define CP_2WHEEL_MOUSE_HACK 0x02
#define CP_2WHEEL_MOUSE_HACK_ON 0x04
+#define VA_INVAL_LOGICAL_BOUNDARY 0x08
+
/*
* Some USB barcode readers from cypress have usage min and usage max in
* the wrong order
*/
-static __u8 *cp_report_fixup(struct hid_device *hdev, __u8 *rdesc,
+static __u8 *cp_rdesc_fixup(struct hid_device *hdev, __u8 *rdesc,
unsigned int *rsize)
{
- unsigned long quirks = (unsigned long)hid_get_drvdata(hdev);
unsigned int i;
- if (!(quirks & CP_RDESC_SWAPPED_MIN_MAX))
- return rdesc;
-
if (*rsize < 4)
return rdesc;
return rdesc;
}
+static __u8 *va_logical_boundary_fixup(struct hid_device *hdev, __u8 *rdesc,
+ unsigned int *rsize)
+{
+ /*
+ * Varmilo VA104M (with VID Cypress and device ID 07B1) incorrectly
+ * reports Logical Minimum of its Consumer Control device as 572
+ * (0x02 0x3c). Fix this by setting its Logical Minimum to zero.
+ */
+ if (*rsize == 25 &&
+ rdesc[0] == 0x05 && rdesc[1] == 0x0c &&
+ rdesc[2] == 0x09 && rdesc[3] == 0x01 &&
+ rdesc[6] == 0x19 && rdesc[7] == 0x00 &&
+ rdesc[11] == 0x16 && rdesc[12] == 0x3c && rdesc[13] == 0x02) {
+ hid_info(hdev,
+ "fixing up varmilo VA104M consumer control report descriptor\n");
+ rdesc[12] = 0x00;
+ rdesc[13] = 0x00;
+ }
+ return rdesc;
+}
+
+static __u8 *cp_report_fixup(struct hid_device *hdev, __u8 *rdesc,
+ unsigned int *rsize)
+{
+ unsigned long quirks = (unsigned long)hid_get_drvdata(hdev);
+
+ if (quirks & CP_RDESC_SWAPPED_MIN_MAX)
+ rdesc = cp_rdesc_fixup(hdev, rdesc, rsize);
+ if (quirks & VA_INVAL_LOGICAL_BOUNDARY)
+ rdesc = va_logical_boundary_fixup(hdev, rdesc, rsize);
+
+ return rdesc;
+}
+
static int cp_input_mapped(struct hid_device *hdev, struct hid_input *hi,
struct hid_field *field, struct hid_usage *usage,
unsigned long **bit, int *max)
.driver_data = CP_RDESC_SWAPPED_MIN_MAX },
{ HID_USB_DEVICE(USB_VENDOR_ID_CYPRESS, USB_DEVICE_ID_CYPRESS_MOUSE),
.driver_data = CP_2WHEEL_MOUSE_HACK },
+ { HID_USB_DEVICE(USB_VENDOR_ID_CYPRESS, USB_DEVICE_ID_CYPRESS_VARMILO_VA104M_07B1),
+ .driver_data = VA_INVAL_LOGICAL_BOUNDARY },
{ }
};
MODULE_DEVICE_TABLE(hid, cp_devices);
#define USB_DEVICE_ID_CYPRESS_BARCODE_4 0xed81
#define USB_DEVICE_ID_CYPRESS_TRUETOUCH 0xc001
+#define USB_DEVICE_ID_CYPRESS_VARMILO_VA104M_07B1 0X07b1
+
#define USB_VENDOR_ID_DATA_MODUL 0x7374
#define USB_VENDOR_ID_DATA_MODUL_EASYMAXTOUCH 0x1201
#define USB_VENDOR_ID_FRUCTEL 0x25B6
#define USB_DEVICE_ID_GAMETEL_MT_MODE 0x0002
+#define USB_VENDOR_ID_GAMEVICE 0x27F8
+#define USB_DEVICE_ID_GAMEVICE_GV186 0x0BBE
+#define USB_DEVICE_ID_GAMEVICE_KISHI 0x0BBF
+
#define USB_VENDOR_ID_GAMERON 0x0810
#define USB_DEVICE_ID_GAMERON_DUAL_PSX_ADAPTOR 0x0001
#define USB_DEVICE_ID_GAMERON_DUAL_PCS_ADAPTOR 0x0002
#define USB_DEVICE_ID_PENPOWER 0x00f4
#define USB_VENDOR_ID_GREENASIA 0x0e8f
+#define USB_DEVICE_ID_GREENASIA_DUAL_SAT_ADAPTOR 0x3010
#define USB_DEVICE_ID_GREENASIA_DUAL_USB_JOYPAD 0x3013
#define USB_VENDOR_ID_GRETAGMACBETH 0x0971
#define USB_VENDOR_ID_LOGITECH 0x046d
#define USB_DEVICE_ID_LOGITECH_AUDIOHUB 0x0a0e
#define USB_DEVICE_ID_LOGITECH_T651 0xb00c
+#define USB_DEVICE_ID_LOGITECH_DINOVO_EDGE_KBD 0xb309
#define USB_DEVICE_ID_LOGITECH_C007 0xc007
#define USB_DEVICE_ID_LOGITECH_C077 0xc077
#define USB_DEVICE_ID_LOGITECH_RECEIVER 0xc101
#define USB_VENDOR_ID_UGTIZER 0x2179
#define USB_DEVICE_ID_UGTIZER_TABLET_GP0610 0x0053
+#define USB_DEVICE_ID_UGTIZER_TABLET_GT5040 0x0077
#define USB_VENDOR_ID_VIEWSONIC 0x0543
#define USB_DEVICE_ID_VIEWSONIC_PD1011 0xe621
{ HID_BLUETOOTH_DEVICE(USB_VENDOR_ID_ASUSTEK,
USB_DEVICE_ID_ASUSTEK_T100CHI_KEYBOARD),
HID_BATTERY_QUIRK_IGNORE },
+ { HID_BLUETOOTH_DEVICE(USB_VENDOR_ID_LOGITECH,
+ USB_DEVICE_ID_LOGITECH_DINOVO_EDGE_KBD),
+ HID_BATTERY_QUIRK_IGNORE },
{}
};
#include "hid-ids.h"
+#define QUIRK_TOUCHPAD_ON_OFF_REPORT BIT(0)
+
+static __u8 *ite_report_fixup(struct hid_device *hdev, __u8 *rdesc, unsigned int *rsize)
+{
+ unsigned long quirks = (unsigned long)hid_get_drvdata(hdev);
+
+ if (quirks & QUIRK_TOUCHPAD_ON_OFF_REPORT) {
+ if (*rsize == 188 && rdesc[162] == 0x81 && rdesc[163] == 0x02) {
+ hid_info(hdev, "Fixing up ITE keyboard report descriptor\n");
+ rdesc[163] = HID_MAIN_ITEM_RELATIVE;
+ }
+ }
+
+ return rdesc;
+}
+
+static int ite_input_mapping(struct hid_device *hdev,
+ struct hid_input *hi, struct hid_field *field,
+ struct hid_usage *usage, unsigned long **bit,
+ int *max)
+{
+
+ unsigned long quirks = (unsigned long)hid_get_drvdata(hdev);
+
+ if ((quirks & QUIRK_TOUCHPAD_ON_OFF_REPORT) &&
+ (usage->hid & HID_USAGE_PAGE) == 0x00880000) {
+ if (usage->hid == 0x00880078) {
+ /* Touchpad on, userspace expects F22 for this */
+ hid_map_usage_clear(hi, usage, bit, max, EV_KEY, KEY_F22);
+ return 1;
+ }
+ if (usage->hid == 0x00880079) {
+ /* Touchpad off, userspace expects F23 for this */
+ hid_map_usage_clear(hi, usage, bit, max, EV_KEY, KEY_F23);
+ return 1;
+ }
+ return -1;
+ }
+
+ return 0;
+}
+
static int ite_event(struct hid_device *hdev, struct hid_field *field,
struct hid_usage *usage, __s32 value)
{
return 0;
}
+static int ite_probe(struct hid_device *hdev, const struct hid_device_id *id)
+{
+ int ret;
+
+ hid_set_drvdata(hdev, (void *)id->driver_data);
+
+ ret = hid_open_report(hdev);
+ if (ret)
+ return ret;
+
+ return hid_hw_start(hdev, HID_CONNECT_DEFAULT);
+}
+
static const struct hid_device_id ite_devices[] = {
{ HID_USB_DEVICE(USB_VENDOR_ID_ITE, USB_DEVICE_ID_ITE8595) },
{ HID_USB_DEVICE(USB_VENDOR_ID_258A, USB_DEVICE_ID_258A_6A88) },
/* ITE8595 USB kbd ctlr, with Synaptics touchpad connected to it. */
{ HID_DEVICE(BUS_USB, HID_GROUP_GENERIC,
USB_VENDOR_ID_SYNAPTICS,
- USB_DEVICE_ID_SYNAPTICS_ACER_SWITCH5_012) },
+ USB_DEVICE_ID_SYNAPTICS_ACER_SWITCH5_012),
+ .driver_data = QUIRK_TOUCHPAD_ON_OFF_REPORT },
/* ITE8910 USB kbd ctlr, with Synaptics touchpad connected to it. */
{ HID_DEVICE(BUS_USB, HID_GROUP_GENERIC,
USB_VENDOR_ID_SYNAPTICS,
static struct hid_driver ite_driver = {
.name = "itetech",
.id_table = ite_devices,
+ .probe = ite_probe,
+ .report_fixup = ite_report_fixup,
+ .input_mapping = ite_input_mapping,
.event = ite_event,
};
module_hid_driver(ite_driver);
0x25, 0x01, /* LOGICAL_MAX (1) */
0x75, 0x01, /* REPORT_SIZE (1) */
0x95, 0x04, /* REPORT_COUNT (4) */
- 0x81, 0x06, /* INPUT */
+ 0x81, 0x02, /* INPUT (Data,Var,Abs) */
0xC0, /* END_COLLECTION */
0xC0, /* END_COLLECTION */
};
schedule_work(&djrcv_dev->work);
}
+/*
+ * Some quad/bluetooth keyboards have a builtin touchpad in this case we see
+ * only 1 paired device with a device_type of REPORT_TYPE_KEYBOARD. For the
+ * touchpad to work we must also forward mouse input reports to the dj_hiddev
+ * created for the keyboard (instead of forwarding them to a second paired
+ * device with a device_type of REPORT_TYPE_MOUSE as we normally would).
+ */
+static const u16 kbd_builtin_touchpad_ids[] = {
+ 0xb309, /* Dinovo Edge */
+ 0xb30c, /* Dinovo Mini */
+};
+
static void logi_hidpp_dev_conn_notif_equad(struct hid_device *hdev,
struct hidpp_event *hidpp_report,
struct dj_workitem *workitem)
{
struct dj_receiver_dev *djrcv_dev = hid_get_drvdata(hdev);
+ int i, id;
workitem->type = WORKITEM_TYPE_PAIRED;
workitem->device_type = hidpp_report->params[HIDPP_PARAM_DEVICE_INFO] &
workitem->reports_supported |= STD_KEYBOARD | MULTIMEDIA |
POWER_KEYS | MEDIA_CENTER |
HIDPP;
+ id = (workitem->quad_id_msb << 8) | workitem->quad_id_lsb;
+ for (i = 0; i < ARRAY_SIZE(kbd_builtin_touchpad_ids); i++) {
+ if (id == kbd_builtin_touchpad_ids[i]) {
+ workitem->reports_supported |= STD_MOUSE;
+ break;
+ }
+ }
break;
case REPORT_TYPE_MOUSE:
workitem->reports_supported |= STD_MOUSE | HIDPP;
#define HIDPP_CAPABILITY_BATTERY_LEVEL_STATUS BIT(3)
#define HIDPP_CAPABILITY_BATTERY_VOLTAGE BIT(4)
+#define lg_map_key_clear(c) hid_map_usage_clear(hi, usage, bit, max, EV_KEY, (c))
+
/*
* There are two hidpp protocols in use, the first version hidpp10 is known
* as register access protocol or RAP, the second version hidpp20 is known as
return g920_ff_set_autocenter(hidpp, data);
}
+/* -------------------------------------------------------------------------- */
+/* Logitech Dinovo Mini keyboard with builtin touchpad */
+/* -------------------------------------------------------------------------- */
+#define DINOVO_MINI_PRODUCT_ID 0xb30c
+
+static int lg_dinovo_input_mapping(struct hid_device *hdev, struct hid_input *hi,
+ struct hid_field *field, struct hid_usage *usage,
+ unsigned long **bit, int *max)
+{
+ if ((usage->hid & HID_USAGE_PAGE) != HID_UP_LOGIVENDOR)
+ return 0;
+
+ switch (usage->hid & HID_USAGE) {
+ case 0x00d: lg_map_key_clear(KEY_MEDIA); break;
+ default:
+ return 0;
+ }
+ return 1;
+}
+
/* -------------------------------------------------------------------------- */
/* HID++1.0 devices which use HID++ reports for their wheels */
/* -------------------------------------------------------------------------- */
field->application != HID_GD_MOUSE)
return m560_input_mapping(hdev, hi, field, usage, bit, max);
+ if (hdev->product == DINOVO_MINI_PRODUCT_ID)
+ return lg_dinovo_input_mapping(hdev, hi, field, usage, bit, max);
+
return 0;
}
LDJ_DEVICE(0x405e), .driver_data = HIDPP_QUIRK_HI_RES_SCROLL_X2121 },
{ /* Mouse Logitech MX Anywhere 2 */
LDJ_DEVICE(0x404a), .driver_data = HIDPP_QUIRK_HI_RES_SCROLL_X2121 },
+ { LDJ_DEVICE(0x4072), .driver_data = HIDPP_QUIRK_HI_RES_SCROLL_X2121 },
{ LDJ_DEVICE(0xb013), .driver_data = HIDPP_QUIRK_HI_RES_SCROLL_X2121 },
{ LDJ_DEVICE(0xb018), .driver_data = HIDPP_QUIRK_HI_RES_SCROLL_X2121 },
{ LDJ_DEVICE(0xb01f), .driver_data = HIDPP_QUIRK_HI_RES_SCROLL_X2121 },
{ /* Keyboard MX5000 (Bluetooth-receiver in HID proxy mode) */
LDJ_DEVICE(0xb305),
.driver_data = HIDPP_QUIRK_HIDPP_CONSUMER_VENDOR_KEYS },
+ { /* Dinovo Edge (Bluetooth-receiver in HID proxy mode) */
+ LDJ_DEVICE(0xb309),
+ .driver_data = HIDPP_QUIRK_HIDPP_CONSUMER_VENDOR_KEYS },
{ /* Keyboard MX5500 (Bluetooth-receiver in HID proxy mode) */
LDJ_DEVICE(0xb30b),
.driver_data = HIDPP_QUIRK_HIDPP_CONSUMER_VENDOR_KEYS },
{ /* MX5000 keyboard over Bluetooth */
HID_BLUETOOTH_DEVICE(USB_VENDOR_ID_LOGITECH, 0xb305),
.driver_data = HIDPP_QUIRK_HIDPP_CONSUMER_VENDOR_KEYS },
+ { /* Dinovo Edge keyboard over Bluetooth */
+ HID_BLUETOOTH_DEVICE(USB_VENDOR_ID_LOGITECH, 0xb309),
+ .driver_data = HIDPP_QUIRK_HIDPP_CONSUMER_VENDOR_KEYS },
{ /* MX5500 keyboard over Bluetooth */
HID_BLUETOOTH_DEVICE(USB_VENDOR_ID_LOGITECH, 0xb30b),
.driver_data = HIDPP_QUIRK_HIDPP_CONSUMER_VENDOR_KEYS },
MCP2221_ALT_F_NOT_GPIOD = 0xEF,
};
+/* MCP GPIO direction encoding */
+enum {
+ MCP2221_DIR_OUT = 0x00,
+ MCP2221_DIR_IN = 0x01,
+};
+
+#define MCP_NGPIO 4
+
+/* MCP GPIO set command layout */
+struct mcp_set_gpio {
+ u8 cmd;
+ u8 dummy;
+ struct {
+ u8 change_value;
+ u8 value;
+ u8 change_direction;
+ u8 direction;
+ } gpio[MCP_NGPIO];
+} __packed;
+
+/* MCP GPIO get command layout */
+struct mcp_get_gpio {
+ u8 cmd;
+ u8 dummy;
+ struct {
+ u8 direction;
+ u8 value;
+ } gpio[MCP_NGPIO];
+} __packed;
+
/*
* There is no way to distinguish responses. Therefore next command
* is sent only after response to previous has been received. Mutex
mcp->txbuf[0] = MCP2221_GPIO_GET;
- mcp->gp_idx = (offset + 1) * 2;
+ mcp->gp_idx = offsetof(struct mcp_get_gpio, gpio[offset].value);
mutex_lock(&mcp->lock);
ret = mcp_send_data_req_status(mcp, mcp->txbuf, 1);
memset(mcp->txbuf, 0, 18);
mcp->txbuf[0] = MCP2221_GPIO_SET;
- mcp->gp_idx = ((offset + 1) * 4) - 1;
+ mcp->gp_idx = offsetof(struct mcp_set_gpio, gpio[offset].value);
mcp->txbuf[mcp->gp_idx - 1] = 1;
mcp->txbuf[mcp->gp_idx] = !!value;
memset(mcp->txbuf, 0, 18);
mcp->txbuf[0] = MCP2221_GPIO_SET;
- mcp->gp_idx = (offset + 1) * 5;
+ mcp->gp_idx = offsetof(struct mcp_set_gpio, gpio[offset].direction);
mcp->txbuf[mcp->gp_idx - 1] = 1;
mcp->txbuf[mcp->gp_idx] = val;
struct mcp2221 *mcp = gpiochip_get_data(gc);
mutex_lock(&mcp->lock);
- ret = mcp_gpio_dir_set(mcp, offset, 0);
+ ret = mcp_gpio_dir_set(mcp, offset, MCP2221_DIR_IN);
mutex_unlock(&mcp->lock);
return ret;
struct mcp2221 *mcp = gpiochip_get_data(gc);
mutex_lock(&mcp->lock);
- ret = mcp_gpio_dir_set(mcp, offset, 1);
+ ret = mcp_gpio_dir_set(mcp, offset, MCP2221_DIR_OUT);
mutex_unlock(&mcp->lock);
/* Can't configure as output, bailout early */
mcp->txbuf[0] = MCP2221_GPIO_GET;
- mcp->gp_idx = (offset + 1) * 2;
+ mcp->gp_idx = offsetof(struct mcp_get_gpio, gpio[offset].direction);
mutex_lock(&mcp->lock);
ret = mcp_send_data_req_status(mcp, mcp->txbuf, 1);
if (ret)
return ret;
- if (mcp->gpio_dir)
+ if (mcp->gpio_dir == MCP2221_DIR_IN)
return GPIO_LINE_DIRECTION_IN;
return GPIO_LINE_DIRECTION_OUT;
mcp->status = -ENOENT;
} else {
mcp->status = !!data[mcp->gp_idx];
- mcp->gpio_dir = !!data[mcp->gp_idx + 1];
+ mcp->gpio_dir = data[mcp->gp_idx + 1];
}
break;
default:
mcp->gc->get_direction = mcp_gpio_get_direction;
mcp->gc->set = mcp_gpio_set;
mcp->gc->get = mcp_gpio_get;
- mcp->gc->ngpio = 4;
+ mcp->gc->ngpio = MCP_NGPIO;
mcp->gc->base = -1;
mcp->gc->can_sleep = 1;
mcp->gc->parent = &hdev->dev;
{ HID_USB_DEVICE(USB_VENDOR_ID_FORMOSA, USB_DEVICE_ID_FORMOSA_IR_RECEIVER), HID_QUIRK_NO_INIT_REPORTS },
{ HID_USB_DEVICE(USB_VENDOR_ID_FREESCALE, USB_DEVICE_ID_FREESCALE_MX28), HID_QUIRK_NOGET },
{ HID_USB_DEVICE(USB_VENDOR_ID_FUTABA, USB_DEVICE_ID_LED_DISPLAY), HID_QUIRK_NO_INIT_REPORTS },
+ { HID_USB_DEVICE(USB_VENDOR_ID_GREENASIA, USB_DEVICE_ID_GREENASIA_DUAL_SAT_ADAPTOR), HID_QUIRK_MULTI_INPUT },
{ HID_USB_DEVICE(USB_VENDOR_ID_GREENASIA, USB_DEVICE_ID_GREENASIA_DUAL_USB_JOYPAD), HID_QUIRK_MULTI_INPUT },
+ { HID_BLUETOOTH_DEVICE(USB_VENDOR_ID_GAMEVICE, USB_DEVICE_ID_GAMEVICE_GV186),
+ HID_QUIRK_INCREMENT_USAGE_ON_DUPLICATE },
+ { HID_USB_DEVICE(USB_VENDOR_ID_GAMEVICE, USB_DEVICE_ID_GAMEVICE_KISHI),
+ HID_QUIRK_INCREMENT_USAGE_ON_DUPLICATE },
{ HID_USB_DEVICE(USB_VENDOR_ID_HAPP, USB_DEVICE_ID_UGCI_DRIVING), HID_QUIRK_BADPAD | HID_QUIRK_MULTI_INPUT },
{ HID_USB_DEVICE(USB_VENDOR_ID_HAPP, USB_DEVICE_ID_UGCI_FIGHTING), HID_QUIRK_BADPAD | HID_QUIRK_MULTI_INPUT },
{ HID_USB_DEVICE(USB_VENDOR_ID_HAPP, USB_DEVICE_ID_UGCI_FLYING), HID_QUIRK_BADPAD | HID_QUIRK_MULTI_INPUT },
return 1;
ptr = raw_data;
- ptr++; /* Skip report id */
+ if (report->id)
+ ptr++; /* Skip report id */
spin_lock_irqsave(&pdata->lock, flags);
USB_DEVICE_ID_UCLOGIC_DRAWIMAGE_G3) },
{ HID_USB_DEVICE(USB_VENDOR_ID_UGTIZER,
USB_DEVICE_ID_UGTIZER_TABLET_GP0610) },
+ { HID_USB_DEVICE(USB_VENDOR_ID_UGTIZER,
+ USB_DEVICE_ID_UGTIZER_TABLET_GT5040) },
{ HID_USB_DEVICE(USB_VENDOR_ID_UGEE,
USB_DEVICE_ID_UGEE_TABLET_G5) },
{ HID_USB_DEVICE(USB_VENDOR_ID_UGEE,
break;
case VID_PID(USB_VENDOR_ID_UGTIZER,
USB_DEVICE_ID_UGTIZER_TABLET_GP0610):
+ case VID_PID(USB_VENDOR_ID_UGTIZER,
+ USB_DEVICE_ID_UGTIZER_TABLET_GT5040):
case VID_PID(USB_VENDOR_ID_UGEE,
USB_DEVICE_ID_UGEE_XPPEN_TABLET_G540):
case VID_PID(USB_VENDOR_ID_UGEE,
}
}
+static void i2c_hid_acpi_shutdown(struct device *dev)
+{
+ acpi_device_set_power(ACPI_COMPANION(dev), ACPI_STATE_D3_COLD);
+}
+
static const struct acpi_device_id i2c_hid_acpi_match[] = {
{"ACPI0C50", 0 },
{"PNP0C50", 0 },
static inline void i2c_hid_acpi_fix_up_power(struct device *dev) {}
static inline void i2c_hid_acpi_enable_wakeup(struct device *dev) {}
+
+static inline void i2c_hid_acpi_shutdown(struct device *dev) {}
#endif
#ifdef CONFIG_OF
i2c_hid_set_power(client, I2C_HID_PWR_SLEEP);
free_irq(client->irq, ihid);
+
+ i2c_hid_acpi_shutdown(&client->dev);
}
#ifdef CONFIG_PM_SLEEP
/*
* Hyper-V does not provide a way to change the connect CPU once
- * it is set; we must prevent the connect CPU from going offline.
+ * it is set; we must prevent the connect CPU from going offline
+ * while the VM is running normally. But in the panic or kexec()
+ * path where the vmbus is already disconnected, the CPU must be
+ * allowed to shut down.
*/
- if (cpu == VMBUS_CONNECT_CPU)
+ if (cpu == VMBUS_CONNECT_CPU &&
+ vmbus_connection.conn_state == CONNECTED)
return -EBUSY;
/*
enum hwmon_sensor_types type,
u32 attr, int channel)
{
- return 0444;
+ return 0440;
}
static int energy_accumulator(void *p)
#include <linux/hwmon.h>
#include <linux/workqueue.h>
#include <linux/err.h>
+#include <linux/bits.h>
/* data port used by Apple SMC */
#define APPLESMC_DATA_PORT 0x300
#define APPLESMC_MAX_DATA_LENGTH 32
-/* wait up to 128 ms for a status change. */
-#define APPLESMC_MIN_WAIT 0x0010
-#define APPLESMC_RETRY_WAIT 0x0100
-#define APPLESMC_MAX_WAIT 0x20000
+/* Apple SMC status bits */
+#define SMC_STATUS_AWAITING_DATA BIT(0) /* SMC has data waiting to be read */
+#define SMC_STATUS_IB_CLOSED BIT(1) /* Will ignore any input */
+#define SMC_STATUS_BUSY BIT(2) /* Command in progress */
+
+/* Initial wait is 8us */
+#define APPLESMC_MIN_WAIT 0x0008
#define APPLESMC_READ_CMD 0x10
#define APPLESMC_WRITE_CMD 0x11
static struct workqueue_struct *applesmc_led_wq;
/*
- * wait_read - Wait for a byte to appear on SMC port. Callers must
- * hold applesmc_lock.
+ * Wait for specific status bits with a mask on the SMC.
+ * Used before all transactions.
+ * This does 10 fast loops of 8us then exponentially backs off for a
+ * minimum total wait of 262ms. Depending on usleep_range this could
+ * run out past 500ms.
*/
-static int wait_read(void)
+
+static int wait_status(u8 val, u8 mask)
{
- unsigned long end = jiffies + (APPLESMC_MAX_WAIT * HZ) / USEC_PER_SEC;
u8 status;
int us;
+ int i;
- for (us = APPLESMC_MIN_WAIT; us < APPLESMC_MAX_WAIT; us <<= 1) {
- usleep_range(us, us * 16);
+ us = APPLESMC_MIN_WAIT;
+ for (i = 0; i < 24 ; i++) {
status = inb(APPLESMC_CMD_PORT);
- /* read: wait for smc to settle */
- if (status & 0x01)
+ if ((status & mask) == val)
return 0;
- /* timeout: give up */
- if (time_after(jiffies, end))
- break;
+ usleep_range(us, us * 2);
+ if (i > 9)
+ us <<= 1;
}
-
- pr_warn("wait_read() fail: 0x%02x\n", status);
return -EIO;
}
-/*
- * send_byte - Write to SMC port, retrying when necessary. Callers
- * must hold applesmc_lock.
- */
+/* send_byte - Write to SMC data port. Callers must hold applesmc_lock. */
+
static int send_byte(u8 cmd, u16 port)
{
- u8 status;
- int us;
- unsigned long end = jiffies + (APPLESMC_MAX_WAIT * HZ) / USEC_PER_SEC;
+ int status;
+
+ status = wait_status(0, SMC_STATUS_IB_CLOSED);
+ if (status)
+ return status;
+ /*
+ * This needs to be a separate read looking for bit 0x04
+ * after bit 0x02 falls. If consolidated with the wait above
+ * this extra read may not happen if status returns both
+ * simultaneously and this would appear to be required.
+ */
+ status = wait_status(SMC_STATUS_BUSY, SMC_STATUS_BUSY);
+ if (status)
+ return status;
outb(cmd, port);
- for (us = APPLESMC_MIN_WAIT; us < APPLESMC_MAX_WAIT; us <<= 1) {
- usleep_range(us, us * 16);
- status = inb(APPLESMC_CMD_PORT);
- /* write: wait for smc to settle */
- if (status & 0x02)
- continue;
- /* ready: cmd accepted, return */
- if (status & 0x04)
- return 0;
- /* timeout: give up */
- if (time_after(jiffies, end))
- break;
- /* busy: long wait and resend */
- udelay(APPLESMC_RETRY_WAIT);
- outb(cmd, port);
- }
-
- pr_warn("send_byte(0x%02x, 0x%04x) fail: 0x%02x\n", cmd, port, status);
- return -EIO;
+ return 0;
}
+/* send_command - Write a command to the SMC. Callers must hold applesmc_lock. */
+
static int send_command(u8 cmd)
{
- return send_byte(cmd, APPLESMC_CMD_PORT);
+ int ret;
+
+ ret = wait_status(0, SMC_STATUS_IB_CLOSED);
+ if (ret)
+ return ret;
+ outb(cmd, APPLESMC_CMD_PORT);
+ return 0;
+}
+
+/*
+ * Based on logic from the Apple driver. This is issued before any interaction
+ * If busy is stuck high, issue a read command to reset the SMC state machine.
+ * If busy is stuck high after the command then the SMC is jammed.
+ */
+
+static int smc_sane(void)
+{
+ int ret;
+
+ ret = wait_status(0, SMC_STATUS_BUSY);
+ if (!ret)
+ return ret;
+ ret = send_command(APPLESMC_READ_CMD);
+ if (ret)
+ return ret;
+ return wait_status(0, SMC_STATUS_BUSY);
}
static int send_argument(const char *key)
{
u8 status, data = 0;
int i;
+ int ret;
+
+ ret = smc_sane();
+ if (ret)
+ return ret;
if (send_command(cmd) || send_argument(key)) {
pr_warn("%.4s: read arg fail\n", key);
}
for (i = 0; i < len; i++) {
- if (wait_read()) {
+ if (wait_status(SMC_STATUS_AWAITING_DATA | SMC_STATUS_BUSY,
+ SMC_STATUS_AWAITING_DATA | SMC_STATUS_BUSY)) {
pr_warn("%.4s: read data[%d] fail\n", key, i);
return -EIO;
}
for (i = 0; i < 16; i++) {
udelay(APPLESMC_MIN_WAIT);
status = inb(APPLESMC_CMD_PORT);
- if (!(status & 0x01))
+ if (!(status & SMC_STATUS_AWAITING_DATA))
break;
data = inb(APPLESMC_DATA_PORT);
}
if (i)
pr_warn("flushed %d bytes, last value is: %d\n", i, data);
- return 0;
+ return wait_status(0, SMC_STATUS_BUSY);
}
static int write_smc(u8 cmd, const char *key, const u8 *buffer, u8 len)
{
int i;
+ int ret;
+
+ ret = smc_sane();
+ if (ret)
+ return ret;
if (send_command(cmd) || send_argument(key)) {
pr_warn("%s: write arg fail\n", key);
}
}
- return 0;
+ return wait_status(0, SMC_STATUS_BUSY);
}
static int read_register_count(unsigned int *count)
switch (idx) {
case MAX20730_DEBUGFS_VOUT_MIN:
ret = VOLT_FROM_REG(data->mfr_voutmin * 10000);
- len = snprintf(tbuf, DEBUG_FS_DATA_MAX, "%d.%d\n",
- ret / 10000, ret % 10000);
+ len = scnprintf(tbuf, DEBUG_FS_DATA_MAX, "%d.%d\n",
+ ret / 10000, ret % 10000);
break;
case MAX20730_DEBUGFS_FREQUENCY:
val = (data->mfr_devset1 & MAX20730_MFR_DEVSET1_FSW_MASK)
ret = 800;
else
ret = 900;
- len = snprintf(tbuf, DEBUG_FS_DATA_MAX, "%d\n", ret);
+ len = scnprintf(tbuf, DEBUG_FS_DATA_MAX, "%d\n", ret);
break;
case MAX20730_DEBUGFS_PG_DELAY:
val = (data->mfr_devset1 & MAX20730_MFR_DEVSET1_TSTAT_MASK)
case MAX20730_DEBUGFS_OC_PROTECT_MODE:
ret = (data->mfr_devset2 & MAX20730_MFR_DEVSET2_OCPM_MASK)
>> MAX20730_MFR_DEVSET2_OCPM_BIT_POS;
- len = snprintf(tbuf, DEBUG_FS_DATA_MAX, "%d\n", ret);
+ len = scnprintf(tbuf, DEBUG_FS_DATA_MAX, "%d\n", ret);
break;
case MAX20730_DEBUGFS_SS_TIMING:
val = (data->mfr_devset2 & MAX20730_MFR_DEVSET2_SS_MASK)
case MAX20730_DEBUGFS_IMAX:
ret = (data->mfr_devset2 & MAX20730_MFR_DEVSET2_IMAX_MASK)
>> MAX20730_MFR_DEVSET2_IMAX_BIT_POS;
- len = snprintf(tbuf, DEBUG_FS_DATA_MAX, "%d\n", ret);
+ len = scnprintf(tbuf, DEBUG_FS_DATA_MAX, "%d\n", ret);
break;
case MAX20730_DEBUGFS_OPERATION:
ret = i2c_smbus_read_byte_data(psu->client, PMBUS_OPERATION);
if (ret < 0)
return ret;
- len = snprintf(tbuf, DEBUG_FS_DATA_MAX, "%d\n", ret);
+ len = scnprintf(tbuf, DEBUG_FS_DATA_MAX, "%d\n", ret);
break;
case MAX20730_DEBUGFS_ON_OFF_CONFIG:
ret = i2c_smbus_read_byte_data(psu->client, PMBUS_ON_OFF_CONFIG);
if (ret < 0)
return ret;
- len = snprintf(tbuf, DEBUG_FS_DATA_MAX, "%d\n", ret);
+ len = scnprintf(tbuf, DEBUG_FS_DATA_MAX, "%d\n", ret);
break;
case MAX20730_DEBUGFS_SMBALERT_MASK:
ret = i2c_smbus_read_word_data(psu->client,
PMBUS_SMB_ALERT_MASK);
if (ret < 0)
return ret;
- len = snprintf(tbuf, DEBUG_FS_DATA_MAX, "%d\n", ret);
+ len = scnprintf(tbuf, DEBUG_FS_DATA_MAX, "%d\n", ret);
break;
case MAX20730_DEBUGFS_VOUT_MODE:
ret = i2c_smbus_read_byte_data(psu->client, PMBUS_VOUT_MODE);
if (ret < 0)
return ret;
- len = snprintf(tbuf, DEBUG_FS_DATA_MAX, "%d\n", ret);
+ len = scnprintf(tbuf, DEBUG_FS_DATA_MAX, "%d\n", ret);
break;
case MAX20730_DEBUGFS_VOUT_COMMAND:
ret = i2c_smbus_read_word_data(psu->client, PMBUS_VOUT_COMMAND);
return ret;
ret = VOLT_FROM_REG(ret * 10000);
- len = snprintf(tbuf, DEBUG_FS_DATA_MAX,
- "%d.%d\n", ret / 10000, ret % 10000);
+ len = scnprintf(tbuf, DEBUG_FS_DATA_MAX,
+ "%d.%d\n", ret / 10000, ret % 10000);
break;
case MAX20730_DEBUGFS_VOUT_MAX:
ret = i2c_smbus_read_word_data(psu->client, PMBUS_VOUT_MAX);
return ret;
ret = VOLT_FROM_REG(ret * 10000);
- len = snprintf(tbuf, DEBUG_FS_DATA_MAX,
- "%d.%d\n", ret / 10000, ret % 10000);
+ len = scnprintf(tbuf, DEBUG_FS_DATA_MAX,
+ "%d.%d\n", ret / 10000, ret % 10000);
break;
default:
len = strlcpy(tbuf, "Invalid\n", DEBUG_FS_DATA_MAX);
struct i2c_client *client = to_i2c_client(dev->parent);
struct pmbus_sensor *sensor = to_pmbus_sensor(devattr);
struct pmbus_data *data = i2c_get_clientdata(client);
+ ssize_t ret;
+ mutex_lock(&data->update_lock);
pmbus_update_sensor_data(client, sensor);
if (sensor->data < 0)
- return sensor->data;
-
- return snprintf(buf, PAGE_SIZE, "%lld\n", pmbus_reg2data(data, sensor));
+ ret = sensor->data;
+ else
+ ret = snprintf(buf, PAGE_SIZE, "%lld\n", pmbus_reg2data(data, sensor));
+ mutex_unlock(&data->update_lock);
+ return ret;
}
static ssize_t pmbus_set_sensor(struct device *dev,
int val;
struct i2c_client *client = to_i2c_client(dev->parent);
struct pmbus_samples_reg *reg = to_samples_reg(devattr);
+ struct pmbus_data *data = i2c_get_clientdata(client);
+ mutex_lock(&data->update_lock);
val = _pmbus_read_word_data(client, reg->page, 0xff, reg->attr->reg);
+ mutex_unlock(&data->update_lock);
if (val < 0)
return val;
static void sample_timer(struct timer_list *t)
{
struct pwm_fan_ctx *ctx = from_timer(ctx, t, rpm_timer);
+ unsigned int delta = ktime_ms_delta(ktime_get(), ctx->sample_start);
int pulses;
- u64 tmp;
- pulses = atomic_read(&ctx->pulses);
- atomic_sub(pulses, &ctx->pulses);
- tmp = (u64)pulses * ktime_ms_delta(ktime_get(), ctx->sample_start) * 60;
- do_div(tmp, ctx->pulses_per_revolution * 1000);
- ctx->rpm = tmp;
+ if (delta) {
+ pulses = atomic_read(&ctx->pulses);
+ atomic_sub(pulses, &ctx->pulses);
+ ctx->rpm = (unsigned int)(pulses * 1000 * 60) /
+ (ctx->pulses_per_revolution * delta);
+
+ ctx->sample_start = ktime_get();
+ }
- ctx->sample_start = ktime_get();
mod_timer(&ctx->rpm_timer, jiffies + HZ);
}
struct cpuidle_state *state = &drv->states[index];
unsigned long eax = flg2MWAIT(state->flags);
unsigned long ecx = 1; /* break on interrupt flag */
- bool tick;
-
- if (!static_cpu_has(X86_FEATURE_ARAT)) {
- /*
- * Switch over to one-shot tick broadcast if the target C-state
- * is deeper than C1.
- */
- if ((eax >> MWAIT_SUBSTATE_SIZE) & MWAIT_CSTATE_MASK) {
- tick = true;
- tick_broadcast_enter();
- } else {
- tick = false;
- }
- }
mwait_idle_with_hints(eax, ecx);
- if (!static_cpu_has(X86_FEATURE_ARAT) && tick)
- tick_broadcast_exit();
-
return index;
}
return false;
}
+static bool __init intel_idle_state_needs_timer_stop(struct cpuidle_state *state)
+{
+ unsigned long eax = flg2MWAIT(state->flags);
+
+ if (boot_cpu_has(X86_FEATURE_ARAT))
+ return false;
+
+ /*
+ * Switch over to one-shot tick broadcast if the target C-state
+ * is deeper than C1.
+ */
+ return !!((eax >> MWAIT_SUBSTATE_SIZE) & MWAIT_CSTATE_MASK);
+}
+
static void __init intel_idle_init_cstates_acpi(struct cpuidle_driver *drv)
{
int cstate, limit = min_t(int, CPUIDLE_STATE_MAX, acpi_state_table.count);
if (disabled_states_mask & BIT(cstate))
state->flags |= CPUIDLE_FLAG_OFF;
+ if (intel_idle_state_needs_timer_stop(state))
+ state->flags |= CPUIDLE_FLAG_TIMER_STOP;
+
state->enter = intel_idle;
state->enter_s2idle = intel_idle_s2idle;
}
!(cpuidle_state_table[cstate].flags & CPUIDLE_FLAG_ALWAYS_ENABLE)))
drv->states[drv->state_count].flags |= CPUIDLE_FLAG_OFF;
+ if (intel_idle_state_needs_timer_stop(&drv->states[drv->state_count]))
+ drv->states[drv->state_count].flags |= CPUIDLE_FLAG_TIMER_STOP;
+
drv->state_count++;
}
KX_MAX_CHIPS /* this must be last */
};
+enum kx_acpi_type {
+ ACPI_GENERIC,
+ ACPI_SMO8500,
+ ACPI_KIOX010A,
+};
+
struct kxcjk1013_data {
struct i2c_client *client;
struct iio_trigger *dready_trig;
bool motion_trigger_on;
int64_t timestamp;
enum kx_chipset chipset;
- bool is_smo8500_device;
+ enum kx_acpi_type acpi_type;
};
enum kxcjk1013_axis {
{19163, 1, 0},
{38326, 0, 1} };
+#ifdef CONFIG_ACPI
+enum kiox010a_fn_index {
+ KIOX010A_SET_LAPTOP_MODE = 1,
+ KIOX010A_SET_TABLET_MODE = 2,
+};
+
+static int kiox010a_dsm(struct device *dev, int fn_index)
+{
+ acpi_handle handle = ACPI_HANDLE(dev);
+ guid_t kiox010a_dsm_guid;
+ union acpi_object *obj;
+
+ if (!handle)
+ return -ENODEV;
+
+ guid_parse("1f339696-d475-4e26-8cad-2e9f8e6d7a91", &kiox010a_dsm_guid);
+
+ obj = acpi_evaluate_dsm(handle, &kiox010a_dsm_guid, 1, fn_index, NULL);
+ if (!obj)
+ return -EIO;
+
+ ACPI_FREE(obj);
+ return 0;
+}
+#endif
+
static int kxcjk1013_set_mode(struct kxcjk1013_data *data,
enum kxcjk1013_mode mode)
{
{
int ret;
+#ifdef CONFIG_ACPI
+ if (data->acpi_type == ACPI_KIOX010A) {
+ /* Make sure the kbd and touchpad on 2-in-1s using 2 KXCJ91008-s work */
+ kiox010a_dsm(&data->client->dev, KIOX010A_SET_LAPTOP_MODE);
+ }
+#endif
+
ret = i2c_smbus_read_byte_data(data->client, KXCJK1013_REG_WHO_AM_I);
if (ret < 0) {
dev_err(&data->client->dev, "Error reading who_am_i\n");
static const char *kxcjk1013_match_acpi_device(struct device *dev,
enum kx_chipset *chipset,
- bool *is_smo8500_device)
+ enum kx_acpi_type *acpi_type)
{
const struct acpi_device_id *id;
return NULL;
if (strcmp(id->id, "SMO8500") == 0)
- *is_smo8500_device = true;
+ *acpi_type = ACPI_SMO8500;
+ else if (strcmp(id->id, "KIOX010A") == 0)
+ *acpi_type = ACPI_KIOX010A;
*chipset = (enum kx_chipset)id->driver_data;
} else if (ACPI_HANDLE(&client->dev)) {
name = kxcjk1013_match_acpi_device(&client->dev,
&data->chipset,
- &data->is_smo8500_device);
+ &data->acpi_type);
} else
return -ENODEV;
indio_dev->modes = INDIO_DIRECT_MODE;
indio_dev->info = &kxcjk1013_info;
- if (client->irq > 0 && !data->is_smo8500_device) {
+ if (client->irq > 0 && data->acpi_type != ACPI_SMO8500) {
ret = devm_request_threaded_irq(&client->dev, client->irq,
kxcjk1013_data_rdy_trig_poll,
kxcjk1013_event_handler,
#define JZ4725B_ADC_BATTERY_HIGH_VREF_BITS 10
#define JZ4740_ADC_BATTERY_HIGH_VREF (7500 * 0.986)
#define JZ4740_ADC_BATTERY_HIGH_VREF_BITS 12
-#define JZ4770_ADC_BATTERY_VREF 6600
+#define JZ4770_ADC_BATTERY_VREF 1200
#define JZ4770_ADC_BATTERY_VREF_BITS 12
#define JZ_ADC_IRQ_AUX BIT(0)
mutex_unlock(&adc->lock);
}
-static void ingenic_adc_enable(struct ingenic_adc *adc,
- int engine,
- bool enabled)
+static void ingenic_adc_enable_unlocked(struct ingenic_adc *adc,
+ int engine,
+ bool enabled)
{
u8 val;
- mutex_lock(&adc->lock);
val = readb(adc->base + JZ_ADC_REG_ENABLE);
if (enabled)
val &= ~BIT(engine);
writeb(val, adc->base + JZ_ADC_REG_ENABLE);
+}
+
+static void ingenic_adc_enable(struct ingenic_adc *adc,
+ int engine,
+ bool enabled)
+{
+ mutex_lock(&adc->lock);
+ ingenic_adc_enable_unlocked(adc, engine, enabled);
mutex_unlock(&adc->lock);
}
static int ingenic_adc_capture(struct ingenic_adc *adc,
int engine)
{
+ u32 cfg;
u8 val;
int ret;
- ingenic_adc_enable(adc, engine, true);
+ /*
+ * Disable CMD_SEL temporarily, because it causes wrong VBAT readings,
+ * probably due to the switch of VREF. We must keep the lock here to
+ * avoid races with the buffer enable/disable functions.
+ */
+ mutex_lock(&adc->lock);
+ cfg = readl(adc->base + JZ_ADC_REG_CFG);
+ writel(cfg & ~JZ_ADC_REG_CFG_CMD_SEL, adc->base + JZ_ADC_REG_CFG);
+
+ ingenic_adc_enable_unlocked(adc, engine, true);
ret = readb_poll_timeout(adc->base + JZ_ADC_REG_ENABLE, val,
!(val & BIT(engine)), 250, 1000);
if (ret)
- ingenic_adc_enable(adc, engine, false);
+ ingenic_adc_enable_unlocked(adc, engine, false);
+
+ writel(cfg, adc->base + JZ_ADC_REG_CFG);
+ mutex_unlock(&adc->lock);
return ret;
}
#include <linux/err.h>
#include <linux/kernel.h>
#include <linux/module.h>
-#include <linux/of.h>
-#include <linux/of_device.h>
+#include <linux/mod_devicetable.h>
#include <linux/platform_device.h>
+#include <linux/property.h>
#include <linux/iopoll.h>
#include <linux/io.h>
#include <linux/iio/iio.h>
goto err_disable_clk;
}
+ adc_dev->dev_comp = device_get_match_data(&pdev->dev);
+
mutex_init(&adc_dev->lock);
mt6577_auxadc_mod_reg(adc_dev->reg_base + MT6577_AUXADC_MISC,
* struct stm32_adc_common_regs - stm32 common registers
* @csr: common status register offset
* @ccr: common control register offset
- * @eoc1_msk: adc1 end of conversion flag in @csr
- * @eoc2_msk: adc2 end of conversion flag in @csr
- * @eoc3_msk: adc3 end of conversion flag in @csr
+ * @eoc_msk: array of eoc (end of conversion flag) masks in csr for adc1..n
+ * @ovr_msk: array of ovr (overrun flag) masks in csr for adc1..n
* @ier: interrupt enable register offset for each adc
* @eocie_msk: end of conversion interrupt enable mask in @ier
*/
struct stm32_adc_common_regs {
u32 csr;
u32 ccr;
- u32 eoc1_msk;
- u32 eoc2_msk;
- u32 eoc3_msk;
+ u32 eoc_msk[STM32_ADC_MAX_ADCS];
+ u32 ovr_msk[STM32_ADC_MAX_ADCS];
u32 ier;
u32 eocie_msk;
};
static const struct stm32_adc_common_regs stm32f4_adc_common_regs = {
.csr = STM32F4_ADC_CSR,
.ccr = STM32F4_ADC_CCR,
- .eoc1_msk = STM32F4_EOC1 | STM32F4_OVR1,
- .eoc2_msk = STM32F4_EOC2 | STM32F4_OVR2,
- .eoc3_msk = STM32F4_EOC3 | STM32F4_OVR3,
+ .eoc_msk = { STM32F4_EOC1, STM32F4_EOC2, STM32F4_EOC3},
+ .ovr_msk = { STM32F4_OVR1, STM32F4_OVR2, STM32F4_OVR3},
.ier = STM32F4_ADC_CR1,
- .eocie_msk = STM32F4_EOCIE | STM32F4_OVRIE,
+ .eocie_msk = STM32F4_EOCIE,
};
/* STM32H7 common registers definitions */
static const struct stm32_adc_common_regs stm32h7_adc_common_regs = {
.csr = STM32H7_ADC_CSR,
.ccr = STM32H7_ADC_CCR,
- .eoc1_msk = STM32H7_EOC_MST | STM32H7_OVR_MST,
- .eoc2_msk = STM32H7_EOC_SLV | STM32H7_OVR_SLV,
+ .eoc_msk = { STM32H7_EOC_MST, STM32H7_EOC_SLV},
+ .ovr_msk = { STM32H7_OVR_MST, STM32H7_OVR_SLV},
.ier = STM32H7_ADC_IER,
- .eocie_msk = STM32H7_EOCIE | STM32H7_OVRIE,
+ .eocie_msk = STM32H7_EOCIE,
};
static const unsigned int stm32_adc_offset[STM32_ADC_MAX_ADCS] = {
{
struct stm32_adc_priv *priv = irq_desc_get_handler_data(desc);
struct irq_chip *chip = irq_desc_get_chip(desc);
+ int i;
u32 status;
chained_irq_enter(chip, desc);
* before invoking the interrupt handler (e.g. call ISR only for
* IRQ-enabled ADCs).
*/
- if (status & priv->cfg->regs->eoc1_msk &&
- stm32_adc_eoc_enabled(priv, 0))
- generic_handle_irq(irq_find_mapping(priv->domain, 0));
-
- if (status & priv->cfg->regs->eoc2_msk &&
- stm32_adc_eoc_enabled(priv, 1))
- generic_handle_irq(irq_find_mapping(priv->domain, 1));
-
- if (status & priv->cfg->regs->eoc3_msk &&
- stm32_adc_eoc_enabled(priv, 2))
- generic_handle_irq(irq_find_mapping(priv->domain, 2));
+ for (i = 0; i < priv->cfg->num_irqs; i++) {
+ if ((status & priv->cfg->regs->eoc_msk[i] &&
+ stm32_adc_eoc_enabled(priv, i)) ||
+ (status & priv->cfg->regs->ovr_msk[i]))
+ generic_handle_irq(irq_find_mapping(priv->domain, i));
+ }
chained_irq_exit(chip, desc);
};
* @start_conv: routine to start conversions
* @stop_conv: routine to stop conversions
* @unprepare: optional unprepare routine (disable, power-down)
+ * @irq_clear: routine to clear irqs
* @smp_cycles: programmable sampling time (ADC clock cycles)
*/
struct stm32_adc_cfg {
void (*start_conv)(struct iio_dev *, bool dma);
void (*stop_conv)(struct iio_dev *);
void (*unprepare)(struct iio_dev *);
+ void (*irq_clear)(struct iio_dev *indio_dev, u32 msk);
const unsigned int *smp_cycles;
};
STM32F4_ADON | STM32F4_DMA | STM32F4_DDS);
}
+static void stm32f4_adc_irq_clear(struct iio_dev *indio_dev, u32 msk)
+{
+ struct stm32_adc *adc = iio_priv(indio_dev);
+
+ stm32_adc_clr_bits(adc, adc->cfg->regs->isr_eoc.reg, msk);
+}
+
static void stm32h7_adc_start_conv(struct iio_dev *indio_dev, bool dma)
{
struct stm32_adc *adc = iio_priv(indio_dev);
stm32_adc_clr_bits(adc, STM32H7_ADC_CFGR, STM32H7_DMNGT_MASK);
}
+static void stm32h7_adc_irq_clear(struct iio_dev *indio_dev, u32 msk)
+{
+ struct stm32_adc *adc = iio_priv(indio_dev);
+ /* On STM32H7 IRQs are cleared by writing 1 into ISR register */
+ stm32_adc_set_bits(adc, adc->cfg->regs->isr_eoc.reg, msk);
+}
+
static int stm32h7_adc_exit_pwr_down(struct iio_dev *indio_dev)
{
struct stm32_adc *adc = iio_priv(indio_dev);
}
}
+static void stm32_adc_irq_clear(struct iio_dev *indio_dev, u32 msk)
+{
+ struct stm32_adc *adc = iio_priv(indio_dev);
+
+ adc->cfg->irq_clear(indio_dev, msk);
+}
+
static irqreturn_t stm32_adc_threaded_isr(int irq, void *data)
{
struct iio_dev *indio_dev = data;
struct stm32_adc *adc = iio_priv(indio_dev);
const struct stm32_adc_regspec *regs = adc->cfg->regs;
u32 status = stm32_adc_readl(adc, regs->isr_eoc.reg);
+ u32 mask = stm32_adc_readl(adc, regs->ier_eoc.reg);
- if (status & regs->isr_ovr.mask)
+ /* Check ovr status right now, as ovr mask should be already disabled */
+ if (status & regs->isr_ovr.mask) {
+ /*
+ * Clear ovr bit to avoid subsequent calls to IRQ handler.
+ * This requires to stop ADC first. OVR bit state in ISR,
+ * is propaged to CSR register by hardware.
+ */
+ adc->cfg->stop_conv(indio_dev);
+ stm32_adc_irq_clear(indio_dev, regs->isr_ovr.mask);
dev_err(&indio_dev->dev, "Overrun, stopping: restart needed\n");
+ return IRQ_HANDLED;
+ }
- return IRQ_HANDLED;
+ if (!(status & mask))
+ dev_err_ratelimited(&indio_dev->dev,
+ "Unexpected IRQ: IER=0x%08x, ISR=0x%08x\n",
+ mask, status);
+
+ return IRQ_NONE;
}
static irqreturn_t stm32_adc_isr(int irq, void *data)
struct stm32_adc *adc = iio_priv(indio_dev);
const struct stm32_adc_regspec *regs = adc->cfg->regs;
u32 status = stm32_adc_readl(adc, regs->isr_eoc.reg);
+ u32 mask = stm32_adc_readl(adc, regs->ier_eoc.reg);
+
+ if (!(status & mask))
+ return IRQ_WAKE_THREAD;
if (status & regs->isr_ovr.mask) {
/*
.start_conv = stm32f4_adc_start_conv,
.stop_conv = stm32f4_adc_stop_conv,
.smp_cycles = stm32f4_adc_smp_cycles,
+ .irq_clear = stm32f4_adc_irq_clear,
};
static const struct stm32_adc_cfg stm32h7_adc_cfg = {
.prepare = stm32h7_adc_prepare,
.unprepare = stm32h7_adc_unprepare,
.smp_cycles = stm32h7_adc_smp_cycles,
+ .irq_clear = stm32h7_adc_irq_clear,
};
static const struct stm32_adc_cfg stm32mp1_adc_cfg = {
.prepare = stm32h7_adc_prepare,
.unprepare = stm32h7_adc_unprepare,
.smp_cycles = stm32h7_adc_smp_cycles,
+ .irq_clear = stm32h7_adc_irq_clear,
};
static const struct of_device_id stm32_adc_of_match[] = {
struct cros_ec_sensorhub *sensor_hub = dev_get_drvdata(dev->parent);
struct cros_ec_dev *ec = sensor_hub->ec;
struct cros_ec_sensor_platform *sensor_platform = dev_get_platdata(dev);
- u32 ver_mask;
+ u32 ver_mask, temp;
int frequencies[ARRAY_SIZE(state->frequencies) / 2] = { 0 };
int ret, i;
&frequencies[2],
&state->fifo_max_event_count);
} else {
- frequencies[1] = state->resp->info_3.min_frequency;
- frequencies[2] = state->resp->info_3.max_frequency;
- state->fifo_max_event_count =
- state->resp->info_3.fifo_max_event_count;
+ if (state->resp->info_3.max_frequency == 0) {
+ get_default_min_max_freq(state->resp->info.type,
+ &frequencies[1],
+ &frequencies[2],
+ &temp);
+ } else {
+ frequencies[1] = state->resp->info_3.min_frequency;
+ frequencies[2] = state->resp->info_3.max_frequency;
+ }
+ state->fifo_max_event_count = state->resp->info_3.fifo_max_event_count;
}
for (i = 0; i < ARRAY_SIZE(frequencies); i++) {
state->frequencies[2 * i] = frequencies[i] / 1000;
static void st_lsm6dsx_shub_wait_complete(struct st_lsm6dsx_hw *hw)
{
struct st_lsm6dsx_sensor *sensor;
- u32 odr;
+ u32 odr, timeout;
sensor = iio_priv(hw->iio_devs[ST_LSM6DSX_ID_ACC]);
odr = (hw->enable_mask & BIT(ST_LSM6DSX_ID_ACC)) ? sensor->odr : 12500;
- msleep((2000000U / odr) + 1);
+ /* set 10ms as minimum timeout for i2c slave configuration */
+ timeout = max_t(u32, 2000000U / odr + 1, 10);
+ msleep(timeout);
}
/*
config VCNL4035
tristate "VCNL4035 combined ALS and proximity sensor"
+ select IIO_BUFFER
select IIO_TRIGGERED_BUFFER
select REGMAP_I2C
depends on I2C
This allows the user to config the default GID type that the CM
uses for each device, when initiaing new connections.
+config INFINIBAND_VIRT_DMA
+ def_bool !HIGHMEM
+
if INFINIBAND_USER_ACCESS || !INFINIBAND_USER_ACCESS
source "drivers/infiniband/hw/mthca/Kconfig"
source "drivers/infiniband/hw/qib/Kconfig"
atomic_set(&cm_id_priv->work_count, -1);
refcount_set(&cm_id_priv->refcount, 1);
- ret = xa_alloc_cyclic_irq(&cm.local_id_table, &id, NULL, xa_limit_32b,
- &cm.local_id_next, GFP_KERNEL);
+ ret = xa_alloc_cyclic(&cm.local_id_table, &id, NULL, xa_limit_32b,
+ &cm.local_id_next, GFP_KERNEL);
if (ret < 0)
goto error;
cm_id_priv->id.local_id = (__force __be32)id ^ cm.random_id_operand;
*/
static void cm_finalize_id(struct cm_id_private *cm_id_priv)
{
- xa_store_irq(&cm.local_id_table, cm_local_id(cm_id_priv->id.local_id),
- cm_id_priv, GFP_KERNEL);
+ xa_store(&cm.local_id_table, cm_local_id(cm_id_priv->id.local_id),
+ cm_id_priv, GFP_ATOMIC);
}
struct ib_cm_id *ib_create_cm_id(struct ib_device *device,
spin_unlock(&cm.lock);
spin_unlock_irq(&cm_id_priv->lock);
- xa_erase_irq(&cm.local_id_table, cm_local_id(cm_id->local_id));
+ xa_erase(&cm.local_id_table, cm_local_id(cm_id->local_id));
cm_deref_id(cm_id_priv);
wait_for_completion(&cm_id_priv->comp);
while ((work = cm_dequeue_work(cm_id_priv)) != NULL)
cm.remote_id_table = RB_ROOT;
cm.remote_qp_table = RB_ROOT;
cm.remote_sidr_table = RB_ROOT;
- xa_init_flags(&cm.local_id_table, XA_FLAGS_ALLOC | XA_FLAGS_LOCK_IRQ);
+ xa_init_flags(&cm.local_id_table, XA_FLAGS_ALLOC);
get_random_bytes(&cm.random_id_operand, sizeof cm.random_id_operand);
INIT_LIST_HEAD(&cm.timewait_list);
if (tb[RDMA_NLDEV_ATTR_DEV_NAME]) {
char name[IB_DEVICE_NAME_MAX] = {};
- nla_strlcpy(name, tb[RDMA_NLDEV_ATTR_DEV_NAME],
+ nla_strscpy(name, tb[RDMA_NLDEV_ATTR_DEV_NAME],
IB_DEVICE_NAME_MAX);
if (strlen(name) == 0) {
err = -EINVAL;
!tb[RDMA_NLDEV_ATTR_LINK_TYPE] || !tb[RDMA_NLDEV_ATTR_NDEV_NAME])
return -EINVAL;
- nla_strlcpy(ibdev_name, tb[RDMA_NLDEV_ATTR_DEV_NAME],
+ nla_strscpy(ibdev_name, tb[RDMA_NLDEV_ATTR_DEV_NAME],
sizeof(ibdev_name));
if (strchr(ibdev_name, '%') || strlen(ibdev_name) == 0)
return -EINVAL;
- nla_strlcpy(type, tb[RDMA_NLDEV_ATTR_LINK_TYPE], sizeof(type));
- nla_strlcpy(ndev_name, tb[RDMA_NLDEV_ATTR_NDEV_NAME],
+ nla_strscpy(type, tb[RDMA_NLDEV_ATTR_LINK_TYPE], sizeof(type));
+ nla_strscpy(ndev_name, tb[RDMA_NLDEV_ATTR_NDEV_NAME],
sizeof(ndev_name));
ndev = dev_get_by_name(sock_net(skb->sk), ndev_name);
if (err || !tb[RDMA_NLDEV_ATTR_CHARDEV_TYPE])
return -EINVAL;
- nla_strlcpy(client_name, tb[RDMA_NLDEV_ATTR_CHARDEV_TYPE],
+ nla_strscpy(client_name, tb[RDMA_NLDEV_ATTR_CHARDEV_TYPE],
sizeof(client_name));
if (tb[RDMA_NLDEV_ATTR_DEV_INDEX]) {
& CCE_REVISION_SW_MASK);
/* alloc netdev data */
- if (hfi1_netdev_alloc(dd))
+ ret = hfi1_netdev_alloc(dd);
+ if (ret)
goto bail_cleanup;
ret = set_up_context_variables(dd);
/*
+ * Copyright(c) 2020 Cornelis Networks, Inc.
* Copyright(c) 2015-2020 Intel Corporation.
*
* This file is provided under a dual BSD/GPLv2 license. When using or
spin_lock_init(&fd->tid_lock);
spin_lock_init(&fd->invalid_lock);
fd->rec_cpu_num = -1; /* no cpu affinity by default */
- fd->mm = current->mm;
- mmgrab(fd->mm);
fd->dd = dd;
fp->private_data = fd;
return 0;
deallocate_ctxt(uctxt);
done:
- mmdrop(fdata->mm);
if (atomic_dec_and_test(&dd->user_refcount))
complete(&dd->user_comp);
#ifndef _HFI1_KERNEL_H
#define _HFI1_KERNEL_H
/*
+ * Copyright(c) 2020 Cornelis Networks, Inc.
* Copyright(c) 2015-2020 Intel Corporation.
*
* This file is provided under a dual BSD/GPLv2 license. When using or
u32 invalid_tid_idx;
/* protect invalid_tids array and invalid_tid_idx */
spinlock_t invalid_lock;
- struct mm_struct *mm;
};
extern struct xarray hfi1_dev_table;
/*
+ * Copyright(c) 2020 Cornelis Networks, Inc.
* Copyright(c) 2016 - 2017 Intel Corporation.
*
* This file is provided under a dual BSD/GPLv2 license. When using or
#include <linux/rculist.h>
#include <linux/mmu_notifier.h>
#include <linux/interval_tree_generic.h>
+#include <linux/sched/mm.h>
#include "mmu_rb.h"
#include "trace.h"
-struct mmu_rb_handler {
- struct mmu_notifier mn;
- struct rb_root_cached root;
- void *ops_arg;
- spinlock_t lock; /* protect the RB tree */
- struct mmu_rb_ops *ops;
- struct mm_struct *mm;
- struct list_head lru_list;
- struct work_struct del_work;
- struct list_head del_list;
- struct workqueue_struct *wq;
-};
-
static unsigned long mmu_node_start(struct mmu_rb_node *);
static unsigned long mmu_node_last(struct mmu_rb_node *);
static int mmu_notifier_range_start(struct mmu_notifier *,
return PAGE_ALIGN(node->addr + node->len) - 1;
}
-int hfi1_mmu_rb_register(void *ops_arg, struct mm_struct *mm,
+int hfi1_mmu_rb_register(void *ops_arg,
struct mmu_rb_ops *ops,
struct workqueue_struct *wq,
struct mmu_rb_handler **handler)
{
- struct mmu_rb_handler *handlr;
+ struct mmu_rb_handler *h;
int ret;
- handlr = kmalloc(sizeof(*handlr), GFP_KERNEL);
- if (!handlr)
+ h = kmalloc(sizeof(*h), GFP_KERNEL);
+ if (!h)
return -ENOMEM;
- handlr->root = RB_ROOT_CACHED;
- handlr->ops = ops;
- handlr->ops_arg = ops_arg;
- INIT_HLIST_NODE(&handlr->mn.hlist);
- spin_lock_init(&handlr->lock);
- handlr->mn.ops = &mn_opts;
- handlr->mm = mm;
- INIT_WORK(&handlr->del_work, handle_remove);
- INIT_LIST_HEAD(&handlr->del_list);
- INIT_LIST_HEAD(&handlr->lru_list);
- handlr->wq = wq;
-
- ret = mmu_notifier_register(&handlr->mn, handlr->mm);
+ h->root = RB_ROOT_CACHED;
+ h->ops = ops;
+ h->ops_arg = ops_arg;
+ INIT_HLIST_NODE(&h->mn.hlist);
+ spin_lock_init(&h->lock);
+ h->mn.ops = &mn_opts;
+ INIT_WORK(&h->del_work, handle_remove);
+ INIT_LIST_HEAD(&h->del_list);
+ INIT_LIST_HEAD(&h->lru_list);
+ h->wq = wq;
+
+ ret = mmu_notifier_register(&h->mn, current->mm);
if (ret) {
- kfree(handlr);
+ kfree(h);
return ret;
}
- *handler = handlr;
+ *handler = h;
return 0;
}
struct list_head del_list;
/* Unregister first so we don't get any more notifications. */
- mmu_notifier_unregister(&handler->mn, handler->mm);
+ mmu_notifier_unregister(&handler->mn, handler->mn.mm);
/*
* Make sure the wq delete handler is finished running. It will not
int ret = 0;
trace_hfi1_mmu_rb_insert(mnode->addr, mnode->len);
+
+ if (current->mm != handler->mn.mm)
+ return -EPERM;
+
spin_lock_irqsave(&handler->lock, flags);
node = __mmu_rb_search(handler, mnode->addr, mnode->len);
if (node) {
__mmu_int_rb_remove(mnode, &handler->root);
list_del(&mnode->list); /* remove from LRU list */
}
+ mnode->handler = handler;
unlock:
spin_unlock_irqrestore(&handler->lock, flags);
return ret;
unsigned long flags;
bool ret = false;
+ if (current->mm != handler->mn.mm)
+ return ret;
+
spin_lock_irqsave(&handler->lock, flags);
node = __mmu_rb_search(handler, addr, len);
if (node) {
unsigned long flags;
bool stop = false;
+ if (current->mm != handler->mn.mm)
+ return;
+
INIT_LIST_HEAD(&del_list);
spin_lock_irqsave(&handler->lock, flags);
{
unsigned long flags;
+ if (current->mm != handler->mn.mm)
+ return;
+
/* Validity of handler and node pointers has been checked by caller. */
trace_hfi1_mmu_rb_remove(node->addr, node->len);
spin_lock_irqsave(&handler->lock, flags);
/*
+ * Copyright(c) 2020 Cornelis Networks, Inc.
* Copyright(c) 2016 Intel Corporation.
*
* This file is provided under a dual BSD/GPLv2 license. When using or
unsigned long len;
unsigned long __last;
struct rb_node node;
+ struct mmu_rb_handler *handler;
struct list_head list;
};
void *evict_arg, bool *stop);
};
-int hfi1_mmu_rb_register(void *ops_arg, struct mm_struct *mm,
+struct mmu_rb_handler {
+ struct mmu_notifier mn;
+ struct rb_root_cached root;
+ void *ops_arg;
+ spinlock_t lock; /* protect the RB tree */
+ struct mmu_rb_ops *ops;
+ struct list_head lru_list;
+ struct work_struct del_work;
+ struct list_head del_list;
+ struct workqueue_struct *wq;
+};
+
+int hfi1_mmu_rb_register(void *ops_arg,
struct mmu_rb_ops *ops,
struct workqueue_struct *wq,
struct mmu_rb_handler **handler);
/*
+ * Copyright(c) 2020 Cornelis Networks, Inc.
* Copyright(c) 2015-2018 Intel Corporation.
*
* This file is provided under a dual BSD/GPLv2 license. When using or
{
struct page **pages;
struct hfi1_devdata *dd = fd->uctxt->dd;
+ struct mm_struct *mm;
if (mapped) {
pci_unmap_single(dd->pcidev, node->dma_addr,
node->npages * PAGE_SIZE, PCI_DMA_FROMDEVICE);
pages = &node->pages[idx];
+ mm = mm_from_tid_node(node);
} else {
pages = &tidbuf->pages[idx];
+ mm = current->mm;
}
- hfi1_release_user_pages(fd->mm, pages, npages, mapped);
+ hfi1_release_user_pages(mm, pages, npages, mapped);
fd->tid_n_pinned -= npages;
}
* pages, accept the amount pinned so far and program only that.
* User space knows how to deal with partially programmed buffers.
*/
- if (!hfi1_can_pin_pages(dd, fd->mm, fd->tid_n_pinned, npages)) {
+ if (!hfi1_can_pin_pages(dd, current->mm, fd->tid_n_pinned, npages)) {
kfree(pages);
return -ENOMEM;
}
- pinned = hfi1_acquire_user_pages(fd->mm, vaddr, npages, true, pages);
+ pinned = hfi1_acquire_user_pages(current->mm, vaddr, npages, true, pages);
if (pinned <= 0) {
kfree(pages);
return pinned;
if (fd->use_mn) {
ret = mmu_interval_notifier_insert(
- &node->notifier, fd->mm,
+ &node->notifier, current->mm,
tbuf->vaddr + (pageidx * PAGE_SIZE), npages * PAGE_SIZE,
&tid_mn_ops);
if (ret)
#ifndef _HFI1_USER_EXP_RCV_H
#define _HFI1_USER_EXP_RCV_H
/*
+ * Copyright(c) 2020 - Cornelis Networks, Inc.
* Copyright(c) 2015 - 2017 Intel Corporation.
*
* This file is provided under a dual BSD/GPLv2 license. When using or
int hfi1_user_exp_rcv_invalid(struct hfi1_filedata *fd,
struct hfi1_tid_info *tinfo);
+static inline struct mm_struct *mm_from_tid_node(struct tid_rb_node *node)
+{
+ return node->notifier.mm;
+}
+
#endif /* _HFI1_USER_EXP_RCV_H */
/*
+ * Copyright(c) 2020 - Cornelis Networks, Inc.
* Copyright(c) 2015 - 2018 Intel Corporation.
*
* This file is provided under a dual BSD/GPLv2 license. When using or
atomic_set(&pq->n_reqs, 0);
init_waitqueue_head(&pq->wait);
atomic_set(&pq->n_locked, 0);
- pq->mm = fd->mm;
iowait_init(&pq->busy, 0, NULL, NULL, defer_packet_queue,
activate_packet_queue, NULL, NULL);
cq->nentries = hfi1_sdma_comp_ring_size;
- ret = hfi1_mmu_rb_register(pq, pq->mm, &sdma_rb_ops, dd->pport->hfi1_wq,
+ ret = hfi1_mmu_rb_register(pq, &sdma_rb_ops, dd->pport->hfi1_wq,
&pq->handler);
if (ret) {
dd_dev_err(dd, "Failed to register with MMU %d", ret);
npages -= node->npages;
retry:
- if (!hfi1_can_pin_pages(pq->dd, pq->mm,
+ if (!hfi1_can_pin_pages(pq->dd, current->mm,
atomic_read(&pq->n_locked), npages)) {
cleared = sdma_cache_evict(pq, npages);
if (cleared >= npages)
goto retry;
}
- pinned = hfi1_acquire_user_pages(pq->mm,
+ pinned = hfi1_acquire_user_pages(current->mm,
((unsigned long)iovec->iov.iov_base +
(node->npages * PAGE_SIZE)), npages, 0,
pages + node->npages);
return pinned;
}
if (pinned != npages) {
- unpin_vector_pages(pq->mm, pages, node->npages, pinned);
+ unpin_vector_pages(current->mm, pages, node->npages, pinned);
return -EFAULT;
}
kfree(node->pages);
static void unpin_sdma_pages(struct sdma_mmu_node *node)
{
if (node->npages) {
- unpin_vector_pages(node->pq->mm, node->pages, 0, node->npages);
+ unpin_vector_pages(mm_from_sdma_node(node), node->pages, 0,
+ node->npages);
atomic_sub(node->npages, &node->pq->n_locked);
}
}
#ifndef _HFI1_USER_SDMA_H
#define _HFI1_USER_SDMA_H
/*
+ * Copyright(c) 2020 - Cornelis Networks, Inc.
* Copyright(c) 2015 - 2018 Intel Corporation.
*
* This file is provided under a dual BSD/GPLv2 license. When using or
unsigned long unpinned;
struct mmu_rb_handler *handler;
atomic_t n_locked;
- struct mm_struct *mm;
};
struct hfi1_user_sdma_comp_q {
struct iovec *iovec, unsigned long dim,
unsigned long *count);
+static inline struct mm_struct *mm_from_sdma_node(struct sdma_mmu_node *node)
+{
+ return node->rb.handler->mn.mm;
+}
+
#endif /* _HFI1_USER_SDMA_H */
roce_set_bit(mpt_entry->byte_8_mw_cnt_en, V2_MPT_BYTE_8_R_INV_EN_S, 1);
roce_set_bit(mpt_entry->byte_8_mw_cnt_en, V2_MPT_BYTE_8_L_INV_EN_S, 1);
+ roce_set_bit(mpt_entry->byte_8_mw_cnt_en, V2_MPT_BYTE_8_LW_EN_S, 1);
roce_set_bit(mpt_entry->byte_12_mw_pa, V2_MPT_BYTE_12_PA_S, 0);
roce_set_bit(mpt_entry->byte_12_mw_pa, V2_MPT_BYTE_12_MR_MW_S, 1);
V2_QPC_BYTE_28_AT_M,
V2_QPC_BYTE_28_AT_S);
qp_attr->retry_cnt = roce_get_field(context.byte_212_lsn,
- V2_QPC_BYTE_212_RETRY_CNT_M,
- V2_QPC_BYTE_212_RETRY_CNT_S);
+ V2_QPC_BYTE_212_RETRY_NUM_INIT_M,
+ V2_QPC_BYTE_212_RETRY_NUM_INIT_S);
qp_attr->rnr_retry = roce_get_field(context.byte_244_rnr_rxack,
- V2_QPC_BYTE_244_RNR_CNT_M,
- V2_QPC_BYTE_244_RNR_CNT_S);
+ V2_QPC_BYTE_244_RNR_NUM_INIT_M,
+ V2_QPC_BYTE_244_RNR_NUM_INIT_S);
done:
qp_attr->cur_qp_state = qp_attr->qp_state;
__le32 rsv_uars_rsv_qps;
};
#define V2_QUERY_PF_CAPS_D_NUM_SRQS_S 0
-#define V2_QUERY_PF_CAPS_D_NUM_SRQS_M GENMASK(20, 0)
+#define V2_QUERY_PF_CAPS_D_NUM_SRQS_M GENMASK(19, 0)
#define V2_QUERY_PF_CAPS_D_RQWQE_HOP_NUM_S 20
#define V2_QUERY_PF_CAPS_D_RQWQE_HOP_NUM_M GENMASK(21, 20)
#define DRV_VERSION __stringify(DRV_VERSION_MAJOR) "." \
__stringify(DRV_VERSION_MINOR) "." __stringify(DRV_VERSION_BUILD)
-static int push_mode;
-module_param(push_mode, int, 0644);
-MODULE_PARM_DESC(push_mode, "Low latency mode: 0=disabled (default), 1=enabled)");
-
static int debug;
module_param(debug, int, 0644);
MODULE_PARM_DESC(debug, "debug flags: 0=disabled (default), 0x7fffffff=all");
if (status)
goto exit;
iwdev->obj_next = iwdev->obj_mem;
- iwdev->push_mode = push_mode;
init_waitqueue_head(&iwdev->vchnl_waitq);
init_waitqueue_head(&dev->vf_reqs);
*/
static int i40iw_mmap(struct ib_ucontext *context, struct vm_area_struct *vma)
{
- struct i40iw_ucontext *ucontext;
- u64 db_addr_offset, push_offset, pfn;
-
- ucontext = to_ucontext(context);
- if (ucontext->iwdev->sc_dev.is_pf) {
- db_addr_offset = I40IW_DB_ADDR_OFFSET;
- push_offset = I40IW_PUSH_OFFSET;
- if (vma->vm_pgoff)
- vma->vm_pgoff += I40IW_PF_FIRST_PUSH_PAGE_INDEX - 1;
- } else {
- db_addr_offset = I40IW_VF_DB_ADDR_OFFSET;
- push_offset = I40IW_VF_PUSH_OFFSET;
- if (vma->vm_pgoff)
- vma->vm_pgoff += I40IW_VF_FIRST_PUSH_PAGE_INDEX - 1;
- }
+ struct i40iw_ucontext *ucontext = to_ucontext(context);
+ u64 dbaddr;
- vma->vm_pgoff += db_addr_offset >> PAGE_SHIFT;
-
- if (vma->vm_pgoff == (db_addr_offset >> PAGE_SHIFT)) {
- vma->vm_page_prot = pgprot_noncached(vma->vm_page_prot);
- } else {
- if ((vma->vm_pgoff - (push_offset >> PAGE_SHIFT)) % 2)
- vma->vm_page_prot = pgprot_noncached(vma->vm_page_prot);
- else
- vma->vm_page_prot = pgprot_writecombine(vma->vm_page_prot);
- }
+ if (vma->vm_pgoff || vma->vm_end - vma->vm_start != PAGE_SIZE)
+ return -EINVAL;
- pfn = vma->vm_pgoff +
- (pci_resource_start(ucontext->iwdev->ldev->pcidev, 0) >>
- PAGE_SHIFT);
+ dbaddr = I40IW_DB_ADDR_OFFSET + pci_resource_start(ucontext->iwdev->ldev->pcidev, 0);
- return rdma_user_mmap_io(context, vma, pfn, PAGE_SIZE,
- vma->vm_page_prot, NULL);
+ return rdma_user_mmap_io(context, vma, dbaddr >> PAGE_SHIFT, PAGE_SIZE,
+ pgprot_noncached(vma->vm_page_prot), NULL);
}
/**
}
mailbox = mthca_alloc_mailbox(dev, GFP_KERNEL);
- if (IS_ERR(mailbox))
+ if (IS_ERR(mailbox)) {
+ err = PTR_ERR(mailbox);
goto err_out_arm;
+ }
cq_context = mailbox->buf;
}
spin_lock_irq(&dev->cq_table.lock);
- if (mthca_array_set(&dev->cq_table.cq,
- cq->cqn & (dev->limits.num_cqs - 1),
- cq)) {
+ err = mthca_array_set(&dev->cq_table.cq,
+ cq->cqn & (dev->limits.num_cqs - 1), cq);
+ if (err) {
spin_unlock_irq(&dev->cq_table.lock);
goto err_out_free_mr;
}
}
ret = ib_device_set_netdev(&dev->ib_dev, dev->netdev, 1);
if (ret)
- return ret;
+ goto err_srq_free;
spin_lock_init(&dev->srq_tbl_lock);
rdma_set_device_sysfs_group(&dev->ib_dev, &pvrdma_attr_group);
# SPDX-License-Identifier: GPL-2.0-only
config INFINIBAND_RDMAVT
tristate "RDMA verbs transport library"
- depends on X86_64 && ARCH_DMA_ADDR_T_64BIT
+ depends on INFINIBAND_VIRT_DMA
+ depends on X86_64
depends on PCI
select DMA_VIRT_OPS
help
config RDMA_RXE
tristate "Software RDMA over Ethernet (RoCE) driver"
depends on INET && PCI && INFINIBAND
- depends on !64BIT || ARCH_DMA_ADDR_T_64BIT
+ depends on INFINIBAND_VIRT_DMA
select NET_UDP_TUNNEL
select CRYPTO_CRC32
select DMA_VIRT_OPS
config RDMA_SIW
tristate "Software RDMA over TCP/IP (iWARP) driver"
depends on INET && INFINIBAND && LIBCRC32C
+ depends on INFINIBAND_VIRT_DMA
select DMA_VIRT_OPS
help
This driver implements the iWARP RDMA transport over
switch (data) {
case SUNKBD_RET_RESET:
- schedule_work(&sunkbd->tq);
+ if (sunkbd->enabled)
+ schedule_work(&sunkbd->tq);
sunkbd->reset = -1;
break;
}
/*
- * sunkbd_reinit() sets leds and beeps to a state the computer remembers they
- * were in.
+ * sunkbd_set_leds_beeps() sets leds and beeps to a state the computer remembers
+ * they were in.
*/
-static void sunkbd_reinit(struct work_struct *work)
+static void sunkbd_set_leds_beeps(struct sunkbd *sunkbd)
{
- struct sunkbd *sunkbd = container_of(work, struct sunkbd, tq);
-
- wait_event_interruptible_timeout(sunkbd->wait, sunkbd->reset >= 0, HZ);
-
serio_write(sunkbd->serio, SUNKBD_CMD_SETLED);
serio_write(sunkbd->serio,
(!!test_bit(LED_CAPSL, sunkbd->dev->led) << 3) |
SUNKBD_CMD_BELLOFF - !!test_bit(SND_BELL, sunkbd->dev->snd));
}
+
+/*
+ * sunkbd_reinit() wait for the keyboard reset to complete and restores state
+ * of leds and beeps.
+ */
+
+static void sunkbd_reinit(struct work_struct *work)
+{
+ struct sunkbd *sunkbd = container_of(work, struct sunkbd, tq);
+
+ /*
+ * It is OK that we check sunkbd->enabled without pausing serio,
+ * as we only want to catch true->false transition that will
+ * happen once and we will be woken up for it.
+ */
+ wait_event_interruptible_timeout(sunkbd->wait,
+ sunkbd->reset >= 0 || !sunkbd->enabled,
+ HZ);
+
+ if (sunkbd->reset >= 0 && sunkbd->enabled)
+ sunkbd_set_leds_beeps(sunkbd);
+}
+
static void sunkbd_enable(struct sunkbd *sunkbd, bool enable)
{
serio_pause_rx(sunkbd->serio);
sunkbd->enabled = enable;
serio_continue_rx(sunkbd->serio);
+
+ if (!enable) {
+ wake_up_interruptible(&sunkbd->wait);
+ cancel_work_sync(&sunkbd->tq);
+ }
}
/*
struct input_dev *input_dev;
const struct adxl34x_platform_data *pdata;
int err, range, i;
- unsigned char revid;
+ int revid;
if (!irq) {
dev_err(dev, "no IRQ?\n");
int (*iap_reset)(struct i2c_client *client);
int (*prepare_fw_update)(struct i2c_client *client, u16 ic_type,
- u8 iap_version);
+ u8 iap_version, u16 fw_page_size);
int (*write_fw_block)(struct i2c_client *client, u16 fw_page_size,
const u8 *page, u16 checksum, int idx);
int (*finish_fw_update)(struct i2c_client *client,
u16 sw_checksum = 0, fw_checksum = 0;
error = data->ops->prepare_fw_update(client, data->ic_type,
- data->iap_version);
+ data->iap_version,
+ data->fw_page_size);
if (error)
return error;
return 0;
}
-static int elan_read_write_iap_type(struct i2c_client *client)
+static int elan_read_write_iap_type(struct i2c_client *client, u16 fw_page_size)
{
int error;
u16 constant;
do {
error = elan_i2c_write_cmd(client, ETP_I2C_IAP_TYPE_CMD,
- ETP_I2C_IAP_TYPE_REG);
+ fw_page_size / 2);
if (error) {
dev_err(&client->dev,
"cannot write iap type: %d\n", error);
constant = le16_to_cpup((__le16 *)val);
dev_dbg(&client->dev, "iap type reg: 0x%04x\n", constant);
- if (constant == ETP_I2C_IAP_TYPE_REG)
+ if (constant == fw_page_size / 2)
return 0;
} while (--retry > 0);
}
static int elan_i2c_prepare_fw_update(struct i2c_client *client, u16 ic_type,
- u8 iap_version)
+ u8 iap_version, u16 fw_page_size)
{
struct device *dev = &client->dev;
int error;
}
if (ic_type >= 0x0D && iap_version >= 1) {
- error = elan_read_write_iap_type(client);
+ error = elan_read_write_iap_type(client, fw_page_size);
if (error)
return error;
}
}
static int elan_smbus_prepare_fw_update(struct i2c_client *client, u16 ic_type,
- u8 iap_version)
+ u8 iap_version, u16 fw_page_size)
{
struct device *dev = &client->dev;
int len;
MODULE_PARM_DESC(unmask_kbd_data, "Unconditional enable (may reveal sensitive data) of normally sanitize-filtered kbd data traffic debug log [pre-condition: i8042.debug=1 enabled]");
#endif
+static bool i8042_present;
static bool i8042_bypass_aux_irq_test;
static char i8042_kbd_firmware_id[128];
static char i8042_aux_firmware_id[128];
unsigned long flags;
int retval;
+ if (!i8042_present)
+ return -1;
+
spin_lock_irqsave(&i8042_lock, flags);
retval = __i8042_command(param, command);
spin_unlock_irqrestore(&i8042_lock, flags);
err = i8042_platform_init();
if (err)
- return err;
+ return (err == -ENODEV) ? 0 : err;
err = i8042_controller_check();
if (err)
goto err_platform_exit;
+ /* Set this before creating the dev to allow i8042_command to work right away */
+ i8042_present = true;
+
pdev = platform_create_bundle(&i8042_driver, i8042_probe, NULL, 0, NULL, 0);
if (IS_ERR(pdev)) {
err = PTR_ERR(pdev);
static void __exit i8042_exit(void)
{
+ if (!i8042_present)
+ return;
+
platform_device_unregister(i8042_platform_device);
platform_driver_unregister(&i8042_driver);
i8042_platform_exit();
config TOUCHSCREEN_ADC
tristate "Generic ADC based resistive touchscreen"
depends on IIO
+ select IIO_BUFFER
select IIO_BUFFER_CB
help
Say Y here if you want to use the generic ADC
count++;
count += of_count_icc_providers(child);
}
- of_node_put(np);
return count;
}
DEFINE_QNODE(mas_pcnoc_sdcc_2, MSM8916_MASTER_SDCC_2, 8, -1, -1, MSM8916_PNOC_INT_1);
DEFINE_QNODE(mas_qdss_bam, MSM8916_MASTER_QDSS_BAM, 8, -1, -1, MSM8916_SNOC_QDSS_INT);
DEFINE_QNODE(mas_qdss_etr, MSM8916_MASTER_QDSS_ETR, 8, -1, -1, MSM8916_SNOC_QDSS_INT);
-DEFINE_QNODE(mas_snoc_cfg, MSM8916_MASTER_SNOC_CFG, 4, 20, -1, MSM8916_SNOC_QDSS_INT);
+DEFINE_QNODE(mas_snoc_cfg, MSM8916_MASTER_SNOC_CFG, 4, -1, -1, MSM8916_SNOC_QDSS_INT);
DEFINE_QNODE(mas_spdm, MSM8916_MASTER_SPDM, 4, -1, -1, MSM8916_PNOC_MAS_0);
DEFINE_QNODE(mas_tcu0, MSM8916_MASTER_TCU0, 8, -1, -1, MSM8916_SLAVE_EBI_CH0, MSM8916_BIMC_SNOC_MAS, MSM8916_SLAVE_AMPSS_L2);
DEFINE_QNODE(mas_tcu1, MSM8916_MASTER_TCU1, 8, -1, -1, MSM8916_SLAVE_EBI_CH0, MSM8916_BIMC_SNOC_MAS, MSM8916_SLAVE_AMPSS_L2);
DEFINE_QNODE(pcnoc_snoc_slv, MSM8916_PNOC_SNOC_SLV, 8, -1, 45, MSM8916_SNOC_INT_0, MSM8916_SNOC_INT_BIMC, MSM8916_SNOC_INT_1);
DEFINE_QNODE(qdss_int, MSM8916_SNOC_QDSS_INT, 8, -1, -1, MSM8916_SNOC_INT_0, MSM8916_SNOC_INT_BIMC);
DEFINE_QNODE(slv_apps_l2, MSM8916_SLAVE_AMPSS_L2, 8, -1, -1, 0);
-DEFINE_QNODE(slv_apss, MSM8916_SLAVE_APSS, 4, -1, 20, 0);
+DEFINE_QNODE(slv_apss, MSM8916_SLAVE_APSS, 4, -1, -1, 0);
DEFINE_QNODE(slv_audio, MSM8916_SLAVE_LPASS, 4, -1, -1, 0);
DEFINE_QNODE(slv_bimc_cfg, MSM8916_SLAVE_BIMC_CFG, 4, -1, -1, 0);
DEFINE_QNODE(slv_blsp_1, MSM8916_SLAVE_BLSP_1, 4, -1, -1, 0);
DEFINE_QNODE(slv_boot_rom, MSM8916_SLAVE_BOOT_ROM, 4, -1, -1, 0);
DEFINE_QNODE(slv_camera_cfg, MSM8916_SLAVE_CAMERA_CFG, 4, -1, -1, 0);
-DEFINE_QNODE(slv_cats_0, MSM8916_SLAVE_CATS_128, 16, -1, 106, 0);
-DEFINE_QNODE(slv_cats_1, MSM8916_SLAVE_OCMEM_64, 8, -1, 107, 0);
+DEFINE_QNODE(slv_cats_0, MSM8916_SLAVE_CATS_128, 16, -1, -1, 0);
+DEFINE_QNODE(slv_cats_1, MSM8916_SLAVE_OCMEM_64, 8, -1, -1, 0);
DEFINE_QNODE(slv_clk_ctl, MSM8916_SLAVE_CLK_CTL, 4, -1, -1, 0);
DEFINE_QNODE(slv_crypto_0_cfg, MSM8916_SLAVE_CRYPTO_0_CFG, 4, -1, -1, 0);
DEFINE_QNODE(slv_dehr_cfg, MSM8916_SLAVE_DEHR_CFG, 4, -1, -1, 0);
DEFINE_QNODE(slv_security, MSM8916_SLAVE_SECURITY, 4, -1, -1, 0);
DEFINE_QNODE(slv_snoc_cfg, MSM8916_SLAVE_SNOC_CFG, 4, -1, -1, 0);
DEFINE_QNODE(slv_spdm, MSM8916_SLAVE_SPDM, 4, -1, -1, 0);
-DEFINE_QNODE(slv_srvc_snoc, MSM8916_SLAVE_SRVC_SNOC, 8, -1, 29, 0);
+DEFINE_QNODE(slv_srvc_snoc, MSM8916_SLAVE_SRVC_SNOC, 8, -1, -1, 0);
DEFINE_QNODE(slv_tcsr, MSM8916_SLAVE_TCSR, 4, -1, -1, 0);
DEFINE_QNODE(slv_tlmm, MSM8916_SLAVE_TLMM, 4, -1, -1, 0);
DEFINE_QNODE(slv_usb_hs, MSM8916_SLAVE_USB_HS, 4, -1, -1, 0);
DEFINE_QNODE(snoc_bimc_1_mas, MSM8916_SNOC_BIMC_1_MAS, 16, -1, -1, MSM8916_SNOC_BIMC_1_SLV);
DEFINE_QNODE(snoc_bimc_1_slv, MSM8916_SNOC_BIMC_1_SLV, 8, -1, -1, MSM8916_SLAVE_EBI_CH0);
DEFINE_QNODE(snoc_int_0, MSM8916_SNOC_INT_0, 8, 99, 130, MSM8916_SLAVE_QDSS_STM, MSM8916_SLAVE_IMEM, MSM8916_SNOC_PNOC_MAS);
-DEFINE_QNODE(snoc_int_1, MSM8916_SNOC_INT_1, 8, 100, 131, MSM8916_SLAVE_APSS, MSM8916_SLAVE_CATS_128, MSM8916_SLAVE_OCMEM_64);
+DEFINE_QNODE(snoc_int_1, MSM8916_SNOC_INT_1, 8, -1, -1, MSM8916_SLAVE_APSS, MSM8916_SLAVE_CATS_128, MSM8916_SLAVE_OCMEM_64);
DEFINE_QNODE(snoc_int_bimc, MSM8916_SNOC_INT_BIMC, 8, 101, 132, MSM8916_SNOC_BIMC_0_MAS);
DEFINE_QNODE(snoc_pcnoc_mas, MSM8916_SNOC_PNOC_MAS, 8, -1, -1, MSM8916_SNOC_PNOC_SLV);
DEFINE_QNODE(snoc_pcnoc_slv, MSM8916_SNOC_PNOC_SLV, 8, -1, -1, MSM8916_PNOC_INT_0);
do_div(rate, src_qn->buswidth);
+ rate = min_t(u32, rate, INT_MAX);
+
if (src_qn->rate == rate)
return 0;
return 0;
}
+static int msm8974_get_bw(struct icc_node *node, u32 *avg, u32 *peak)
+{
+ *avg = 0;
+ *peak = 0;
+
+ return 0;
+}
+
static int msm8974_icc_probe(struct platform_device *pdev)
{
const struct msm8974_icc_desc *desc;
provider->aggregate = icc_std_aggregate;
provider->xlate = of_icc_xlate_onecell;
provider->data = data;
+ provider->get_bw = msm8974_get_bw;
ret = icc_provider_add(provider);
if (ret) {
.driver = {
.name = "qnoc-msm8974",
.of_match_table = msm8974_noc_of_match,
+ .sync_state = icc_sync_state,
},
};
module_platform_driver(msm8974_noc_driver);
}
DEFINE_QNODE(mas_apps_proc, QCS404_MASTER_AMPSS_M0, 8, 0, -1, QCS404_SLAVE_EBI_CH0, QCS404_BIMC_SNOC_SLV);
-DEFINE_QNODE(mas_oxili, QCS404_MASTER_GRAPHICS_3D, 8, 6, -1, QCS404_SLAVE_EBI_CH0, QCS404_BIMC_SNOC_SLV);
-DEFINE_QNODE(mas_mdp, QCS404_MASTER_MDP_PORT0, 8, 8, -1, QCS404_SLAVE_EBI_CH0, QCS404_BIMC_SNOC_SLV);
+DEFINE_QNODE(mas_oxili, QCS404_MASTER_GRAPHICS_3D, 8, -1, -1, QCS404_SLAVE_EBI_CH0, QCS404_BIMC_SNOC_SLV);
+DEFINE_QNODE(mas_mdp, QCS404_MASTER_MDP_PORT0, 8, -1, -1, QCS404_SLAVE_EBI_CH0, QCS404_BIMC_SNOC_SLV);
DEFINE_QNODE(mas_snoc_bimc_1, QCS404_SNOC_BIMC_1_MAS, 8, 76, -1, QCS404_SLAVE_EBI_CH0);
DEFINE_QNODE(mas_tcu_0, QCS404_MASTER_TCU_0, 8, -1, -1, QCS404_SLAVE_EBI_CH0, QCS404_BIMC_SNOC_SLV);
DEFINE_QNODE(mas_spdm, QCS404_MASTER_SPDM, 4, -1, -1, QCS404_PNOC_INT_3);
#include <asm/iommu_table.h>
#include <asm/io_apic.h>
#include <asm/irq_remapping.h>
+#include <asm/set_memory.h>
#include <linux/crash_dump.h>
free_pages((unsigned long)iommu->cmd_buf, get_order(CMD_BUFFER_SIZE));
}
+static void *__init iommu_alloc_4k_pages(struct amd_iommu *iommu,
+ gfp_t gfp, size_t size)
+{
+ int order = get_order(size);
+ void *buf = (void *)__get_free_pages(gfp, order);
+
+ if (buf &&
+ iommu_feature(iommu, FEATURE_SNP) &&
+ set_memory_4k((unsigned long)buf, (1 << order))) {
+ free_pages((unsigned long)buf, order);
+ buf = NULL;
+ }
+
+ return buf;
+}
+
/* allocates the memory where the IOMMU will log its events to */
static int __init alloc_event_buffer(struct amd_iommu *iommu)
{
- iommu->evt_buf = (void *)__get_free_pages(GFP_KERNEL | __GFP_ZERO,
- get_order(EVT_BUFFER_SIZE));
+ iommu->evt_buf = iommu_alloc_4k_pages(iommu, GFP_KERNEL | __GFP_ZERO,
+ EVT_BUFFER_SIZE);
return iommu->evt_buf ? 0 : -ENOMEM;
}
/* allocates the memory where the IOMMU will log its events to */
static int __init alloc_ppr_log(struct amd_iommu *iommu)
{
- iommu->ppr_log = (void *)__get_free_pages(GFP_KERNEL | __GFP_ZERO,
- get_order(PPR_LOG_SIZE));
+ iommu->ppr_log = iommu_alloc_4k_pages(iommu, GFP_KERNEL | __GFP_ZERO,
+ PPR_LOG_SIZE);
return iommu->ppr_log ? 0 : -ENOMEM;
}
static int __init alloc_cwwb_sem(struct amd_iommu *iommu)
{
- iommu->cmd_sem = (void *)get_zeroed_page(GFP_KERNEL);
+ iommu->cmd_sem = iommu_alloc_4k_pages(iommu, GFP_KERNEL | __GFP_ZERO, 1);
return iommu->cmd_sem ? 0 : -ENOMEM;
}
{
struct qcom_smmu *qsmmu;
+ /* Check to make sure qcom_scm has finished probing */
+ if (!qcom_scm_is_available())
+ return ERR_PTR(-EPROBE_DEFER);
+
qsmmu = devm_kzalloc(smmu->dev, sizeof(*qsmmu), GFP_KERNEL);
if (!qsmmu)
return ERR_PTR(-ENOMEM);
dmar_iommu_notify_scope_dev(info);
}
+static inline void vf_inherit_msi_domain(struct pci_dev *pdev)
+{
+ struct pci_dev *physfn = pci_physfn(pdev);
+
+ dev_set_msi_domain(&pdev->dev, dev_get_msi_domain(&physfn->dev));
+}
+
static int dmar_pci_bus_notifier(struct notifier_block *nb,
unsigned long action, void *data)
{
/* Only care about add/remove events for physical functions.
* For VFs we actually do the lookup based on the corresponding
* PF in device_to_iommu() anyway. */
- if (pdev->is_virtfn)
+ if (pdev->is_virtfn) {
+ /*
+ * Ensure that the VF device inherits the irq domain of the
+ * PF device. Ideally the device would inherit the domain
+ * from the bus, but DMAR can have multiple units per bus
+ * which makes this impossible. The VF 'bus' could inherit
+ * from the PF device, but that's yet another x86'sism to
+ * inflict on everybody else.
+ */
+ if (action == BUS_NOTIFY_ADD_DEVICE)
+ vf_inherit_msi_domain(pdev);
return NOTIFY_DONE;
+ }
+
if (action != BUS_NOTIFY_ADD_DEVICE &&
action != BUS_NOTIFY_REMOVED_DEVICE)
return NOTIFY_DONE;
warn_invalid_dmar(phys_addr, " returns all ones");
goto unmap;
}
- iommu->vccap = dmar_readq(iommu->reg + DMAR_VCCAP_REG);
+ if (ecap_vcs(iommu->ecap))
+ iommu->vccap = dmar_readq(iommu->reg + DMAR_VCCAP_REG);
/* the registers might be more than one page */
map_size = max_t(int, ecap_max_iotlb_offset(iommu->ecap),
* (used when kernel is launched w/ TXT)
*/
static int force_on = 0;
-int intel_iommu_tboot_noforce;
+static int intel_iommu_tboot_noforce;
static int no_platform_optin;
#define ROOT_ENTRY_NR (VTD_PAGE_SIZE/sizeof(struct root_entry))
if (ecap_prs(iommu->ecap))
intel_svm_finish_prq(iommu);
}
- if (ecap_vcs(iommu->ecap) && vccap_pasid(iommu->vccap))
+ if (vccap_pasid(iommu->vccap))
ioasid_unregister_allocator(&iommu->pasid_allocator);
#endif
* is active. All vIOMMU allocators will eventually be calling the same
* host allocator.
*/
- if (!ecap_vcs(iommu->ecap) || !vccap_pasid(iommu->vccap))
+ if (!vccap_pasid(iommu->vccap))
return;
pr_info("Register custom PASID allocator\n");
* Intel IOMMU is required for a TXT/tboot launch or platform
* opt in, so enforce that.
*/
- force_on = tboot_force_iommu() || platform_optin_force_iommu();
+ force_on = (!intel_iommu_tboot_noforce && tboot_force_iommu()) ||
+ platform_optin_force_iommu();
if (iommu_init_mempool()) {
if (force_on)
*/
iommu_alloc_default_domain(group, dev);
- if (group->default_domain)
+ if (group->default_domain) {
ret = __iommu_attach_device(group->default_domain, dev);
+ if (ret) {
+ iommu_group_put(group);
+ goto err_release;
+ }
+ }
iommu_create_device_direct_mappings(group, dev);
iommu_group_put(group);
- if (ret)
- goto err_release;
-
if (ops->probe_finalize)
ops->probe_finalize(dev);
#define ITS_FLAGS_CMDQ_NEEDS_FLUSHING (1ULL << 0)
#define ITS_FLAGS_WORKAROUND_CAVIUM_22375 (1ULL << 1)
#define ITS_FLAGS_WORKAROUND_CAVIUM_23144 (1ULL << 2)
-#define ITS_FLAGS_SAVE_SUSPEND_STATE (1ULL << 3)
#define RDIST_FLAGS_PROPBASE_NEEDS_FLUSHING (1 << 0)
#define RDIST_FLAGS_RD_TABLES_PREALLOCATED (1 << 1)
list_for_each_entry(its, &its_nodes, entry) {
void __iomem *base;
- if (!(its->flags & ITS_FLAGS_SAVE_SUSPEND_STATE))
- continue;
-
base = its->base;
its->ctlr_save = readl_relaxed(base + GITS_CTLR);
err = its_force_quiescent(base);
list_for_each_entry_continue_reverse(its, &its_nodes, entry) {
void __iomem *base;
- if (!(its->flags & ITS_FLAGS_SAVE_SUSPEND_STATE))
- continue;
-
base = its->base;
writel_relaxed(its->ctlr_save, base + GITS_CTLR);
}
void __iomem *base;
int i;
- if (!(its->flags & ITS_FLAGS_SAVE_SUSPEND_STATE))
- continue;
-
base = its->base;
/*
* don't restore it since writing to CBASER or BASER<n>
* registers is undefined according to the GIC v3 ITS
* Specification.
+ *
+ * Firmware resuming with the ITS enabled is terminally broken.
*/
+ WARN_ON(readl_relaxed(base + GITS_CTLR) & GITS_CTLR_ENABLE);
ret = its_force_quiescent(base);
if (ret) {
pr_err("ITS@%pa: failed to quiesce on resume: %d\n",
ctlr |= GITS_CTLR_ImDe;
writel_relaxed(ctlr, its->base + GITS_CTLR);
- if (GITS_TYPER_HCC(typer))
- its->flags |= ITS_FLAGS_SAVE_SUSPEND_STATE;
-
err = its_init_domain(handle, its);
if (err)
goto out_free_tables;
if (fwspec->param_count != 2)
return -EINVAL;
*hwirq = fwspec->param[0];
- *type = fwspec->param[2] & IRQ_TYPE_SENSE_MASK;
+ *type = fwspec->param[1] & IRQ_TYPE_SENSE_MASK;
}
return 0;
}
#include <linux/compiler.h>
#include <linux/module.h>
+#include <linux/ethtool.h>
#include <linux/errno.h>
#include <linux/kernel.h>
#include <linux/major.h>
* the project's page is at https://linuxtv.org
*/
+#include <linux/ethtool.h>
#include <linux/types.h>
#include <linux/kernel.h>
#include <linux/string.h>
depends on MTK_IOMMU || COMPILE_TEST
depends on VIDEO_DEV && VIDEO_V4L2
depends on ARCH_MEDIATEK || COMPILE_TEST
+ depends on VIDEO_MEDIATEK_VPU || MTK_SCP
+ # The two following lines ensure we have the same state ("m" or "y") as
+ # our dependencies, to avoid missing symbols during link.
+ depends on VIDEO_MEDIATEK_VPU || !VIDEO_MEDIATEK_VPU
+ depends on MTK_SCP || !MTK_SCP
select VIDEOBUF2_DMA_CONTIG
select V4L2_MEM2MEM_DEV
- select VIDEO_MEDIATEK_VPU
- select MTK_SCP
+ select VIDEO_MEDIATEK_VCODEC_VPU if VIDEO_MEDIATEK_VPU
+ select VIDEO_MEDIATEK_VCODEC_SCP if MTK_SCP
help
- Mediatek video codec driver provides HW capability to
- encode and decode in a range of video formats
- This driver rely on VPU driver to communicate with VPU.
+ Mediatek video codec driver provides HW capability to
+ encode and decode in a range of video formats on MT8173
+ and MT8183.
+
+ Note that support for MT8173 requires VIDEO_MEDIATEK_VPU to
+ also be selected. Support for MT8183 depends on MTK_SCP.
+
+ To compile this driver as modules, choose M here: the
+ modules will be called mtk-vcodec-dec and mtk-vcodec-enc.
+
+config VIDEO_MEDIATEK_VCODEC_VPU
+ bool
- To compile this driver as modules, choose M here: the
- modules will be called mtk-vcodec-dec and mtk-vcodec-enc.
+config VIDEO_MEDIATEK_VCODEC_SCP
+ bool
config VIDEO_MEM2MEM_DEINTERLACE
tristate "Deinterlace support"
* Suspend/resume support.
*/
+#ifdef CONFIG_PM
static int mmpcam_runtime_resume(struct device *dev)
{
struct mmp_camera *cam = dev_get_drvdata(dev);
return mccic_resume(&cam->mcam);
return 0;
}
+#endif
static const struct dev_pm_ops mmpcam_pm_ops = {
SET_RUNTIME_PM_OPS(mmpcam_runtime_suspend, mmpcam_runtime_resume, NULL)
mtk-vcodec-common-y := mtk_vcodec_intr.o \
mtk_vcodec_util.o \
- mtk_vcodec_fw.o
+ mtk_vcodec_fw.o \
+
+ifneq ($(CONFIG_VIDEO_MEDIATEK_VCODEC_VPU),)
+mtk-vcodec-common-y += mtk_vcodec_fw_vpu.o
+endif
+
+ifneq ($(CONFIG_VIDEO_MEDIATEK_VCODEC_SCP),)
+mtk-vcodec-common-y += mtk_vcodec_fw_scp.o
+endif
}
dma_set_max_seg_size(&pdev->dev, DMA_BIT_MASK(32));
- dev->fw_handler = mtk_vcodec_fw_select(dev, fw_type, VPU_RST_DEC);
+ dev->fw_handler = mtk_vcodec_fw_select(dev, fw_type, DECODER);
if (IS_ERR(dev->fw_handler))
return PTR_ERR(dev->fw_handler);
}
dma_set_max_seg_size(&pdev->dev, DMA_BIT_MASK(32));
- dev->fw_handler = mtk_vcodec_fw_select(dev, fw_type, VPU_RST_ENC);
+ dev->fw_handler = mtk_vcodec_fw_select(dev, fw_type, ENCODER);
if (IS_ERR(dev->fw_handler))
return PTR_ERR(dev->fw_handler);
// SPDX-License-Identifier: GPL-2.0
#include "mtk_vcodec_fw.h"
+#include "mtk_vcodec_fw_priv.h"
#include "mtk_vcodec_util.h"
#include "mtk_vcodec_drv.h"
-struct mtk_vcodec_fw_ops {
- int (*load_firmware)(struct mtk_vcodec_fw *fw);
- unsigned int (*get_vdec_capa)(struct mtk_vcodec_fw *fw);
- unsigned int (*get_venc_capa)(struct mtk_vcodec_fw *fw);
- void * (*map_dm_addr)(struct mtk_vcodec_fw *fw, u32 dtcm_dmem_addr);
- int (*ipi_register)(struct mtk_vcodec_fw *fw, int id,
- mtk_vcodec_ipi_handler handler, const char *name, void *priv);
- int (*ipi_send)(struct mtk_vcodec_fw *fw, int id, void *buf,
- unsigned int len, unsigned int wait);
-};
-
-struct mtk_vcodec_fw {
- enum mtk_vcodec_fw_type type;
- const struct mtk_vcodec_fw_ops *ops;
- struct platform_device *pdev;
- struct mtk_scp *scp;
-};
-
-static int mtk_vcodec_vpu_load_firmware(struct mtk_vcodec_fw *fw)
-{
- return vpu_load_firmware(fw->pdev);
-}
-
-static unsigned int mtk_vcodec_vpu_get_vdec_capa(struct mtk_vcodec_fw *fw)
-{
- return vpu_get_vdec_hw_capa(fw->pdev);
-}
-
-static unsigned int mtk_vcodec_vpu_get_venc_capa(struct mtk_vcodec_fw *fw)
-{
- return vpu_get_venc_hw_capa(fw->pdev);
-}
-
-static void *mtk_vcodec_vpu_map_dm_addr(struct mtk_vcodec_fw *fw,
- u32 dtcm_dmem_addr)
-{
- return vpu_mapping_dm_addr(fw->pdev, dtcm_dmem_addr);
-}
-
-static int mtk_vcodec_vpu_set_ipi_register(struct mtk_vcodec_fw *fw, int id,
- mtk_vcodec_ipi_handler handler,
- const char *name, void *priv)
-{
- /*
- * The handler we receive takes a void * as its first argument. We
- * cannot change this because it needs to be passed down to the rproc
- * subsystem when SCP is used. VPU takes a const argument, which is
- * more constrained, so the conversion below is safe.
- */
- ipi_handler_t handler_const = (ipi_handler_t)handler;
-
- return vpu_ipi_register(fw->pdev, id, handler_const, name, priv);
-}
-
-static int mtk_vcodec_vpu_ipi_send(struct mtk_vcodec_fw *fw, int id, void *buf,
- unsigned int len, unsigned int wait)
-{
- return vpu_ipi_send(fw->pdev, id, buf, len);
-}
-
-static const struct mtk_vcodec_fw_ops mtk_vcodec_vpu_msg = {
- .load_firmware = mtk_vcodec_vpu_load_firmware,
- .get_vdec_capa = mtk_vcodec_vpu_get_vdec_capa,
- .get_venc_capa = mtk_vcodec_vpu_get_venc_capa,
- .map_dm_addr = mtk_vcodec_vpu_map_dm_addr,
- .ipi_register = mtk_vcodec_vpu_set_ipi_register,
- .ipi_send = mtk_vcodec_vpu_ipi_send,
-};
-
-static int mtk_vcodec_scp_load_firmware(struct mtk_vcodec_fw *fw)
-{
- return rproc_boot(scp_get_rproc(fw->scp));
-}
-
-static unsigned int mtk_vcodec_scp_get_vdec_capa(struct mtk_vcodec_fw *fw)
-{
- return scp_get_vdec_hw_capa(fw->scp);
-}
-
-static unsigned int mtk_vcodec_scp_get_venc_capa(struct mtk_vcodec_fw *fw)
-{
- return scp_get_venc_hw_capa(fw->scp);
-}
-
-static void *mtk_vcodec_vpu_scp_dm_addr(struct mtk_vcodec_fw *fw,
- u32 dtcm_dmem_addr)
-{
- return scp_mapping_dm_addr(fw->scp, dtcm_dmem_addr);
-}
-
-static int mtk_vcodec_scp_set_ipi_register(struct mtk_vcodec_fw *fw, int id,
- mtk_vcodec_ipi_handler handler,
- const char *name, void *priv)
-{
- return scp_ipi_register(fw->scp, id, handler, priv);
-}
-
-static int mtk_vcodec_scp_ipi_send(struct mtk_vcodec_fw *fw, int id, void *buf,
- unsigned int len, unsigned int wait)
-{
- return scp_ipi_send(fw->scp, id, buf, len, wait);
-}
-
-static const struct mtk_vcodec_fw_ops mtk_vcodec_rproc_msg = {
- .load_firmware = mtk_vcodec_scp_load_firmware,
- .get_vdec_capa = mtk_vcodec_scp_get_vdec_capa,
- .get_venc_capa = mtk_vcodec_scp_get_venc_capa,
- .map_dm_addr = mtk_vcodec_vpu_scp_dm_addr,
- .ipi_register = mtk_vcodec_scp_set_ipi_register,
- .ipi_send = mtk_vcodec_scp_ipi_send,
-};
-
-static void mtk_vcodec_reset_handler(void *priv)
-{
- struct mtk_vcodec_dev *dev = priv;
- struct mtk_vcodec_ctx *ctx;
-
- mtk_v4l2_err("Watchdog timeout!!");
-
- mutex_lock(&dev->dev_mutex);
- list_for_each_entry(ctx, &dev->ctx_list, list) {
- ctx->state = MTK_STATE_ABORT;
- mtk_v4l2_debug(0, "[%d] Change to state MTK_STATE_ABORT",
- ctx->id);
- }
- mutex_unlock(&dev->dev_mutex);
-}
-
struct mtk_vcodec_fw *mtk_vcodec_fw_select(struct mtk_vcodec_dev *dev,
enum mtk_vcodec_fw_type type,
- enum rst_id rst_id)
+ enum mtk_vcodec_fw_use fw_use)
{
- const struct mtk_vcodec_fw_ops *ops;
- struct mtk_vcodec_fw *fw;
- struct platform_device *fw_pdev = NULL;
- struct mtk_scp *scp = NULL;
-
switch (type) {
case VPU:
- ops = &mtk_vcodec_vpu_msg;
- fw_pdev = vpu_get_plat_device(dev->plat_dev);
- if (!fw_pdev) {
- mtk_v4l2_err("firmware device is not ready");
- return ERR_PTR(-EINVAL);
- }
- vpu_wdt_reg_handler(fw_pdev, mtk_vcodec_reset_handler,
- dev, rst_id);
- break;
+ return mtk_vcodec_fw_vpu_init(dev, fw_use);
case SCP:
- ops = &mtk_vcodec_rproc_msg;
- scp = scp_get(dev->plat_dev);
- if (!scp) {
- mtk_v4l2_err("could not get vdec scp handle");
- return ERR_PTR(-EPROBE_DEFER);
- }
- break;
+ return mtk_vcodec_fw_scp_init(dev);
default:
mtk_v4l2_err("invalid vcodec fw type");
return ERR_PTR(-EINVAL);
}
-
- fw = devm_kzalloc(&dev->plat_dev->dev, sizeof(*fw), GFP_KERNEL);
- if (!fw)
- return ERR_PTR(-EINVAL);
-
- fw->type = type;
- fw->ops = ops;
- fw->pdev = fw_pdev;
- fw->scp = scp;
-
- return fw;
}
EXPORT_SYMBOL_GPL(mtk_vcodec_fw_select);
void mtk_vcodec_fw_release(struct mtk_vcodec_fw *fw)
{
- switch (fw->type) {
- case VPU:
- put_device(&fw->pdev->dev);
- break;
- case SCP:
- scp_put(fw->scp);
- break;
- }
+ fw->ops->release(fw);
}
EXPORT_SYMBOL_GPL(mtk_vcodec_fw_release);
SCP,
};
+enum mtk_vcodec_fw_use {
+ DECODER,
+ ENCODER,
+};
+
struct mtk_vcodec_fw;
typedef void (*mtk_vcodec_ipi_handler) (void *data,
struct mtk_vcodec_fw *mtk_vcodec_fw_select(struct mtk_vcodec_dev *dev,
enum mtk_vcodec_fw_type type,
- enum rst_id rst_id);
+ enum mtk_vcodec_fw_use fw_use);
void mtk_vcodec_fw_release(struct mtk_vcodec_fw *fw);
int mtk_vcodec_fw_load_firmware(struct mtk_vcodec_fw *fw);
--- /dev/null
+/* SPDX-License-Identifier: GPL-2.0 */
+
+#ifndef _MTK_VCODEC_FW_PRIV_H_
+#define _MTK_VCODEC_FW_PRIV_H_
+
+#include "mtk_vcodec_fw.h"
+
+struct mtk_vcodec_dev;
+
+struct mtk_vcodec_fw {
+ enum mtk_vcodec_fw_type type;
+ const struct mtk_vcodec_fw_ops *ops;
+ struct platform_device *pdev;
+ struct mtk_scp *scp;
+};
+
+struct mtk_vcodec_fw_ops {
+ int (*load_firmware)(struct mtk_vcodec_fw *fw);
+ unsigned int (*get_vdec_capa)(struct mtk_vcodec_fw *fw);
+ unsigned int (*get_venc_capa)(struct mtk_vcodec_fw *fw);
+ void *(*map_dm_addr)(struct mtk_vcodec_fw *fw, u32 dtcm_dmem_addr);
+ int (*ipi_register)(struct mtk_vcodec_fw *fw, int id,
+ mtk_vcodec_ipi_handler handler, const char *name,
+ void *priv);
+ int (*ipi_send)(struct mtk_vcodec_fw *fw, int id, void *buf,
+ unsigned int len, unsigned int wait);
+ void (*release)(struct mtk_vcodec_fw *fw);
+};
+
+#if IS_ENABLED(CONFIG_VIDEO_MEDIATEK_VCODEC_VPU)
+struct mtk_vcodec_fw *mtk_vcodec_fw_vpu_init(struct mtk_vcodec_dev *dev,
+ enum mtk_vcodec_fw_use fw_use);
+#else
+static inline struct mtk_vcodec_fw *
+mtk_vcodec_fw_vpu_init(struct mtk_vcodec_dev *dev,
+ enum mtk_vcodec_fw_use fw_use)
+{
+ return ERR_PTR(-ENODEV);
+}
+#endif /* CONFIG_VIDEO_MEDIATEK_VCODEC_VPU */
+
+#if IS_ENABLED(CONFIG_VIDEO_MEDIATEK_VCODEC_SCP)
+struct mtk_vcodec_fw *mtk_vcodec_fw_scp_init(struct mtk_vcodec_dev *dev);
+#else
+static inline struct mtk_vcodec_fw *
+mtk_vcodec_fw_scp_init(struct mtk_vcodec_dev *dev)
+{
+ return ERR_PTR(-ENODEV);
+}
+#endif /* CONFIG_VIDEO_MEDIATEK_VCODEC_SCP */
+
+#endif /* _MTK_VCODEC_FW_PRIV_H_ */
--- /dev/null
+// SPDX-License-Identifier: GPL-2.0
+
+#include "mtk_vcodec_fw_priv.h"
+#include "mtk_vcodec_util.h"
+#include "mtk_vcodec_drv.h"
+
+static int mtk_vcodec_scp_load_firmware(struct mtk_vcodec_fw *fw)
+{
+ return rproc_boot(scp_get_rproc(fw->scp));
+}
+
+static unsigned int mtk_vcodec_scp_get_vdec_capa(struct mtk_vcodec_fw *fw)
+{
+ return scp_get_vdec_hw_capa(fw->scp);
+}
+
+static unsigned int mtk_vcodec_scp_get_venc_capa(struct mtk_vcodec_fw *fw)
+{
+ return scp_get_venc_hw_capa(fw->scp);
+}
+
+static void *mtk_vcodec_vpu_scp_dm_addr(struct mtk_vcodec_fw *fw,
+ u32 dtcm_dmem_addr)
+{
+ return scp_mapping_dm_addr(fw->scp, dtcm_dmem_addr);
+}
+
+static int mtk_vcodec_scp_set_ipi_register(struct mtk_vcodec_fw *fw, int id,
+ mtk_vcodec_ipi_handler handler,
+ const char *name, void *priv)
+{
+ return scp_ipi_register(fw->scp, id, handler, priv);
+}
+
+static int mtk_vcodec_scp_ipi_send(struct mtk_vcodec_fw *fw, int id, void *buf,
+ unsigned int len, unsigned int wait)
+{
+ return scp_ipi_send(fw->scp, id, buf, len, wait);
+}
+
+static void mtk_vcodec_scp_release(struct mtk_vcodec_fw *fw)
+{
+ scp_put(fw->scp);
+}
+
+static const struct mtk_vcodec_fw_ops mtk_vcodec_rproc_msg = {
+ .load_firmware = mtk_vcodec_scp_load_firmware,
+ .get_vdec_capa = mtk_vcodec_scp_get_vdec_capa,
+ .get_venc_capa = mtk_vcodec_scp_get_venc_capa,
+ .map_dm_addr = mtk_vcodec_vpu_scp_dm_addr,
+ .ipi_register = mtk_vcodec_scp_set_ipi_register,
+ .ipi_send = mtk_vcodec_scp_ipi_send,
+ .release = mtk_vcodec_scp_release,
+};
+
+struct mtk_vcodec_fw *mtk_vcodec_fw_scp_init(struct mtk_vcodec_dev *dev)
+{
+ struct mtk_vcodec_fw *fw;
+ struct mtk_scp *scp;
+
+ scp = scp_get(dev->plat_dev);
+ if (!scp) {
+ mtk_v4l2_err("could not get vdec scp handle");
+ return ERR_PTR(-EPROBE_DEFER);
+ }
+
+ fw = devm_kzalloc(&dev->plat_dev->dev, sizeof(*fw), GFP_KERNEL);
+ fw->type = SCP;
+ fw->ops = &mtk_vcodec_rproc_msg;
+ fw->scp = scp;
+
+ return fw;
+}
--- /dev/null
+// SPDX-License-Identifier: GPL-2.0
+
+#include "mtk_vcodec_fw_priv.h"
+#include "mtk_vcodec_util.h"
+#include "mtk_vcodec_drv.h"
+
+static int mtk_vcodec_vpu_load_firmware(struct mtk_vcodec_fw *fw)
+{
+ return vpu_load_firmware(fw->pdev);
+}
+
+static unsigned int mtk_vcodec_vpu_get_vdec_capa(struct mtk_vcodec_fw *fw)
+{
+ return vpu_get_vdec_hw_capa(fw->pdev);
+}
+
+static unsigned int mtk_vcodec_vpu_get_venc_capa(struct mtk_vcodec_fw *fw)
+{
+ return vpu_get_venc_hw_capa(fw->pdev);
+}
+
+static void *mtk_vcodec_vpu_map_dm_addr(struct mtk_vcodec_fw *fw,
+ u32 dtcm_dmem_addr)
+{
+ return vpu_mapping_dm_addr(fw->pdev, dtcm_dmem_addr);
+}
+
+static int mtk_vcodec_vpu_set_ipi_register(struct mtk_vcodec_fw *fw, int id,
+ mtk_vcodec_ipi_handler handler,
+ const char *name, void *priv)
+{
+ /*
+ * The handler we receive takes a void * as its first argument. We
+ * cannot change this because it needs to be passed down to the rproc
+ * subsystem when SCP is used. VPU takes a const argument, which is
+ * more constrained, so the conversion below is safe.
+ */
+ ipi_handler_t handler_const = (ipi_handler_t)handler;
+
+ return vpu_ipi_register(fw->pdev, id, handler_const, name, priv);
+}
+
+static int mtk_vcodec_vpu_ipi_send(struct mtk_vcodec_fw *fw, int id, void *buf,
+ unsigned int len, unsigned int wait)
+{
+ return vpu_ipi_send(fw->pdev, id, buf, len);
+}
+
+static void mtk_vcodec_vpu_release(struct mtk_vcodec_fw *fw)
+{
+ put_device(&fw->pdev->dev);
+}
+
+static void mtk_vcodec_vpu_reset_handler(void *priv)
+{
+ struct mtk_vcodec_dev *dev = priv;
+ struct mtk_vcodec_ctx *ctx;
+
+ mtk_v4l2_err("Watchdog timeout!!");
+
+ mutex_lock(&dev->dev_mutex);
+ list_for_each_entry(ctx, &dev->ctx_list, list) {
+ ctx->state = MTK_STATE_ABORT;
+ mtk_v4l2_debug(0, "[%d] Change to state MTK_STATE_ABORT",
+ ctx->id);
+ }
+ mutex_unlock(&dev->dev_mutex);
+}
+
+static const struct mtk_vcodec_fw_ops mtk_vcodec_vpu_msg = {
+ .load_firmware = mtk_vcodec_vpu_load_firmware,
+ .get_vdec_capa = mtk_vcodec_vpu_get_vdec_capa,
+ .get_venc_capa = mtk_vcodec_vpu_get_venc_capa,
+ .map_dm_addr = mtk_vcodec_vpu_map_dm_addr,
+ .ipi_register = mtk_vcodec_vpu_set_ipi_register,
+ .ipi_send = mtk_vcodec_vpu_ipi_send,
+ .release = mtk_vcodec_vpu_release,
+};
+
+struct mtk_vcodec_fw *mtk_vcodec_fw_vpu_init(struct mtk_vcodec_dev *dev,
+ enum mtk_vcodec_fw_use fw_use)
+{
+ struct platform_device *fw_pdev;
+ struct mtk_vcodec_fw *fw;
+ enum rst_id rst_id;
+
+ switch (fw_use) {
+ case ENCODER:
+ rst_id = VPU_RST_ENC;
+ break;
+ case DECODER:
+ default:
+ rst_id = VPU_RST_DEC;
+ break;
+ }
+
+ fw_pdev = vpu_get_plat_device(dev->plat_dev);
+ if (!fw_pdev) {
+ mtk_v4l2_err("firmware device is not ready");
+ return ERR_PTR(-EINVAL);
+ }
+ vpu_wdt_reg_handler(fw_pdev, mtk_vcodec_vpu_reset_handler, dev, rst_id);
+
+ fw = devm_kzalloc(&dev->plat_dev->dev, sizeof(*fw), GFP_KERNEL);
+ fw->type = VPU;
+ fw->ops = &mtk_vcodec_vpu_msg;
+ fw->pdev = fw_pdev;
+
+ return fw;
+}
u32 header_mode;
- u32 profile;
- u32 level;
+ struct {
+ u32 h264;
+ u32 mpeg4;
+ u32 hevc;
+ u32 vp8;
+ u32 vp9;
+ } profile;
+ struct {
+ u32 h264;
+ u32 mpeg4;
+ u32 hevc;
+ u32 vp9;
+ } level;
};
struct venus_buffer {
return 0;
opp_dl_add_err:
- dev_pm_domain_detach(core->opp_pmdomain, true);
+ dev_pm_opp_detach_genpd(core->opp_table);
opp_attach_err:
if (core->pd_dl_venus) {
device_link_del(core->pd_dl_venus);
if (core->opp_dl_venus)
device_link_del(core->opp_dl_venus);
- dev_pm_domain_detach(core->opp_pmdomain, true);
+ dev_pm_opp_detach_genpd(core->opp_table);
}
static int core_get_v4(struct device *dev)
struct hfi_quantization quant;
struct hfi_quantization_range quant_range;
u32 ptype, rate_control, bitrate;
+ u32 profile, level;
int ret;
ret = venus_helper_set_work_mode(inst, VIDC_WORK_MODE_2);
if (ret)
return ret;
- ret = venus_helper_set_profile_level(inst, ctr->profile, ctr->level);
+ switch (inst->hfi_codec) {
+ case HFI_VIDEO_CODEC_H264:
+ profile = ctr->profile.h264;
+ level = ctr->level.h264;
+ break;
+ case HFI_VIDEO_CODEC_MPEG4:
+ profile = ctr->profile.mpeg4;
+ level = ctr->level.mpeg4;
+ break;
+ case HFI_VIDEO_CODEC_VP8:
+ profile = ctr->profile.vp8;
+ level = 0;
+ break;
+ case HFI_VIDEO_CODEC_VP9:
+ profile = ctr->profile.vp9;
+ level = ctr->level.vp9;
+ break;
+ case HFI_VIDEO_CODEC_HEVC:
+ profile = ctr->profile.hevc;
+ level = ctr->level.hevc;
+ break;
+ case HFI_VIDEO_CODEC_MPEG2:
+ default:
+ profile = 0;
+ level = 0;
+ break;
+ }
+
+ ret = venus_helper_set_profile_level(inst, profile, level);
if (ret)
return ret;
ctr->h264_entropy_mode = ctrl->val;
break;
case V4L2_CID_MPEG_VIDEO_MPEG4_PROFILE:
+ ctr->profile.mpeg4 = ctrl->val;
+ break;
case V4L2_CID_MPEG_VIDEO_H264_PROFILE:
+ ctr->profile.h264 = ctrl->val;
+ break;
case V4L2_CID_MPEG_VIDEO_HEVC_PROFILE:
+ ctr->profile.hevc = ctrl->val;
+ break;
case V4L2_CID_MPEG_VIDEO_VP8_PROFILE:
- ctr->profile = ctrl->val;
+ ctr->profile.vp8 = ctrl->val;
break;
case V4L2_CID_MPEG_VIDEO_MPEG4_LEVEL:
+ ctr->level.mpeg4 = ctrl->val;
+ break;
case V4L2_CID_MPEG_VIDEO_H264_LEVEL:
+ ctr->level.h264 = ctrl->val;
+ break;
case V4L2_CID_MPEG_VIDEO_HEVC_LEVEL:
- ctr->level = ctrl->val;
+ ctr->level.hevc = ctrl->val;
break;
case V4L2_CID_MPEG_VIDEO_H264_I_FRAME_QP:
ctr->h264_i_qp = ctrl->val;
* validate the existing APIs in the media subsystem. It can also aid
* developers working on userspace applications.
*
- * When this module is loaded, it will attempt to modprobe 'dvb_vidtv_tuner' and 'dvb_vidtv_demod'.
+ * When this module is loaded, it will attempt to modprobe 'dvb_vidtv_tuner'
+ * and 'dvb_vidtv_demod'.
*
* Copyright (C) 2020 Daniel W. S. Almeida
*/
+#include <linux/dev_printk.h>
#include <linux/moduleparam.h>
#include <linux/mutex.h>
#include <linux/platform_device.h>
-#include <linux/dev_printk.h>
#include <linux/time.h>
#include <linux/types.h>
#include <linux/workqueue.h>
#include "vidtv_bridge.h"
+#include "vidtv_common.h"
#include "vidtv_demod.h"
-#include "vidtv_tuner.h"
-#include "vidtv_ts.h"
#include "vidtv_mux.h"
-#include "vidtv_common.h"
+#include "vidtv_ts.h"
+#include "vidtv_tuner.h"
-//#define MUX_BUF_MAX_SZ
-//#define MUX_BUF_MIN_SZ
+#define MUX_BUF_MIN_SZ 90164
+#define MUX_BUF_MAX_SZ (MUX_BUF_MIN_SZ * 10)
#define TUNER_DEFAULT_ADDR 0x68
#define DEMOD_DEFAULT_ADDR 0x60
+#define VIDTV_DEFAULT_NETWORK_ID 0xff44
+#define VIDTV_DEFAULT_NETWORK_NAME "LinuxTV.org"
+#define VIDTV_DEFAULT_TS_ID 0x4081
-/* LNBf fake parameters: ranges used by an Universal (extended) European LNBf */
-#define LNB_CUT_FREQUENCY 11700000
-#define LNB_LOW_FREQ 9750000
-#define LNB_HIGH_FREQ 10600000
-
+/*
+ * The LNBf fake parameters here are the ranges used by an
+ * Universal (extended) European LNBf, which is likely the most common LNBf
+ * found on Satellite digital TV system nowadays.
+ */
+#define LNB_CUT_FREQUENCY 11700000 /* high IF frequency */
+#define LNB_LOW_FREQ 9750000 /* low IF frequency */
+#define LNB_HIGH_FREQ 10600000 /* transition frequency */
static unsigned int drop_tslock_prob_on_low_snr;
module_param(drop_tslock_prob_on_low_snr, uint, 0);
static unsigned int pcr_period_msec = 40;
module_param(pcr_period_msec, uint, 0);
-MODULE_PARM_DESC(pcr_period_msec, "How often to send PCR packets. Default: 40ms");
+MODULE_PARM_DESC(pcr_period_msec,
+ "How often to send PCR packets. Default: 40ms");
static unsigned int mux_rate_kbytes_sec = 4096;
module_param(mux_rate_kbytes_sec, uint, 0);
static unsigned int mux_buf_sz_pkts;
module_param(mux_buf_sz_pkts, uint, 0);
-MODULE_PARM_DESC(mux_buf_sz_pkts, "Size for the internal mux buffer in multiples of 188 bytes");
-
-#define MUX_BUF_MIN_SZ 90164
-#define MUX_BUF_MAX_SZ (MUX_BUF_MIN_SZ * 10)
+MODULE_PARM_DESC(mux_buf_sz_pkts,
+ "Size for the internal mux buffer in multiples of 188 bytes");
static u32 vidtv_bridge_mux_buf_sz_for_mux_rate(void)
{
u32 max_elapsed_time_msecs = VIDTV_MAX_SLEEP_USECS / USEC_PER_MSEC;
- u32 nbytes_expected;
u32 mux_buf_sz = mux_buf_sz_pkts * TS_PACKET_LEN;
+ u32 nbytes_expected;
nbytes_expected = mux_rate_kbytes_sec;
nbytes_expected *= max_elapsed_time_msecs;
FE_HAS_LOCK);
}
-static void
-vidtv_bridge_on_new_pkts_avail(void *priv, u8 *buf, u32 npkts)
+/*
+ * called on a separate thread by the mux when new packets become available
+ */
+static void vidtv_bridge_on_new_pkts_avail(void *priv, u8 *buf, u32 npkts)
{
- /*
- * called on a separate thread by the mux when new packets become
- * available
- */
- struct vidtv_dvb *dvb = (struct vidtv_dvb *)priv;
+ struct vidtv_dvb *dvb = priv;
/* drop packets if we lose the lock */
if (vidtv_bridge_check_demod_lock(dvb, 0))
static int vidtv_start_streaming(struct vidtv_dvb *dvb)
{
- struct vidtv_mux_init_args mux_args = {0};
+ struct vidtv_mux_init_args mux_args = {
+ .mux_rate_kbytes_sec = mux_rate_kbytes_sec,
+ .on_new_packets_available_cb = vidtv_bridge_on_new_pkts_avail,
+ .pcr_period_usecs = pcr_period_msec * USEC_PER_MSEC,
+ .si_period_usecs = si_period_msec * USEC_PER_MSEC,
+ .pcr_pid = pcr_pid,
+ .transport_stream_id = VIDTV_DEFAULT_TS_ID,
+ .network_id = VIDTV_DEFAULT_NETWORK_ID,
+ .network_name = VIDTV_DEFAULT_NETWORK_NAME,
+ .priv = dvb,
+ };
struct device *dev = &dvb->pdev->dev;
u32 mux_buf_sz;
return 0;
}
- mux_buf_sz = (mux_buf_sz_pkts) ? mux_buf_sz_pkts : vidtv_bridge_mux_buf_sz_for_mux_rate();
+ if (mux_buf_sz_pkts)
+ mux_buf_sz = mux_buf_sz_pkts;
+ else
+ mux_buf_sz = vidtv_bridge_mux_buf_sz_for_mux_rate();
- mux_args.mux_rate_kbytes_sec = mux_rate_kbytes_sec;
- mux_args.on_new_packets_available_cb = vidtv_bridge_on_new_pkts_avail;
- mux_args.mux_buf_sz = mux_buf_sz;
- mux_args.pcr_period_usecs = pcr_period_msec * 1000;
- mux_args.si_period_usecs = si_period_msec * 1000;
- mux_args.pcr_pid = pcr_pid;
- mux_args.transport_stream_id = VIDTV_DEFAULT_TS_ID;
- mux_args.priv = dvb;
+ mux_args.mux_buf_sz = mux_buf_sz;
dvb->streaming = true;
- dvb->mux = vidtv_mux_init(dvb->fe[0], dev, mux_args);
+ dvb->mux = vidtv_mux_init(dvb->fe[0], dev, &mux_args);
+ if (!dvb->mux)
+ return -ENOMEM;
vidtv_mux_start_thread(dvb->mux);
dev_dbg_ratelimited(dev, "Started streaming\n");
{
struct dvb_demux *demux = feed->demux;
struct vidtv_dvb *dvb = demux->priv;
- int rc;
int ret;
+ int rc;
if (!demux->dmx.frontend)
return -EINVAL;
static struct dvb_frontend *vidtv_get_frontend_ptr(struct i2c_client *c)
{
- /* the demod will set this when its probe function runs */
struct vidtv_demod_state *state = i2c_get_clientdata(c);
+ /* the demod will set this when its probe function runs */
return &state->frontend;
}
struct i2c_msg msgs[],
int num)
{
+ /*
+ * Right now, this virtual driver doesn't really send or receive
+ * messages from I2C. A real driver will require an implementation
+ * here.
+ */
return 0;
}
static int vidtv_bridge_probe_demod(struct vidtv_dvb *dvb, u32 n)
{
- struct vidtv_demod_config cfg = {};
-
- cfg.drop_tslock_prob_on_low_snr = drop_tslock_prob_on_low_snr;
- cfg.recover_tslock_prob_on_good_snr = recover_tslock_prob_on_good_snr;
-
+ struct vidtv_demod_config cfg = {
+ .drop_tslock_prob_on_low_snr = drop_tslock_prob_on_low_snr,
+ .recover_tslock_prob_on_good_snr = recover_tslock_prob_on_good_snr,
+ };
dvb->i2c_client_demod[n] = dvb_module_probe("dvb_vidtv_demod",
NULL,
&dvb->i2c_adapter,
static int vidtv_bridge_probe_tuner(struct vidtv_dvb *dvb, u32 n)
{
- struct vidtv_tuner_config cfg = {};
+ struct vidtv_tuner_config cfg = {
+ .fe = dvb->fe[n],
+ .mock_power_up_delay_msec = mock_power_up_delay_msec,
+ .mock_tune_delay_msec = mock_tune_delay_msec,
+ };
u32 freq;
int i;
- cfg.fe = dvb->fe[n];
- cfg.mock_power_up_delay_msec = mock_power_up_delay_msec;
- cfg.mock_tune_delay_msec = mock_tune_delay_msec;
-
/* TODO: check if the frequencies are at a valid range */
memcpy(cfg.vidtv_valid_dvb_t_freqs,
static int vidtv_bridge_dvb_init(struct vidtv_dvb *dvb)
{
- int ret;
- int i;
- int j;
+ int ret, i, j;
ret = vidtv_bridge_i2c_register_adap(dvb);
if (ret < 0)
#include <linux/i2c.h>
#include <linux/platform_device.h>
#include <linux/types.h>
+
#include <media/dmxdev.h>
#include <media/dvb_demux.h>
#include <media/dvb_frontend.h>
+
#include "vidtv_mux.h"
/**
* @adapter: Represents a DTV adapter. See 'dvb_register_adapter'.
* @demux: The demux used by the dvb_dmx_swfilter_packets() call.
* @dmx_dev: Represents a demux device.
- * @dmx_frontend: The frontends associated with the demux.
+ * @dmx_fe: The frontends associated with the demux.
* @i2c_adapter: The i2c_adapter associated with the bridge driver.
* @i2c_client_demod: The i2c_clients associated with the demodulator modules.
* @i2c_client_tuner: The i2c_clients associated with the tuner modules.
* When vidtv boots, it will create some hardcoded channels.
* Their services will be concatenated to populate the SDT.
* Their programs will be concatenated to populate the PAT
+ * Their events will be concatenated to populate the EIT
* For each program in the PAT, a PMT section will be created
* The PMT section for a channel will be assigned its streams.
* Every stream will have its corresponding encoder polled to produce TS packets
* Copyright (C) 2020 Daniel W. S. Almeida
*/
-#include <linux/types.h>
-#include <linux/slab.h>
#include <linux/dev_printk.h>
#include <linux/ratelimit.h>
+#include <linux/slab.h>
+#include <linux/types.h>
#include "vidtv_channel.h"
-#include "vidtv_psi.h"
+#include "vidtv_common.h"
#include "vidtv_encoder.h"
#include "vidtv_mux.h"
-#include "vidtv_common.h"
+#include "vidtv_psi.h"
#include "vidtv_s302m.h"
static void vidtv_channel_encoder_destroy(struct vidtv_encoder *e)
{
- struct vidtv_encoder *curr = e;
struct vidtv_encoder *tmp = NULL;
+ struct vidtv_encoder *curr = e;
while (curr) {
/* forward the call to the derived type */
}
#define ENCODING_ISO8859_15 "\x0b"
+#define TS_NIT_PID 0x10
+/*
+ * init an audio only channel with a s302m encoder
+ */
struct vidtv_channel
*vidtv_channel_s302m_init(struct vidtv_channel *head, u16 transport_stream_id)
{
- /*
- * init an audio only channel with a s302m encoder
- */
- const u16 s302m_service_id = 0x880;
- const u16 s302m_program_num = 0x880;
- const u16 s302m_program_pid = 0x101; /* packet id for PMT*/
- const u16 s302m_es_pid = 0x111; /* packet id for the ES */
const __be32 s302m_fid = cpu_to_be32(VIDTV_S302M_FORMAT_IDENTIFIER);
-
- char *name = ENCODING_ISO8859_15 "Beethoven";
+ char *event_text = ENCODING_ISO8859_15 "Bagatelle No. 25 in A minor for solo piano, also known as F\xfcr Elise, composed by Ludwig van Beethoven";
+ char *event_name = ENCODING_ISO8859_15 "Ludwig van Beethoven: F\xfcr Elise";
+ struct vidtv_s302m_encoder_init_args encoder_args = {};
+ char *iso_language_code = ENCODING_ISO8859_15 "eng";
char *provider = ENCODING_ISO8859_15 "LinuxTV.org";
+ char *name = ENCODING_ISO8859_15 "Beethoven";
+ const u16 s302m_es_pid = 0x111; /* packet id for the ES */
+ const u16 s302m_program_pid = 0x101; /* packet id for PMT*/
+ const u16 s302m_service_id = 0x880;
+ const u16 s302m_program_num = 0x880;
+ const u16 s302m_beethoven_event_id = 1;
+ struct vidtv_channel *s302m;
- struct vidtv_channel *s302m = kzalloc(sizeof(*s302m), GFP_KERNEL);
- struct vidtv_s302m_encoder_init_args encoder_args = {};
+ s302m = kzalloc(sizeof(*s302m), GFP_KERNEL);
+ if (!s302m)
+ return NULL;
s302m->name = kstrdup(name, GFP_KERNEL);
+ if (!s302m->name)
+ goto free_s302m;
- s302m->service = vidtv_psi_sdt_service_init(NULL, s302m_service_id);
+ s302m->service = vidtv_psi_sdt_service_init(NULL, s302m_service_id, false, true);
+ if (!s302m->service)
+ goto free_name;
s302m->service->descriptor = (struct vidtv_psi_desc *)
vidtv_psi_service_desc_init(NULL,
- DIGITAL_TELEVISION_SERVICE,
+ DIGITAL_RADIO_SOUND_SERVICE,
name,
provider);
+ if (!s302m->service->descriptor)
+ goto free_service;
s302m->transport_stream_id = transport_stream_id;
s302m->program = vidtv_psi_pat_program_init(NULL,
s302m_service_id,
s302m_program_pid);
+ if (!s302m->program)
+ goto free_service;
s302m->program_num = s302m_program_num;
s302m->streams = vidtv_psi_pmt_stream_init(NULL,
STREAM_PRIVATE_DATA,
s302m_es_pid);
+ if (!s302m->streams)
+ goto free_program;
s302m->streams->descriptor = (struct vidtv_psi_desc *)
vidtv_psi_registration_desc_init(NULL,
s302m_fid,
NULL,
0);
+ if (!s302m->streams->descriptor)
+ goto free_streams;
+
encoder_args.es_pid = s302m_es_pid;
s302m->encoders = vidtv_s302m_encoder_init(encoder_args);
+ if (!s302m->encoders)
+ goto free_streams;
+
+ s302m->events = vidtv_psi_eit_event_init(NULL, s302m_beethoven_event_id);
+ if (!s302m->events)
+ goto free_encoders;
+ s302m->events->descriptor = (struct vidtv_psi_desc *)
+ vidtv_psi_short_event_desc_init(NULL,
+ iso_language_code,
+ event_name,
+ event_text);
+ if (!s302m->events->descriptor)
+ goto free_events;
if (head) {
while (head->next)
}
return s302m;
+
+free_events:
+ vidtv_psi_eit_event_destroy(s302m->events);
+free_encoders:
+ vidtv_s302m_encoder_destroy(s302m->encoders);
+free_streams:
+ vidtv_psi_pmt_stream_destroy(s302m->streams);
+free_program:
+ vidtv_psi_pat_program_destroy(s302m->program);
+free_service:
+ vidtv_psi_sdt_service_destroy(s302m->service);
+free_name:
+ kfree(s302m->name);
+free_s302m:
+ kfree(s302m);
+
+ return NULL;
+}
+
+static struct vidtv_psi_table_eit_event
+*vidtv_channel_eit_event_cat_into_new(struct vidtv_mux *m)
+{
+ /* Concatenate the events */
+ const struct vidtv_channel *cur_chnl = m->channels;
+ struct vidtv_psi_table_eit_event *curr = NULL;
+ struct vidtv_psi_table_eit_event *head = NULL;
+ struct vidtv_psi_table_eit_event *tail = NULL;
+ struct vidtv_psi_desc *desc = NULL;
+ u16 event_id;
+
+ if (!cur_chnl)
+ return NULL;
+
+ while (cur_chnl) {
+ curr = cur_chnl->events;
+
+ if (!curr)
+ dev_warn_ratelimited(m->dev,
+ "No events found for channel %s\n",
+ cur_chnl->name);
+
+ while (curr) {
+ event_id = be16_to_cpu(curr->event_id);
+ tail = vidtv_psi_eit_event_init(tail, event_id);
+ if (!tail) {
+ vidtv_psi_eit_event_destroy(head);
+ return NULL;
+ }
+
+ desc = vidtv_psi_desc_clone(curr->descriptor);
+ vidtv_psi_desc_assign(&tail->descriptor, desc);
+
+ if (!head)
+ head = tail;
+
+ curr = curr->next;
+ }
+
+ cur_chnl = cur_chnl->next;
+ }
+
+ return head;
}
static struct vidtv_psi_table_sdt_service
if (!curr)
dev_warn_ratelimited(m->dev,
- "No services found for channel %s\n", cur_chnl->name);
+ "No services found for channel %s\n",
+ cur_chnl->name);
while (curr) {
service_id = be16_to_cpu(curr->service_id);
- tail = vidtv_psi_sdt_service_init(tail, service_id);
+ tail = vidtv_psi_sdt_service_init(tail,
+ service_id,
+ curr->EIT_schedule,
+ curr->EIT_present_following);
+ if (!tail)
+ goto free;
desc = vidtv_psi_desc_clone(curr->descriptor);
+ if (!desc)
+ goto free_tail;
vidtv_psi_desc_assign(&tail->descriptor, desc);
if (!head)
}
return head;
+
+free_tail:
+ vidtv_psi_sdt_service_destroy(tail);
+free:
+ vidtv_psi_sdt_service_destroy(head);
+ return NULL;
}
static struct vidtv_psi_table_pat_program*
tail = vidtv_psi_pat_program_init(tail,
serv_id,
pid);
+ if (!tail) {
+ vidtv_psi_pat_program_destroy(head);
+ return NULL;
+ }
if (!head)
head = tail;
cur_chnl = cur_chnl->next;
}
+ /* Add the NIT table */
+ vidtv_psi_pat_program_init(tail, 0, TS_NIT_PID);
return head;
}
+/*
+ * Match channels to their respective PMT sections, then assign the
+ * streams
+ */
static void
vidtv_channel_pmt_match_sections(struct vidtv_channel *channels,
struct vidtv_psi_table_pmt **sections,
u32 nsections)
{
- /*
- * Match channels to their respective PMT sections, then assign the
- * streams
- */
struct vidtv_psi_table_pmt *curr_section = NULL;
- struct vidtv_channel *cur_chnl = channels;
-
- struct vidtv_psi_table_pmt_stream *s = NULL;
struct vidtv_psi_table_pmt_stream *head = NULL;
struct vidtv_psi_table_pmt_stream *tail = NULL;
-
+ struct vidtv_psi_table_pmt_stream *s = NULL;
+ struct vidtv_channel *cur_chnl = channels;
struct vidtv_psi_desc *desc = NULL;
- u32 j;
- u16 curr_id;
u16 e_pid; /* elementary stream pid */
+ u16 curr_id;
+ u32 j;
while (cur_chnl) {
for (j = 0; j < nsections; ++j) {
head = tail;
desc = vidtv_psi_desc_clone(s->descriptor);
- vidtv_psi_desc_assign(&tail->descriptor, desc);
+ vidtv_psi_desc_assign(&tail->descriptor,
+ desc);
s = s->next;
}
}
}
-void vidtv_channel_si_init(struct vidtv_mux *m)
+static void
+vidtv_channel_destroy_service_list(struct vidtv_psi_desc_service_list_entry *e)
+{
+ struct vidtv_psi_desc_service_list_entry *tmp;
+
+ while (e) {
+ tmp = e;
+ e = e->next;
+ kfree(tmp);
+ }
+}
+
+static struct vidtv_psi_desc_service_list_entry
+*vidtv_channel_build_service_list(struct vidtv_psi_table_sdt_service *s)
+{
+ struct vidtv_psi_desc_service_list_entry *curr_e = NULL;
+ struct vidtv_psi_desc_service_list_entry *head_e = NULL;
+ struct vidtv_psi_desc_service_list_entry *prev_e = NULL;
+ struct vidtv_psi_desc *desc = s->descriptor;
+ struct vidtv_psi_desc_service *s_desc;
+
+ while (s) {
+ while (desc) {
+ if (s->descriptor->type != SERVICE_DESCRIPTOR)
+ goto next_desc;
+
+ s_desc = (struct vidtv_psi_desc_service *)desc;
+
+ curr_e = kzalloc(sizeof(*curr_e), GFP_KERNEL);
+ if (!curr_e) {
+ vidtv_channel_destroy_service_list(head_e);
+ return NULL;
+ }
+
+ curr_e->service_id = s->service_id;
+ curr_e->service_type = s_desc->service_type;
+
+ if (!head_e)
+ head_e = curr_e;
+ if (prev_e)
+ prev_e->next = curr_e;
+
+ prev_e = curr_e;
+
+next_desc:
+ desc = desc->next;
+ }
+ s = s->next;
+ }
+ return head_e;
+}
+
+int vidtv_channel_si_init(struct vidtv_mux *m)
{
+ struct vidtv_psi_desc_service_list_entry *service_list = NULL;
struct vidtv_psi_table_pat_program *programs = NULL;
struct vidtv_psi_table_sdt_service *services = NULL;
+ struct vidtv_psi_table_eit_event *events = NULL;
m->si.pat = vidtv_psi_pat_table_init(m->transport_stream_id);
+ if (!m->si.pat)
+ return -ENOMEM;
- m->si.sdt = vidtv_psi_sdt_table_init(m->transport_stream_id);
+ m->si.sdt = vidtv_psi_sdt_table_init(m->network_id,
+ m->transport_stream_id);
+ if (!m->si.sdt)
+ goto free_pat;
programs = vidtv_channel_pat_prog_cat_into_new(m);
+ if (!programs)
+ goto free_sdt;
services = vidtv_channel_sdt_serv_cat_into_new(m);
+ if (!services)
+ goto free_programs;
+
+ events = vidtv_channel_eit_event_cat_into_new(m);
+ if (!events)
+ goto free_services;
+
+ /* look for a service descriptor for every service */
+ service_list = vidtv_channel_build_service_list(services);
+ if (!service_list)
+ goto free_events;
+
+ /* use these descriptors to build the NIT */
+ m->si.nit = vidtv_psi_nit_table_init(m->network_id,
+ m->transport_stream_id,
+ m->network_name,
+ service_list);
+ if (!m->si.nit)
+ goto free_service_list;
+
+ m->si.eit = vidtv_psi_eit_table_init(m->network_id,
+ m->transport_stream_id,
+ programs->service_id);
+ if (!m->si.eit)
+ goto free_nit;
/* assemble all programs and assign to PAT */
vidtv_psi_pat_program_assign(m->si.pat, programs);
/* assemble all services and assign to SDT */
vidtv_psi_sdt_service_assign(m->si.sdt, services);
- m->si.pmt_secs = vidtv_psi_pmt_create_sec_for_each_pat_entry(m->si.pat, m->pcr_pid);
+ /* assemble all events and assign to EIT */
+ vidtv_psi_eit_event_assign(m->si.eit, events);
+
+ m->si.pmt_secs = vidtv_psi_pmt_create_sec_for_each_pat_entry(m->si.pat,
+ m->pcr_pid);
+ if (!m->si.pmt_secs)
+ goto free_eit;
vidtv_channel_pmt_match_sections(m->channels,
m->si.pmt_secs,
- m->si.pat->programs);
+ m->si.pat->num_pmt);
+
+ vidtv_channel_destroy_service_list(service_list);
+
+ return 0;
+
+free_eit:
+ vidtv_psi_eit_table_destroy(m->si.eit);
+free_nit:
+ vidtv_psi_nit_table_destroy(m->si.nit);
+free_service_list:
+ vidtv_channel_destroy_service_list(service_list);
+free_events:
+ vidtv_psi_eit_event_destroy(events);
+free_services:
+ vidtv_psi_sdt_service_destroy(services);
+free_programs:
+ vidtv_psi_pat_program_destroy(programs);
+free_sdt:
+ vidtv_psi_sdt_table_destroy(m->si.sdt);
+free_pat:
+ vidtv_psi_pat_table_destroy(m->si.pat);
+ return 0;
}
void vidtv_channel_si_destroy(struct vidtv_mux *m)
{
u32 i;
- u16 num_programs = m->si.pat->programs;
vidtv_psi_pat_table_destroy(m->si.pat);
- for (i = 0; i < num_programs; ++i)
+ for (i = 0; i < m->si.pat->num_pmt; ++i)
vidtv_psi_pmt_table_destroy(m->si.pmt_secs[i]);
kfree(m->si.pmt_secs);
vidtv_psi_sdt_table_destroy(m->si.sdt);
+ vidtv_psi_nit_table_destroy(m->si.nit);
+ vidtv_psi_eit_table_destroy(m->si.eit);
}
-void vidtv_channels_init(struct vidtv_mux *m)
+int vidtv_channels_init(struct vidtv_mux *m)
{
/* this is the place to add new 'channels' for vidtv */
m->channels = vidtv_channel_s302m_init(NULL, m->transport_stream_id);
+
+ if (!m->channels)
+ return -ENOMEM;
+
+ return 0;
}
void vidtv_channels_destroy(struct vidtv_mux *m)
vidtv_psi_pat_program_destroy(curr->program);
vidtv_psi_pmt_stream_destroy(curr->streams);
vidtv_channel_encoder_destroy(curr->encoders);
+ vidtv_psi_eit_event_destroy(curr->events);
tmp = curr;
curr = curr->next;
* When vidtv boots, it will create some hardcoded channels.
* Their services will be concatenated to populate the SDT.
* Their programs will be concatenated to populate the PAT
+ * Their events will be concatenated to populate the EIT
* For each program in the PAT, a PMT section will be created
* The PMT section for a channel will be assigned its streams.
* Every stream will have its corresponding encoder polled to produce TS packets
#define VIDTV_CHANNEL_H
#include <linux/types.h>
-#include "vidtv_psi.h"
+
#include "vidtv_encoder.h"
#include "vidtv_mux.h"
+#include "vidtv_psi.h"
/**
* struct vidtv_channel - A 'channel' abstraction
* Every stream will have its corresponding encoder polled to produce TS packets
* These packets may be interleaved by the mux and then delivered to the bridge
*
+ * @name: name of the channel
* @transport_stream_id: a number to identify the TS, chosen at will.
* @service: A _single_ service. Will be concatenated into the SDT.
* @program_num: The link between PAT, PMT and SDT.
* Will be concatenated into the PAT.
* @streams: A stream loop used to populate the PMT section for 'program'
* @encoders: A encoder loop. There must be one encoder for each stream.
+ * @events: Optional event information. This will feed into the EIT.
* @next: Optionally chain this channel.
*/
struct vidtv_channel {
struct vidtv_psi_table_pat_program *program;
struct vidtv_psi_table_pmt_stream *streams;
struct vidtv_encoder *encoders;
+ struct vidtv_psi_table_eit_event *events;
struct vidtv_channel *next;
};
* vidtv_channel_si_init - Init the PSI tables from the channels in the mux
* @m: The mux containing the channels.
*/
-void vidtv_channel_si_init(struct vidtv_mux *m);
+int vidtv_channel_si_init(struct vidtv_mux *m);
void vidtv_channel_si_destroy(struct vidtv_mux *m);
/**
* vidtv_channels_init - Init hardcoded, fake 'channels'.
* @m: The mux to store the channels into.
*/
-void vidtv_channels_init(struct vidtv_mux *m);
+int vidtv_channels_init(struct vidtv_mux *m);
struct vidtv_channel
*vidtv_channel_s302m_init(struct vidtv_channel *head, u16 transport_stream_id);
void vidtv_channels_destroy(struct vidtv_mux *m);
#define CLOCK_UNIT_27MHZ 27000000
#define VIDTV_SLEEP_USECS 10000
#define VIDTV_MAX_SLEEP_USECS (2 * VIDTV_SLEEP_USECS)
-#define VIDTV_DEFAULT_TS_ID 0x744
u32 vidtv_memcpy(void *to,
size_t to_offset,
#include <linux/slab.h>
#include <linux/string.h>
#include <linux/workqueue.h>
+
#include <media/dvb_frontend.h>
#include "vidtv_demod.h"
c->cnr.stat[0].svalue = state->tuner_cnr;
c->cnr.stat[0].svalue -= prandom_u32_max(state->tuner_cnr / 50);
-
}
static int vidtv_demod_read_status(struct dvb_frontend *fe,
#define VIDTV_DEMOD_H
#include <linux/dvb/frontend.h>
+
#include <media/dvb_frontend.h>
/**
* modulation and fec_inner
* @modulation: see enum fe_modulation
* @fec: see enum fe_fec_rate
+ * @cnr_ok: S/N threshold to consider the signal as OK. Below that, there's
+ * a chance of losing sync.
+ * @cnr_good: S/N threshold to consider the signal strong.
*
* This struct matches values for 'good' and 'ok' CNRs given the combination
* of modulation and fec_inner in use. We might simulate some noise if the
* struct vidtv_demod_state - The demodulator state
* @frontend: The frontend structure allocated by the demod.
* @config: The config used to init the demod.
- * @poll_snr: The task responsible for periodically checking the simulated
- * signal quality, eventually dropping or reacquiring the TS lock.
* @status: the demod status.
- * @cold_start: Whether the demod has not been init yet.
- * @poll_snr_thread_running: Whether the task responsible for periodically
- * checking the simulated signal quality is running.
- * @poll_snr_thread_restart: Whether we should restart the poll_snr task.
+ * @tuner_cnr: current S/N ratio for the signal carrier
*/
struct vidtv_demod_state {
struct dvb_frontend frontend;
struct vidtv_access_unit *next;
};
-/* Some musical notes, used by a tone generator */
+/* Some musical notes, used by a tone generator. Values are in Hz */
enum musical_notes {
NOTE_SILENT = 0,
* @encoder_buf_sz: The encoder buffer size, in bytes
* @encoder_buf_offset: Our byte position in the encoder buffer.
* @sample_count: How many samples we have encoded in total.
+ * @access_units: encoder payload units, used for clock references
* @src_buf: The source of raw data to be encoded, encoder might set a
* default if null.
+ * @src_buf_sz: size of @src_buf.
* @src_buf_offset: Our position in the source buffer.
* @is_video_encoder: Whether this a video encoder (as opposed to audio)
* @ctx: Encoder-specific state.
* @stream_id: Examples: Audio streams (0xc0-0xdf), Video streams
* (0xe0-0xef).
- * @es_id: The TS PID to use for the elementary stream in this encoder.
+ * @es_pid: The TS PID to use for the elementary stream in this encoder.
* @encode: Prepare enough AUs for the given amount of time.
* @clear: Clear the encoder output.
* @sync: Attempt to synchronize with this encoder.
u32 encoder_buf_offset;
u64 sample_count;
- int last_duration;
- int note_offset;
- enum musical_notes last_tone;
struct vidtv_access_unit *access_units;
* Copyright (C) 2020 Daniel W. S. Almeida
*/
-#include <linux/types.h>
-#include <linux/slab.h>
+#include <linux/delay.h>
+#include <linux/dev_printk.h>
#include <linux/jiffies.h>
#include <linux/kernel.h>
-#include <linux/dev_printk.h>
+#include <linux/math64.h>
#include <linux/ratelimit.h>
-#include <linux/delay.h>
+#include <linux/slab.h>
+#include <linux/types.h>
#include <linux/vmalloc.h>
-#include <linux/math64.h>
-#include "vidtv_mux.h"
-#include "vidtv_ts.h"
-#include "vidtv_pes.h"
-#include "vidtv_encoder.h"
#include "vidtv_channel.h"
#include "vidtv_common.h"
+#include "vidtv_encoder.h"
+#include "vidtv_mux.h"
+#include "vidtv_pes.h"
#include "vidtv_psi.h"
+#include "vidtv_ts.h"
static struct vidtv_mux_pid_ctx
*vidtv_mux_get_pid_ctx(struct vidtv_mux *m, u16 pid)
struct vidtv_mux_pid_ctx *ctx;
ctx = vidtv_mux_get_pid_ctx(m, pid);
-
if (ctx)
- goto end;
+ return ctx;
+
+ ctx = kzalloc(sizeof(*ctx), GFP_KERNEL);
+ if (!ctx)
+ return NULL;
- ctx = kzalloc(sizeof(*ctx), GFP_KERNEL);
ctx->pid = pid;
ctx->cc = 0;
hash_add(m->pid_ctx, &ctx->h, pid);
-end:
return ctx;
}
-static void vidtv_mux_pid_ctx_init(struct vidtv_mux *m)
+static void vidtv_mux_pid_ctx_destroy(struct vidtv_mux *m)
+{
+ struct vidtv_mux_pid_ctx *ctx;
+ struct hlist_node *tmp;
+ int bkt;
+
+ hash_for_each_safe(m->pid_ctx, bkt, tmp, ctx, h) {
+ hash_del(&ctx->h);
+ kfree(ctx);
+ }
+}
+
+static int vidtv_mux_pid_ctx_init(struct vidtv_mux *m)
{
struct vidtv_psi_table_pat_program *p = m->si.pat->program;
u16 pid;
hash_init(m->pid_ctx);
/* push the pcr pid ctx */
- vidtv_mux_create_pid_ctx_once(m, m->pcr_pid);
- /* push the null packet pid ctx */
- vidtv_mux_create_pid_ctx_once(m, TS_NULL_PACKET_PID);
+ if (!vidtv_mux_create_pid_ctx_once(m, m->pcr_pid))
+ return -ENOMEM;
+ /* push the NULL packet pid ctx */
+ if (!vidtv_mux_create_pid_ctx_once(m, TS_NULL_PACKET_PID))
+ goto free;
/* push the PAT pid ctx */
- vidtv_mux_create_pid_ctx_once(m, VIDTV_PAT_PID);
+ if (!vidtv_mux_create_pid_ctx_once(m, VIDTV_PAT_PID))
+ goto free;
/* push the SDT pid ctx */
- vidtv_mux_create_pid_ctx_once(m, VIDTV_SDT_PID);
+ if (!vidtv_mux_create_pid_ctx_once(m, VIDTV_SDT_PID))
+ goto free;
+ /* push the NIT pid ctx */
+ if (!vidtv_mux_create_pid_ctx_once(m, VIDTV_NIT_PID))
+ goto free;
+ /* push the EIT pid ctx */
+ if (!vidtv_mux_create_pid_ctx_once(m, VIDTV_EIT_PID))
+ goto free;
/* add a ctx for all PMT sections */
while (p) {
vidtv_mux_create_pid_ctx_once(m, pid);
p = p->next;
}
-}
-static void vidtv_mux_pid_ctx_destroy(struct vidtv_mux *m)
-{
- int bkt;
- struct vidtv_mux_pid_ctx *ctx;
- struct hlist_node *tmp;
+ return 0;
- hash_for_each_safe(m->pid_ctx, bkt, tmp, ctx, h) {
- hash_del(&ctx->h);
- kfree(ctx);
- }
+free:
+ vidtv_mux_pid_ctx_destroy(m);
+ return -ENOMEM;
}
static void vidtv_mux_update_clk(struct vidtv_mux *m)
static u32 vidtv_mux_push_si(struct vidtv_mux *m)
{
+ struct vidtv_psi_pat_write_args pat_args = {
+ .buf = m->mux_buf,
+ .buf_sz = m->mux_buf_sz,
+ .pat = m->si.pat,
+ };
+ struct vidtv_psi_pmt_write_args pmt_args = {
+ .buf = m->mux_buf,
+ .buf_sz = m->mux_buf_sz,
+ .pcr_pid = m->pcr_pid,
+ };
+ struct vidtv_psi_sdt_write_args sdt_args = {
+ .buf = m->mux_buf,
+ .buf_sz = m->mux_buf_sz,
+ .sdt = m->si.sdt,
+ };
+ struct vidtv_psi_nit_write_args nit_args = {
+ .buf = m->mux_buf,
+ .buf_sz = m->mux_buf_sz,
+ .nit = m->si.nit,
+
+ };
+ struct vidtv_psi_eit_write_args eit_args = {
+ .buf = m->mux_buf,
+ .buf_sz = m->mux_buf_sz,
+ .eit = m->si.eit,
+ };
u32 initial_offset = m->mux_buf_offset;
-
struct vidtv_mux_pid_ctx *pat_ctx;
struct vidtv_mux_pid_ctx *pmt_ctx;
struct vidtv_mux_pid_ctx *sdt_ctx;
-
- struct vidtv_psi_pat_write_args pat_args = {};
- struct vidtv_psi_pmt_write_args pmt_args = {};
- struct vidtv_psi_sdt_write_args sdt_args = {};
-
- u32 nbytes; /* the number of bytes written by this function */
+ struct vidtv_mux_pid_ctx *nit_ctx;
+ struct vidtv_mux_pid_ctx *eit_ctx;
+ u32 nbytes;
u16 pmt_pid;
u32 i;
pat_ctx = vidtv_mux_get_pid_ctx(m, VIDTV_PAT_PID);
sdt_ctx = vidtv_mux_get_pid_ctx(m, VIDTV_SDT_PID);
+ nit_ctx = vidtv_mux_get_pid_ctx(m, VIDTV_NIT_PID);
+ eit_ctx = vidtv_mux_get_pid_ctx(m, VIDTV_EIT_PID);
- pat_args.buf = m->mux_buf;
pat_args.offset = m->mux_buf_offset;
- pat_args.pat = m->si.pat;
- pat_args.buf_sz = m->mux_buf_sz;
pat_args.continuity_counter = &pat_ctx->cc;
- m->mux_buf_offset += vidtv_psi_pat_write_into(pat_args);
+ m->mux_buf_offset += vidtv_psi_pat_write_into(&pat_args);
- for (i = 0; i < m->si.pat->programs; ++i) {
+ for (i = 0; i < m->si.pat->num_pmt; ++i) {
pmt_pid = vidtv_psi_pmt_get_pid(m->si.pmt_secs[i],
m->si.pat);
pmt_ctx = vidtv_mux_get_pid_ctx(m, pmt_pid);
- pmt_args.buf = m->mux_buf;
pmt_args.offset = m->mux_buf_offset;
pmt_args.pmt = m->si.pmt_secs[i];
pmt_args.pid = pmt_pid;
- pmt_args.buf_sz = m->mux_buf_sz;
pmt_args.continuity_counter = &pmt_ctx->cc;
- pmt_args.pcr_pid = m->pcr_pid;
/* write each section into buffer */
- m->mux_buf_offset += vidtv_psi_pmt_write_into(pmt_args);
+ m->mux_buf_offset += vidtv_psi_pmt_write_into(&pmt_args);
}
- sdt_args.buf = m->mux_buf;
sdt_args.offset = m->mux_buf_offset;
- sdt_args.sdt = m->si.sdt;
- sdt_args.buf_sz = m->mux_buf_sz;
sdt_args.continuity_counter = &sdt_ctx->cc;
- m->mux_buf_offset += vidtv_psi_sdt_write_into(sdt_args);
+ m->mux_buf_offset += vidtv_psi_sdt_write_into(&sdt_args);
+
+ nit_args.offset = m->mux_buf_offset;
+ nit_args.continuity_counter = &nit_ctx->cc;
+
+ m->mux_buf_offset += vidtv_psi_nit_write_into(&nit_args);
+
+ eit_args.offset = m->mux_buf_offset;
+ eit_args.continuity_counter = &eit_ctx->cc;
+
+ m->mux_buf_offset += vidtv_psi_eit_write_into(&eit_args);
nbytes = m->mux_buf_offset - initial_offset;
static u32 vidtv_mux_packetize_access_units(struct vidtv_mux *m,
struct vidtv_encoder *e)
{
- u32 nbytes = 0;
-
- struct pes_write_args args = {};
- u32 initial_offset = m->mux_buf_offset;
+ struct pes_write_args args = {
+ .dest_buf = m->mux_buf,
+ .dest_buf_sz = m->mux_buf_sz,
+ .pid = be16_to_cpu(e->es_pid),
+ .encoder_id = e->id,
+ .stream_id = be16_to_cpu(e->stream_id),
+ .send_pts = true, /* forbidden value '01'... */
+ .send_dts = false, /* ...for PTS_DTS flags */
+ };
struct vidtv_access_unit *au = e->access_units;
-
+ u32 initial_offset = m->mux_buf_offset;
+ struct vidtv_mux_pid_ctx *pid_ctx;
+ u32 nbytes = 0;
u8 *buf = NULL;
- struct vidtv_mux_pid_ctx *pid_ctx = vidtv_mux_create_pid_ctx_once(m,
- be16_to_cpu(e->es_pid));
- args.dest_buf = m->mux_buf;
- args.dest_buf_sz = m->mux_buf_sz;
- args.pid = be16_to_cpu(e->es_pid);
- args.encoder_id = e->id;
+ /* see SMPTE 302M clause 6.4 */
+ if (args.encoder_id == S302M) {
+ args.send_dts = false;
+ args.send_pts = true;
+ }
+
+ pid_ctx = vidtv_mux_create_pid_ctx_once(m, be16_to_cpu(e->es_pid));
args.continuity_counter = &pid_ctx->cc;
- args.stream_id = be16_to_cpu(e->stream_id);
- args.send_pts = true;
while (au) {
buf = e->encoder_buf + au->offset;
args.pts = au->pts;
args.pcr = m->timing.clk;
- m->mux_buf_offset += vidtv_pes_write_into(args);
+ m->mux_buf_offset += vidtv_pes_write_into(&args);
au = au->next;
}
static u32 vidtv_mux_poll_encoders(struct vidtv_mux *m)
{
- u32 nbytes = 0;
- u32 au_nbytes;
struct vidtv_channel *cur_chnl = m->channels;
struct vidtv_encoder *e = NULL;
+ u32 nbytes = 0;
+ u32 au_nbytes;
while (cur_chnl) {
e = cur_chnl->encoders;
static u32 vidtv_mux_pad_with_nulls(struct vidtv_mux *m, u32 npkts)
{
- struct null_packet_write_args args = {};
+ struct null_packet_write_args args = {
+ .dest_buf = m->mux_buf,
+ .buf_sz = m->mux_buf_sz,
+ .dest_offset = m->mux_buf_offset,
+ };
u32 initial_offset = m->mux_buf_offset;
- u32 nbytes; /* the number of bytes written by this function */
- u32 i;
struct vidtv_mux_pid_ctx *ctx;
+ u32 nbytes;
+ u32 i;
ctx = vidtv_mux_get_pid_ctx(m, TS_NULL_PACKET_PID);
- args.dest_buf = m->mux_buf;
- args.buf_sz = m->mux_buf_sz;
args.continuity_counter = &ctx->cc;
- args.dest_offset = m->mux_buf_offset;
for (i = 0; i < npkts; ++i) {
m->mux_buf_offset += vidtv_ts_null_write_into(args);
struct vidtv_mux,
mpeg_thread);
struct dtv_frontend_properties *c = &m->fe->dtv_property_cache;
+ u32 tot_bits = 0;
u32 nbytes;
u32 npkts;
- u32 tot_bits = 0;
while (m->streaming) {
nbytes = 0;
struct vidtv_mux *vidtv_mux_init(struct dvb_frontend *fe,
struct device *dev,
- struct vidtv_mux_init_args args)
+ struct vidtv_mux_init_args *args)
{
- struct vidtv_mux *m = kzalloc(sizeof(*m), GFP_KERNEL);
+ struct vidtv_mux *m;
+
+ m = kzalloc(sizeof(*m), GFP_KERNEL);
+ if (!m)
+ return NULL;
m->dev = dev;
m->fe = fe;
- m->timing.pcr_period_usecs = args.pcr_period_usecs;
- m->timing.si_period_usecs = args.si_period_usecs;
+ m->timing.pcr_period_usecs = args->pcr_period_usecs;
+ m->timing.si_period_usecs = args->si_period_usecs;
+
+ m->mux_rate_kbytes_sec = args->mux_rate_kbytes_sec;
- m->mux_rate_kbytes_sec = args.mux_rate_kbytes_sec;
+ m->on_new_packets_available_cb = args->on_new_packets_available_cb;
- m->on_new_packets_available_cb = args.on_new_packets_available_cb;
+ m->mux_buf = vzalloc(args->mux_buf_sz);
+ if (!m->mux_buf)
+ goto free_mux;
- m->mux_buf = vzalloc(args.mux_buf_sz);
- m->mux_buf_sz = args.mux_buf_sz;
+ m->mux_buf_sz = args->mux_buf_sz;
- m->pcr_pid = args.pcr_pid;
- m->transport_stream_id = args.transport_stream_id;
- m->priv = args.priv;
+ m->pcr_pid = args->pcr_pid;
+ m->transport_stream_id = args->transport_stream_id;
+ m->priv = args->priv;
+ m->network_id = args->network_id;
+ m->network_name = kstrdup(args->network_name, GFP_KERNEL);
m->timing.current_jiffies = get_jiffies_64();
- if (args.channels)
- m->channels = args.channels;
+ if (args->channels)
+ m->channels = args->channels;
else
- vidtv_channels_init(m);
+ if (vidtv_channels_init(m) < 0)
+ goto free_mux_buf;
/* will alloc data for pmt_sections after initializing pat */
- vidtv_channel_si_init(m);
+ if (vidtv_channel_si_init(m) < 0)
+ goto free_channels;
INIT_WORK(&m->mpeg_thread, vidtv_mux_tick);
- vidtv_mux_pid_ctx_init(m);
+ if (vidtv_mux_pid_ctx_init(m) < 0)
+ goto free_channel_si;
return m;
+
+free_channel_si:
+ vidtv_channel_si_destroy(m);
+free_channels:
+ vidtv_channels_destroy(m);
+free_mux_buf:
+ vfree(m->mux_buf);
+free_mux:
+ kfree(m);
+ return NULL;
}
void vidtv_mux_destroy(struct vidtv_mux *m)
vidtv_mux_pid_ctx_destroy(m);
vidtv_channel_si_destroy(m);
vidtv_channels_destroy(m);
+ kfree(m->network_name);
vfree(m->mux_buf);
kfree(m);
}
#ifndef VIDTV_MUX_H
#define VIDTV_MUX_H
-#include <linux/types.h>
#include <linux/hashtable.h>
+#include <linux/types.h>
#include <linux/workqueue.h>
+
#include <media/dvb_frontend.h>
#include "vidtv_psi.h"
* @pat: The PAT in use by the muxer.
* @pmt_secs: The PMT sections in use by the muxer. One for each program in the PAT.
* @sdt: The SDT in use by the muxer.
+ * @nit: The NIT in use by the muxer.
+ * @eit: the EIT in use by the muxer.
*/
struct vidtv_mux_si {
/* the SI tables */
struct vidtv_psi_table_pat *pat;
struct vidtv_psi_table_pmt **pmt_secs; /* the PMT sections */
struct vidtv_psi_table_sdt *sdt;
+ struct vidtv_psi_table_nit *nit;
+ struct vidtv_psi_table_eit *eit;
};
/**
/**
* struct vidtv_mux - A muxer abstraction loosely based in libavcodec/mpegtsenc.c
- * @mux_rate_kbytes_sec: The bit rate for the TS, in kbytes.
+ * @fe: The frontend structure allocated by the muxer.
+ * @dev: pointer to struct device.
* @timing: Keeps track of timing related information.
+ * @mux_rate_kbytes_sec: The bit rate for the TS, in kbytes.
* @pid_ctx: A hash table to keep track of per-PID metadata.
* @on_new_packets_available_cb: A callback to inform of new TS packets ready.
* @mux_buf: A pointer to a buffer for this muxer. TS packets are stored there
* @pcr_pid: The TS PID used for the PSI packets. All channels will share the
* same PCR.
* @transport_stream_id: The transport stream ID
+ * @network_id: The network ID
+ * @network_name: The network name
* @priv: Private data.
*/
struct vidtv_mux {
u16 pcr_pid;
u16 transport_stream_id;
+ u16 network_id;
+ char *network_name;
void *priv;
};
* same PCR.
* @transport_stream_id: The transport stream ID
* @channels: an optional list of channels to use
+ * @network_id: The network ID
+ * @network_name: The network name
* @priv: Private data.
*/
struct vidtv_mux_init_args {
u16 pcr_pid;
u16 transport_stream_id;
struct vidtv_channel *channels;
+ u16 network_id;
+ char *network_name;
void *priv;
};
struct vidtv_mux *vidtv_mux_init(struct dvb_frontend *fe,
struct device *dev,
- struct vidtv_mux_init_args args);
+ struct vidtv_mux_init_args *args);
void vidtv_mux_destroy(struct vidtv_mux *m);
void vidtv_mux_start_thread(struct vidtv_mux *m);
#include <linux/types.h>
#include <linux/printk.h>
#include <linux/ratelimit.h>
-#include <asm/byteorder.h>
#include "vidtv_pes.h"
#include "vidtv_common.h"
return len;
}
-static u32 vidtv_pes_write_header_stuffing(struct pes_header_write_args args)
+static u32 vidtv_pes_write_header_stuffing(struct pes_header_write_args *args)
{
/*
* This is a fixed 8-bit value equal to '0xFF' that can be inserted
* It is discarded by the decoder. No more than 32 stuffing bytes shall
* be present in one PES packet header.
*/
- if (args.n_pes_h_s_bytes > PES_HEADER_MAX_STUFFING_BYTES) {
+ if (args->n_pes_h_s_bytes > PES_HEADER_MAX_STUFFING_BYTES) {
pr_warn_ratelimited("More than %d stuffing bytes in PES packet header\n",
PES_HEADER_MAX_STUFFING_BYTES);
- args.n_pes_h_s_bytes = PES_HEADER_MAX_STUFFING_BYTES;
+ args->n_pes_h_s_bytes = PES_HEADER_MAX_STUFFING_BYTES;
}
- return vidtv_memset(args.dest_buf,
- args.dest_offset,
- args.dest_buf_sz,
+ return vidtv_memset(args->dest_buf,
+ args->dest_offset,
+ args->dest_buf_sz,
TS_FILL_BYTE,
- args.n_pes_h_s_bytes);
+ args->n_pes_h_s_bytes);
}
-static u32 vidtv_pes_write_pts_dts(struct pes_header_write_args args)
+static u32 vidtv_pes_write_pts_dts(struct pes_header_write_args *args)
{
u32 nbytes = 0; /* the number of bytes written by this function */
u64 mask2;
u64 mask3;
- if (!args.send_pts && args.send_dts)
+ if (!args->send_pts && args->send_dts)
return 0;
mask1 = GENMASK_ULL(32, 30);
mask3 = GENMASK_ULL(14, 0);
/* see ISO/IEC 13818-1 : 2000 p. 32 */
- if (args.send_pts && args.send_dts) {
- pts_dts.pts1 = (0x3 << 4) | ((args.pts & mask1) >> 29) | 0x1;
- pts_dts.pts2 = cpu_to_be16(((args.pts & mask2) >> 14) | 0x1);
- pts_dts.pts3 = cpu_to_be16(((args.pts & mask3) << 1) | 0x1);
+ if (args->send_pts && args->send_dts) {
+ pts_dts.pts1 = (0x3 << 4) | ((args->pts & mask1) >> 29) | 0x1;
+ pts_dts.pts2 = cpu_to_be16(((args->pts & mask2) >> 14) | 0x1);
+ pts_dts.pts3 = cpu_to_be16(((args->pts & mask3) << 1) | 0x1);
- pts_dts.dts1 = (0x1 << 4) | ((args.dts & mask1) >> 29) | 0x1;
- pts_dts.dts2 = cpu_to_be16(((args.dts & mask2) >> 14) | 0x1);
- pts_dts.dts3 = cpu_to_be16(((args.dts & mask3) << 1) | 0x1);
+ pts_dts.dts1 = (0x1 << 4) | ((args->dts & mask1) >> 29) | 0x1;
+ pts_dts.dts2 = cpu_to_be16(((args->dts & mask2) >> 14) | 0x1);
+ pts_dts.dts3 = cpu_to_be16(((args->dts & mask3) << 1) | 0x1);
op = &pts_dts;
op_sz = sizeof(pts_dts);
- } else if (args.send_pts) {
- pts.pts1 = (0x1 << 5) | ((args.pts & mask1) >> 29) | 0x1;
- pts.pts2 = cpu_to_be16(((args.pts & mask2) >> 14) | 0x1);
- pts.pts3 = cpu_to_be16(((args.pts & mask3) << 1) | 0x1);
+ } else if (args->send_pts) {
+ pts.pts1 = (0x1 << 5) | ((args->pts & mask1) >> 29) | 0x1;
+ pts.pts2 = cpu_to_be16(((args->pts & mask2) >> 14) | 0x1);
+ pts.pts3 = cpu_to_be16(((args->pts & mask3) << 1) | 0x1);
op = &pts;
op_sz = sizeof(pts);
}
/* copy PTS/DTS optional */
- nbytes += vidtv_memcpy(args.dest_buf,
- args.dest_offset + nbytes,
- args.dest_buf_sz,
+ nbytes += vidtv_memcpy(args->dest_buf,
+ args->dest_offset + nbytes,
+ args->dest_buf_sz,
op,
op_sz);
return nbytes;
}
-static u32 vidtv_pes_write_h(struct pes_header_write_args args)
+static u32 vidtv_pes_write_h(struct pes_header_write_args *args)
{
u32 nbytes = 0; /* the number of bytes written by this function */
struct vidtv_mpeg_pes pes_header = {};
struct vidtv_pes_optional pes_optional = {};
- struct pes_header_write_args pts_dts_args = args;
- u32 stream_id = (args.encoder_id == S302M) ? PRIVATE_STREAM_1_ID : args.stream_id;
+ struct pes_header_write_args pts_dts_args;
+ u32 stream_id = (args->encoder_id == S302M) ? PRIVATE_STREAM_1_ID : args->stream_id;
u16 pes_opt_bitfield = 0x01 << 15;
pes_header.bitfield = cpu_to_be32((PES_START_CODE_PREFIX << 8) | stream_id);
- pes_header.length = cpu_to_be16(vidtv_pes_op_get_len(args.send_pts,
- args.send_dts) +
- args.access_unit_len);
+ pes_header.length = cpu_to_be16(vidtv_pes_op_get_len(args->send_pts,
+ args->send_dts) +
+ args->access_unit_len);
- if (args.send_pts && args.send_dts)
+ if (args->send_pts && args->send_dts)
pes_opt_bitfield |= (0x3 << 6);
- else if (args.send_pts)
+ else if (args->send_pts)
pes_opt_bitfield |= (0x1 << 7);
pes_optional.bitfield = cpu_to_be16(pes_opt_bitfield);
- pes_optional.length = vidtv_pes_op_get_len(args.send_pts, args.send_dts) +
- args.n_pes_h_s_bytes -
+ pes_optional.length = vidtv_pes_op_get_len(args->send_pts, args->send_dts) +
+ args->n_pes_h_s_bytes -
sizeof(struct vidtv_pes_optional);
/* copy header */
- nbytes += vidtv_memcpy(args.dest_buf,
- args.dest_offset + nbytes,
- args.dest_buf_sz,
+ nbytes += vidtv_memcpy(args->dest_buf,
+ args->dest_offset + nbytes,
+ args->dest_buf_sz,
&pes_header,
sizeof(pes_header));
/* copy optional header bits */
- nbytes += vidtv_memcpy(args.dest_buf,
- args.dest_offset + nbytes,
- args.dest_buf_sz,
+ nbytes += vidtv_memcpy(args->dest_buf,
+ args->dest_offset + nbytes,
+ args->dest_buf_sz,
&pes_optional,
sizeof(pes_optional));
/* copy the timing information */
- pts_dts_args.dest_offset = args.dest_offset + nbytes;
- nbytes += vidtv_pes_write_pts_dts(pts_dts_args);
+ pts_dts_args = *args;
+ pts_dts_args.dest_offset = args->dest_offset + nbytes;
+ nbytes += vidtv_pes_write_pts_dts(&pts_dts_args);
/* write any PES header stuffing */
nbytes += vidtv_pes_write_header_stuffing(args);
return nbytes;
}
-u32 vidtv_pes_write_into(struct pes_write_args args)
+u32 vidtv_pes_write_into(struct pes_write_args *args)
{
- u32 unaligned_bytes = (args.dest_offset % TS_PACKET_LEN);
- struct pes_ts_header_write_args ts_header_args = {};
- struct pes_header_write_args pes_header_args = {};
- u32 remaining_len = args.access_unit_len;
+ u32 unaligned_bytes = (args->dest_offset % TS_PACKET_LEN);
+ struct pes_ts_header_write_args ts_header_args = {
+ .dest_buf = args->dest_buf,
+ .dest_buf_sz = args->dest_buf_sz,
+ .pid = args->pid,
+ .pcr = args->pcr,
+ .continuity_counter = args->continuity_counter,
+ };
+ struct pes_header_write_args pes_header_args = {
+ .dest_buf = args->dest_buf,
+ .dest_buf_sz = args->dest_buf_sz,
+ .encoder_id = args->encoder_id,
+ .send_pts = args->send_pts,
+ .pts = args->pts,
+ .send_dts = args->send_dts,
+ .dts = args->dts,
+ .stream_id = args->stream_id,
+ .n_pes_h_s_bytes = args->n_pes_h_s_bytes,
+ .access_unit_len = args->access_unit_len,
+ };
+ u32 remaining_len = args->access_unit_len;
bool wrote_pes_header = false;
- u64 last_pcr = args.pcr;
+ u64 last_pcr = args->pcr;
bool need_pcr = true;
u32 available_space;
u32 payload_size;
pr_warn_ratelimited("buffer is misaligned, while starting PES\n");
/* forcibly align and hope for the best */
- nbytes += vidtv_memset(args.dest_buf,
- args.dest_offset + nbytes,
- args.dest_buf_sz,
+ nbytes += vidtv_memset(args->dest_buf,
+ args->dest_offset + nbytes,
+ args->dest_buf_sz,
TS_FILL_BYTE,
TS_PACKET_LEN - unaligned_bytes);
}
- if (args.send_dts && !args.send_pts) {
- pr_warn_ratelimited("forbidden value '01' for PTS_DTS flags\n");
- args.send_pts = true;
- args.pts = args.dts;
- }
-
- /* see SMPTE 302M clause 6.4 */
- if (args.encoder_id == S302M) {
- args.send_dts = false;
- args.send_pts = true;
- }
-
while (remaining_len) {
available_space = TS_PAYLOAD_LEN;
/*
* the space needed for the TS header _and_ for the PES header
*/
if (!wrote_pes_header)
- available_space -= vidtv_pes_h_get_len(args.send_pts,
- args.send_dts);
+ available_space -= vidtv_pes_h_get_len(args->send_pts,
+ args->send_dts);
/*
* if the encoder has inserted stuffing bytes in the PES
* header, account for them.
*/
- available_space -= args.n_pes_h_s_bytes;
+ available_space -= args->n_pes_h_s_bytes;
/* Take the extra adaptation into account if need to send PCR */
if (need_pcr) {
}
/* write ts header */
- ts_header_args.dest_buf = args.dest_buf;
- ts_header_args.dest_offset = args.dest_offset + nbytes;
- ts_header_args.dest_buf_sz = args.dest_buf_sz;
- ts_header_args.pid = args.pid;
- ts_header_args.pcr = args.pcr;
- ts_header_args.continuity_counter = args.continuity_counter;
- ts_header_args.wrote_pes_header = wrote_pes_header;
- ts_header_args.n_stuffing_bytes = stuff_bytes;
+ ts_header_args.dest_offset = args->dest_offset + nbytes;
+ ts_header_args.wrote_pes_header = wrote_pes_header;
+ ts_header_args.n_stuffing_bytes = stuff_bytes;
nbytes += vidtv_pes_write_ts_h(ts_header_args, need_pcr,
&last_pcr);
if (!wrote_pes_header) {
/* write the PES header only once */
- pes_header_args.dest_buf = args.dest_buf;
-
- pes_header_args.dest_offset = args.dest_offset +
- nbytes;
-
- pes_header_args.dest_buf_sz = args.dest_buf_sz;
- pes_header_args.encoder_id = args.encoder_id;
- pes_header_args.send_pts = args.send_pts;
- pes_header_args.pts = args.pts;
- pes_header_args.send_dts = args.send_dts;
- pes_header_args.dts = args.dts;
- pes_header_args.stream_id = args.stream_id;
- pes_header_args.n_pes_h_s_bytes = args.n_pes_h_s_bytes;
- pes_header_args.access_unit_len = args.access_unit_len;
-
- nbytes += vidtv_pes_write_h(pes_header_args);
- wrote_pes_header = true;
+ pes_header_args.dest_offset = args->dest_offset +
+ nbytes;
+ nbytes += vidtv_pes_write_h(&pes_header_args);
+ wrote_pes_header = true;
}
/* write as much of the payload as we possibly can */
- nbytes += vidtv_memcpy(args.dest_buf,
- args.dest_offset + nbytes,
- args.dest_buf_sz,
- args.from,
+ nbytes += vidtv_memcpy(args->dest_buf,
+ args->dest_offset + nbytes,
+ args->dest_buf_sz,
+ args->from,
payload_size);
- args.from += payload_size;
+ args->from += payload_size;
remaining_len -= payload_size;
}
#ifndef VIDTV_PES_H
#define VIDTV_PES_H
-#include <asm/byteorder.h>
#include <linux/types.h>
#include "vidtv_common.h"
* @dest_buf_sz: The size of the dest_buffer
* @pid: The PID to use for the TS packets.
* @continuity_counter: Incremented on every new TS packet.
- * @n_pes_h_s_bytes: Padding bytes. Might be used by an encoder if needed, gets
+ * @wrote_pes_header: Flag to indicate that the PES header was written
+ * @n_stuffing_bytes: Padding bytes. Might be used by an encoder if needed, gets
* discarded by the decoder.
+ * @pcr: counter driven by a 27Mhz clock.
*/
struct pes_ts_header_write_args {
void *dest_buf;
* @dts: DTS value to send.
* @n_pes_h_s_bytes: Padding bytes. Might be used by an encoder if needed, gets
* discarded by the decoder.
+ * @pcr: counter driven by a 27Mhz clock.
*/
struct pes_write_args {
void *dest_buf;
* equal to the size of the access unit, since we need space for PES headers, TS headers
* and padding bytes, if any.
*/
-u32 vidtv_pes_write_into(struct pes_write_args args);
+u32 vidtv_pes_write_into(struct pes_write_args *args);
#endif // VIDTV_PES_H
* technically be broken into one or more sections, we do not do this here,
* hence 'table' and 'section' are interchangeable for vidtv.
*
- * This code currently supports three tables: PAT, PMT and SDT. These are the
- * bare minimum to get userspace to recognize our MPEG transport stream. It can
- * be extended to support more PSI tables in the future.
- *
* Copyright (C) 2020 Daniel W. S. Almeida
*/
#define pr_fmt(fmt) KBUILD_MODNAME ":%s, %d: " fmt, __func__, __LINE__
-#include <linux/kernel.h>
-#include <linux/types.h>
-#include <linux/slab.h>
+#include <linux/bcd.h>
#include <linux/crc32.h>
-#include <linux/string.h>
+#include <linux/kernel.h>
+#include <linux/ktime.h>
#include <linux/printk.h>
#include <linux/ratelimit.h>
+#include <linux/slab.h>
#include <linux/string.h>
-#include <asm/byteorder.h>
+#include <linux/string.h>
+#include <linux/time.h>
+#include <linux/types.h>
-#include "vidtv_psi.h"
#include "vidtv_common.h"
+#include "vidtv_psi.h"
#include "vidtv_ts.h"
#define CRC_SIZE_IN_BYTES 4
#define MAX_VERSION_NUM 32
+#define INITIAL_CRC 0xffffffff
+#define ISO_LANGUAGE_CODE_LEN 3
static const u32 CRC_LUT[256] = {
/* from libdvbv5 */
0xbcb4666d, 0xb8757bda, 0xb5365d03, 0xb1f740b4
};
-static inline u32 dvb_crc32(u32 crc, u8 *data, u32 len)
+static u32 dvb_crc32(u32 crc, u8 *data, u32 len)
{
/* from libdvbv5 */
while (len--)
h->version++;
}
-static inline u16 vidtv_psi_sdt_serv_get_desc_loop_len(struct vidtv_psi_table_sdt_service *s)
-{
- u16 mask;
- u16 ret;
-
- mask = GENMASK(11, 0);
-
- ret = be16_to_cpu(s->bitfield) & mask;
- return ret;
-}
-
-static inline u16 vidtv_psi_pmt_stream_get_desc_loop_len(struct vidtv_psi_table_pmt_stream *s)
-{
- u16 mask;
- u16 ret;
-
- mask = GENMASK(9, 0);
-
- ret = be16_to_cpu(s->bitfield2) & mask;
- return ret;
-}
-
-static inline u16 vidtv_psi_pmt_get_desc_loop_len(struct vidtv_psi_table_pmt *p)
-{
- u16 mask;
- u16 ret;
-
- mask = GENMASK(9, 0);
-
- ret = be16_to_cpu(p->bitfield2) & mask;
- return ret;
-}
-
-static inline u16 vidtv_psi_get_sec_len(struct vidtv_psi_table_header *h)
+static u16 vidtv_psi_get_sec_len(struct vidtv_psi_table_header *h)
{
u16 mask;
u16 ret;
return ret;
}
-inline u16 vidtv_psi_get_pat_program_pid(struct vidtv_psi_table_pat_program *p)
+u16 vidtv_psi_get_pat_program_pid(struct vidtv_psi_table_pat_program *p)
{
u16 mask;
u16 ret;
return ret;
}
-inline u16 vidtv_psi_pmt_stream_get_elem_pid(struct vidtv_psi_table_pmt_stream *s)
+u16 vidtv_psi_pmt_stream_get_elem_pid(struct vidtv_psi_table_pmt_stream *s)
{
u16 mask;
u16 ret;
return ret;
}
-static inline void vidtv_psi_set_desc_loop_len(__be16 *bitfield, u16 new_len, u8 desc_len_nbits)
+static void vidtv_psi_set_desc_loop_len(__be16 *bitfield, u16 new_len,
+ u8 desc_len_nbits)
{
- u16 mask;
__be16 new;
+ u16 mask;
mask = GENMASK(15, desc_len_nbits);
h->bitfield = new;
}
-static u32 vidtv_psi_ts_psi_write_into(struct psi_write_args args)
+/*
+ * Packetize PSI sections into TS packets:
+ * push a TS header (4bytes) every 184 bytes
+ * manage the continuity_counter
+ * add stuffing (i.e. padding bytes) after the CRC
+ */
+static u32 vidtv_psi_ts_psi_write_into(struct psi_write_args *args)
{
- /*
- * Packetize PSI sections into TS packets:
- * push a TS header (4bytes) every 184 bytes
- * manage the continuity_counter
- * add stuffing (i.e. padding bytes) after the CRC
- */
-
- u32 nbytes_past_boundary = (args.dest_offset % TS_PACKET_LEN);
+ struct vidtv_mpeg_ts ts_header = {
+ .sync_byte = TS_SYNC_BYTE,
+ .bitfield = cpu_to_be16((args->new_psi_section << 14) | args->pid),
+ .scrambling = 0,
+ .payload = 1,
+ .adaptation_field = 0, /* no adaptation field */
+ };
+ u32 nbytes_past_boundary = (args->dest_offset % TS_PACKET_LEN);
bool aligned = (nbytes_past_boundary == 0);
- struct vidtv_mpeg_ts ts_header = {};
-
- /* number of bytes written by this function */
- u32 nbytes = 0;
- /* how much there is left to write */
- u32 remaining_len = args.len;
- /* how much we can be written in this packet */
+ u32 remaining_len = args->len;
u32 payload_write_len = 0;
- /* where we are in the source */
u32 payload_offset = 0;
+ u32 nbytes = 0;
- const u16 PAYLOAD_START = args.new_psi_section;
-
- if (!args.crc && !args.is_crc)
+ if (!args->crc && !args->is_crc)
pr_warn_ratelimited("Missing CRC for chunk\n");
- if (args.crc)
- *args.crc = dvb_crc32(*args.crc, args.from, args.len);
+ if (args->crc)
+ *args->crc = dvb_crc32(*args->crc, args->from, args->len);
- if (args.new_psi_section && !aligned) {
+ if (args->new_psi_section && !aligned) {
pr_warn_ratelimited("Cannot write a new PSI section in a misaligned buffer\n");
/* forcibly align and hope for the best */
- nbytes += vidtv_memset(args.dest_buf,
- args.dest_offset + nbytes,
- args.dest_buf_sz,
+ nbytes += vidtv_memset(args->dest_buf,
+ args->dest_offset + nbytes,
+ args->dest_buf_sz,
TS_FILL_BYTE,
TS_PACKET_LEN - nbytes_past_boundary);
}
while (remaining_len) {
- nbytes_past_boundary = (args.dest_offset + nbytes) % TS_PACKET_LEN;
+ nbytes_past_boundary = (args->dest_offset + nbytes) % TS_PACKET_LEN;
aligned = (nbytes_past_boundary == 0);
if (aligned) {
/* if at a packet boundary, write a new TS header */
- ts_header.sync_byte = TS_SYNC_BYTE;
- ts_header.bitfield = cpu_to_be16((PAYLOAD_START << 14) | args.pid);
- ts_header.scrambling = 0;
- ts_header.continuity_counter = *args.continuity_counter;
- ts_header.payload = 1;
- /* no adaptation field */
- ts_header.adaptation_field = 0;
-
- /* copy the header */
- nbytes += vidtv_memcpy(args.dest_buf,
- args.dest_offset + nbytes,
- args.dest_buf_sz,
+ ts_header.continuity_counter = *args->continuity_counter;
+
+ nbytes += vidtv_memcpy(args->dest_buf,
+ args->dest_offset + nbytes,
+ args->dest_buf_sz,
&ts_header,
sizeof(ts_header));
/*
* This will trigger a discontinuity if the buffer is full,
* effectively dropping the packet.
*/
- vidtv_ts_inc_cc(args.continuity_counter);
+ vidtv_ts_inc_cc(args->continuity_counter);
}
/* write the pointer_field in the first byte of the payload */
- if (args.new_psi_section)
- nbytes += vidtv_memset(args.dest_buf,
- args.dest_offset + nbytes,
- args.dest_buf_sz,
+ if (args->new_psi_section)
+ nbytes += vidtv_memset(args->dest_buf,
+ args->dest_offset + nbytes,
+ args->dest_buf_sz,
0x0,
1);
/* write as much of the payload as possible */
- nbytes_past_boundary = (args.dest_offset + nbytes) % TS_PACKET_LEN;
+ nbytes_past_boundary = (args->dest_offset + nbytes) % TS_PACKET_LEN;
payload_write_len = min(TS_PACKET_LEN - nbytes_past_boundary, remaining_len);
- nbytes += vidtv_memcpy(args.dest_buf,
- args.dest_offset + nbytes,
- args.dest_buf_sz,
- args.from + payload_offset,
+ nbytes += vidtv_memcpy(args->dest_buf,
+ args->dest_offset + nbytes,
+ args->dest_buf_sz,
+ args->from + payload_offset,
payload_write_len);
/* 'payload_write_len' written from a total of 'len' requested*/
* fill the rest of the packet if there is any remaining space unused
*/
- nbytes_past_boundary = (args.dest_offset + nbytes) % TS_PACKET_LEN;
+ nbytes_past_boundary = (args->dest_offset + nbytes) % TS_PACKET_LEN;
- if (args.is_crc)
- nbytes += vidtv_memset(args.dest_buf,
- args.dest_offset + nbytes,
- args.dest_buf_sz,
+ if (args->is_crc)
+ nbytes += vidtv_memset(args->dest_buf,
+ args->dest_offset + nbytes,
+ args->dest_buf_sz,
TS_FILL_BYTE,
TS_PACKET_LEN - nbytes_past_boundary);
return nbytes;
}
-static u32 table_section_crc32_write_into(struct crc32_write_args args)
+static u32 table_section_crc32_write_into(struct crc32_write_args *args)
{
+ struct psi_write_args psi_args = {
+ .dest_buf = args->dest_buf,
+ .from = &args->crc,
+ .len = CRC_SIZE_IN_BYTES,
+ .dest_offset = args->dest_offset,
+ .pid = args->pid,
+ .new_psi_section = false,
+ .continuity_counter = args->continuity_counter,
+ .is_crc = true,
+ .dest_buf_sz = args->dest_buf_sz,
+ };
+
/* the CRC is the last entry in the section */
- u32 nbytes = 0;
- struct psi_write_args psi_args = {};
- psi_args.dest_buf = args.dest_buf;
- psi_args.from = &args.crc;
- psi_args.len = CRC_SIZE_IN_BYTES;
- psi_args.dest_offset = args.dest_offset;
- psi_args.pid = args.pid;
- psi_args.new_psi_section = false;
- psi_args.continuity_counter = args.continuity_counter;
- psi_args.is_crc = true;
- psi_args.dest_buf_sz = args.dest_buf_sz;
+ return vidtv_psi_ts_psi_write_into(&psi_args);
+}
- nbytes += vidtv_psi_ts_psi_write_into(psi_args);
+static void vidtv_psi_desc_chain(struct vidtv_psi_desc *head, struct vidtv_psi_desc *desc)
+{
+ if (head) {
+ while (head->next)
+ head = head->next;
- return nbytes;
+ head->next = desc;
+ }
}
struct vidtv_psi_desc_service *vidtv_psi_service_desc_init(struct vidtv_psi_desc *head,
u32 provider_name_len = provider_name ? strlen(provider_name) : 0;
desc = kzalloc(sizeof(*desc), GFP_KERNEL);
+ if (!desc)
+ return NULL;
desc->type = SERVICE_DESCRIPTOR;
if (provider_name && provider_name_len)
desc->provider_name = kstrdup(provider_name, GFP_KERNEL);
- if (head) {
- while (head->next)
- head = head->next;
-
- head->next = (struct vidtv_psi_desc *)desc;
- }
+ vidtv_psi_desc_chain(head, (struct vidtv_psi_desc *)desc);
return desc;
}
struct vidtv_psi_desc_registration *desc;
desc = kzalloc(sizeof(*desc) + sizeof(format_id) + additional_info_len, GFP_KERNEL);
+ if (!desc)
+ return NULL;
desc->type = REGISTRATION_DESCRIPTOR;
additional_ident_info,
additional_info_len);
- if (head) {
- while (head->next)
- head = head->next;
+ vidtv_psi_desc_chain(head, (struct vidtv_psi_desc *)desc);
+ return desc;
+}
- head->next = (struct vidtv_psi_desc *)desc;
+struct vidtv_psi_desc_network_name
+*vidtv_psi_network_name_desc_init(struct vidtv_psi_desc *head, char *network_name)
+{
+ u32 network_name_len = network_name ? strlen(network_name) : 0;
+ struct vidtv_psi_desc_network_name *desc;
+
+ desc = kzalloc(sizeof(*desc), GFP_KERNEL);
+ if (!desc)
+ return NULL;
+
+ desc->type = NETWORK_NAME_DESCRIPTOR;
+
+ desc->length = network_name_len;
+
+ if (network_name && network_name_len)
+ desc->network_name = kstrdup(network_name, GFP_KERNEL);
+
+ vidtv_psi_desc_chain(head, (struct vidtv_psi_desc *)desc);
+ return desc;
+}
+
+struct vidtv_psi_desc_service_list
+*vidtv_psi_service_list_desc_init(struct vidtv_psi_desc *head,
+ struct vidtv_psi_desc_service_list_entry *entry)
+{
+ struct vidtv_psi_desc_service_list_entry *curr_e = NULL;
+ struct vidtv_psi_desc_service_list_entry *head_e = NULL;
+ struct vidtv_psi_desc_service_list_entry *prev_e = NULL;
+ struct vidtv_psi_desc_service_list *desc;
+ u16 length = 0;
+
+ desc = kzalloc(sizeof(*desc), GFP_KERNEL);
+ if (!desc)
+ return NULL;
+
+ desc->type = SERVICE_LIST_DESCRIPTOR;
+
+ while (entry) {
+ curr_e = kzalloc(sizeof(*curr_e), GFP_KERNEL);
+ if (!curr_e) {
+ while (head_e) {
+ curr_e = head_e;
+ head_e = head_e->next;
+ kfree(curr_e);
+ }
+ kfree(desc);
+ return NULL;
+ }
+
+ curr_e->service_id = entry->service_id;
+ curr_e->service_type = entry->service_type;
+
+ length += sizeof(struct vidtv_psi_desc_service_list_entry) -
+ sizeof(struct vidtv_psi_desc_service_list_entry *);
+
+ if (!head_e)
+ head_e = curr_e;
+ if (prev_e)
+ prev_e->next = curr_e;
+
+ prev_e = curr_e;
+ entry = entry->next;
}
+ desc->length = length;
+ desc->service_list = head_e;
+
+ vidtv_psi_desc_chain(head, (struct vidtv_psi_desc *)desc);
+ return desc;
+}
+
+struct vidtv_psi_desc_short_event
+*vidtv_psi_short_event_desc_init(struct vidtv_psi_desc *head,
+ char *iso_language_code,
+ char *event_name,
+ char *text)
+{
+ u32 iso_len = iso_language_code ? strlen(iso_language_code) : 0;
+ u32 event_name_len = event_name ? strlen(event_name) : 0;
+ struct vidtv_psi_desc_short_event *desc;
+ u32 text_len = text ? strlen(text) : 0;
+
+ desc = kzalloc(sizeof(*desc), GFP_KERNEL);
+ if (!desc)
+ return NULL;
+
+ desc->type = SHORT_EVENT_DESCRIPTOR;
+
+ desc->length = ISO_LANGUAGE_CODE_LEN +
+ sizeof_field(struct vidtv_psi_desc_short_event, event_name_len) +
+ event_name_len +
+ sizeof_field(struct vidtv_psi_desc_short_event, text_len) +
+ text_len;
+
+ desc->event_name_len = event_name_len;
+ desc->text_len = text_len;
+
+ if (iso_len != ISO_LANGUAGE_CODE_LEN)
+ iso_language_code = "eng";
+
+ desc->iso_language_code = kstrdup(iso_language_code, GFP_KERNEL);
+
+ if (event_name && event_name_len)
+ desc->event_name = kstrdup(event_name, GFP_KERNEL);
+
+ if (text && text_len)
+ desc->text = kstrdup(text, GFP_KERNEL);
+
+ vidtv_psi_desc_chain(head, (struct vidtv_psi_desc *)desc);
return desc;
}
struct vidtv_psi_desc *vidtv_psi_desc_clone(struct vidtv_psi_desc *desc)
{
+ struct vidtv_psi_desc_network_name *desc_network_name;
+ struct vidtv_psi_desc_service_list *desc_service_list;
+ struct vidtv_psi_desc_short_event *desc_short_event;
+ struct vidtv_psi_desc_service *service;
struct vidtv_psi_desc *head = NULL;
struct vidtv_psi_desc *prev = NULL;
struct vidtv_psi_desc *curr = NULL;
- struct vidtv_psi_desc_service *service;
-
while (desc) {
switch (desc->type) {
case SERVICE_DESCRIPTOR:
service = (struct vidtv_psi_desc_service *)desc;
curr = (struct vidtv_psi_desc *)
- vidtv_psi_service_desc_init(head,
- service->service_type,
- service->service_name,
- service->provider_name);
+ vidtv_psi_service_desc_init(head,
+ service->service_type,
+ service->service_name,
+ service->provider_name);
+ break;
+
+ case NETWORK_NAME_DESCRIPTOR:
+ desc_network_name = (struct vidtv_psi_desc_network_name *)desc;
+ curr = (struct vidtv_psi_desc *)
+ vidtv_psi_network_name_desc_init(head,
+ desc_network_name->network_name);
+ break;
+
+ case SERVICE_LIST_DESCRIPTOR:
+ desc_service_list = (struct vidtv_psi_desc_service_list *)desc;
+ curr = (struct vidtv_psi_desc *)
+ vidtv_psi_service_list_desc_init(head,
+ desc_service_list->service_list);
+ break;
+
+ case SHORT_EVENT_DESCRIPTOR:
+ desc_short_event = (struct vidtv_psi_desc_short_event *)desc;
+ curr = (struct vidtv_psi_desc *)
+ vidtv_psi_short_event_desc_init(head,
+ desc_short_event->iso_language_code,
+ desc_short_event->event_name,
+ desc_short_event->text);
break;
case REGISTRATION_DESCRIPTOR:
default:
curr = kzalloc(sizeof(*desc) + desc->length, GFP_KERNEL);
+ if (!curr)
+ return NULL;
memcpy(curr, desc, sizeof(*desc) + desc->length);
- break;
- }
+ }
- if (curr)
- curr->next = NULL;
+ if (!curr)
+ return NULL;
+
+ curr->next = NULL;
if (!head)
head = curr;
if (prev)
void vidtv_psi_desc_destroy(struct vidtv_psi_desc *desc)
{
+ struct vidtv_psi_desc_service_list_entry *sl_entry_tmp = NULL;
+ struct vidtv_psi_desc_service_list_entry *sl_entry = NULL;
struct vidtv_psi_desc *curr = desc;
struct vidtv_psi_desc *tmp = NULL;
/* nothing to do */
break;
+ case NETWORK_NAME_DESCRIPTOR:
+ kfree(((struct vidtv_psi_desc_network_name *)tmp)->network_name);
+ break;
+
+ case SERVICE_LIST_DESCRIPTOR:
+ sl_entry = ((struct vidtv_psi_desc_service_list *)tmp)->service_list;
+ while (sl_entry) {
+ sl_entry_tmp = sl_entry;
+ sl_entry = sl_entry->next;
+ kfree(sl_entry_tmp);
+ }
+ break;
+
+ case SHORT_EVENT_DESCRIPTOR:
+ kfree(((struct vidtv_psi_desc_short_event *)tmp)->iso_language_code);
+ kfree(((struct vidtv_psi_desc_short_event *)tmp)->event_name);
+ kfree(((struct vidtv_psi_desc_short_event *)tmp)->text);
+ break;
+
default:
pr_warn_ratelimited("Possible leak: not handling descriptor type %d\n",
tmp->type);
vidtv_psi_update_version_num(&sdt->header);
}
-static u32 vidtv_psi_desc_write_into(struct desc_write_args args)
+static u32 vidtv_psi_desc_write_into(struct desc_write_args *args)
{
- /* the number of bytes written by this function */
+ struct psi_write_args psi_args = {
+ .dest_buf = args->dest_buf,
+ .from = &args->desc->type,
+ .pid = args->pid,
+ .new_psi_section = false,
+ .continuity_counter = args->continuity_counter,
+ .is_crc = false,
+ .dest_buf_sz = args->dest_buf_sz,
+ .crc = args->crc,
+ .len = sizeof_field(struct vidtv_psi_desc, type) +
+ sizeof_field(struct vidtv_psi_desc, length),
+ };
+ struct vidtv_psi_desc_service_list_entry *serv_list_entry = NULL;
u32 nbytes = 0;
- struct psi_write_args psi_args = {};
-
- psi_args.dest_buf = args.dest_buf;
- psi_args.from = &args.desc->type;
- psi_args.len = sizeof_field(struct vidtv_psi_desc, type) +
- sizeof_field(struct vidtv_psi_desc, length);
+ psi_args.dest_offset = args->dest_offset + nbytes;
- psi_args.dest_offset = args.dest_offset + nbytes;
- psi_args.pid = args.pid;
- psi_args.new_psi_section = false;
- psi_args.continuity_counter = args.continuity_counter;
- psi_args.is_crc = false;
- psi_args.dest_buf_sz = args.dest_buf_sz;
- psi_args.crc = args.crc;
+ nbytes += vidtv_psi_ts_psi_write_into(&psi_args);
- nbytes += vidtv_psi_ts_psi_write_into(psi_args);
-
- switch (args.desc->type) {
+ switch (args->desc->type) {
case SERVICE_DESCRIPTOR:
- psi_args.dest_offset = args.dest_offset + nbytes;
+ psi_args.dest_offset = args->dest_offset + nbytes;
psi_args.len = sizeof_field(struct vidtv_psi_desc_service, service_type) +
sizeof_field(struct vidtv_psi_desc_service, provider_name_len);
- psi_args.from = &((struct vidtv_psi_desc_service *)args.desc)->service_type;
+ psi_args.from = &((struct vidtv_psi_desc_service *)args->desc)->service_type;
- nbytes += vidtv_psi_ts_psi_write_into(psi_args);
+ nbytes += vidtv_psi_ts_psi_write_into(&psi_args);
- psi_args.dest_offset = args.dest_offset + nbytes;
- psi_args.len = ((struct vidtv_psi_desc_service *)args.desc)->provider_name_len;
- psi_args.from = ((struct vidtv_psi_desc_service *)args.desc)->provider_name;
+ psi_args.dest_offset = args->dest_offset + nbytes;
+ psi_args.len = ((struct vidtv_psi_desc_service *)args->desc)->provider_name_len;
+ psi_args.from = ((struct vidtv_psi_desc_service *)args->desc)->provider_name;
- nbytes += vidtv_psi_ts_psi_write_into(psi_args);
+ nbytes += vidtv_psi_ts_psi_write_into(&psi_args);
- psi_args.dest_offset = args.dest_offset + nbytes;
+ psi_args.dest_offset = args->dest_offset + nbytes;
psi_args.len = sizeof_field(struct vidtv_psi_desc_service, service_name_len);
- psi_args.from = &((struct vidtv_psi_desc_service *)args.desc)->service_name_len;
+ psi_args.from = &((struct vidtv_psi_desc_service *)args->desc)->service_name_len;
- nbytes += vidtv_psi_ts_psi_write_into(psi_args);
+ nbytes += vidtv_psi_ts_psi_write_into(&psi_args);
- psi_args.dest_offset = args.dest_offset + nbytes;
- psi_args.len = ((struct vidtv_psi_desc_service *)args.desc)->service_name_len;
- psi_args.from = ((struct vidtv_psi_desc_service *)args.desc)->service_name;
+ psi_args.dest_offset = args->dest_offset + nbytes;
+ psi_args.len = ((struct vidtv_psi_desc_service *)args->desc)->service_name_len;
+ psi_args.from = ((struct vidtv_psi_desc_service *)args->desc)->service_name;
+
+ nbytes += vidtv_psi_ts_psi_write_into(&psi_args);
+ break;
+
+ case NETWORK_NAME_DESCRIPTOR:
+ psi_args.dest_offset = args->dest_offset + nbytes;
+ psi_args.len = args->desc->length;
+ psi_args.from = ((struct vidtv_psi_desc_network_name *)args->desc)->network_name;
+
+ nbytes += vidtv_psi_ts_psi_write_into(&psi_args);
+ break;
+
+ case SERVICE_LIST_DESCRIPTOR:
+ serv_list_entry = ((struct vidtv_psi_desc_service_list *)args->desc)->service_list;
+ while (serv_list_entry) {
+ psi_args.dest_offset = args->dest_offset + nbytes;
+ psi_args.len = sizeof(struct vidtv_psi_desc_service_list_entry) -
+ sizeof(struct vidtv_psi_desc_service_list_entry *);
+ psi_args.from = serv_list_entry;
+
+ nbytes += vidtv_psi_ts_psi_write_into(&psi_args);
+
+ serv_list_entry = serv_list_entry->next;
+ }
+ break;
+
+ case SHORT_EVENT_DESCRIPTOR:
+ psi_args.dest_offset = args->dest_offset + nbytes;
+ psi_args.len = ISO_LANGUAGE_CODE_LEN;
+ psi_args.from = ((struct vidtv_psi_desc_short_event *)
+ args->desc)->iso_language_code;
+
+ nbytes += vidtv_psi_ts_psi_write_into(&psi_args);
+
+ psi_args.dest_offset = args->dest_offset + nbytes;
+ psi_args.len = sizeof_field(struct vidtv_psi_desc_short_event, event_name_len);
+ psi_args.from = &((struct vidtv_psi_desc_short_event *)
+ args->desc)->event_name_len;
+
+ nbytes += vidtv_psi_ts_psi_write_into(&psi_args);
+
+ psi_args.dest_offset = args->dest_offset + nbytes;
+ psi_args.len = ((struct vidtv_psi_desc_short_event *)args->desc)->event_name_len;
+ psi_args.from = ((struct vidtv_psi_desc_short_event *)args->desc)->event_name;
+
+ nbytes += vidtv_psi_ts_psi_write_into(&psi_args);
+
+ psi_args.dest_offset = args->dest_offset + nbytes;
+ psi_args.len = sizeof_field(struct vidtv_psi_desc_short_event, text_len);
+ psi_args.from = &((struct vidtv_psi_desc_short_event *)args->desc)->text_len;
+
+ nbytes += vidtv_psi_ts_psi_write_into(&psi_args);
+
+ psi_args.dest_offset = args->dest_offset + nbytes;
+ psi_args.len = ((struct vidtv_psi_desc_short_event *)args->desc)->text_len;
+ psi_args.from = ((struct vidtv_psi_desc_short_event *)args->desc)->text;
+
+ nbytes += vidtv_psi_ts_psi_write_into(&psi_args);
- nbytes += vidtv_psi_ts_psi_write_into(psi_args);
break;
case REGISTRATION_DESCRIPTOR:
default:
- psi_args.dest_offset = args.dest_offset + nbytes;
- psi_args.len = args.desc->length;
- psi_args.from = &args.desc->data;
+ psi_args.dest_offset = args->dest_offset + nbytes;
+ psi_args.len = args->desc->length;
+ psi_args.from = &args->desc->data;
- nbytes += vidtv_psi_ts_psi_write_into(psi_args);
+ nbytes += vidtv_psi_ts_psi_write_into(&psi_args);
break;
}
}
static u32
-vidtv_psi_table_header_write_into(struct header_write_args args)
+vidtv_psi_table_header_write_into(struct header_write_args *args)
{
- /* the number of bytes written by this function */
- u32 nbytes = 0;
- struct psi_write_args psi_args = {};
-
- psi_args.dest_buf = args.dest_buf;
- psi_args.from = args.h;
- psi_args.len = sizeof(struct vidtv_psi_table_header);
- psi_args.dest_offset = args.dest_offset;
- psi_args.pid = args.pid;
- psi_args.new_psi_section = true;
- psi_args.continuity_counter = args.continuity_counter;
- psi_args.is_crc = false;
- psi_args.dest_buf_sz = args.dest_buf_sz;
- psi_args.crc = args.crc;
-
- nbytes += vidtv_psi_ts_psi_write_into(psi_args);
-
- return nbytes;
+ struct psi_write_args psi_args = {
+ .dest_buf = args->dest_buf,
+ .from = args->h,
+ .len = sizeof(struct vidtv_psi_table_header),
+ .dest_offset = args->dest_offset,
+ .pid = args->pid,
+ .new_psi_section = true,
+ .continuity_counter = args->continuity_counter,
+ .is_crc = false,
+ .dest_buf_sz = args->dest_buf_sz,
+ .crc = args->crc,
+ };
+
+ return vidtv_psi_ts_psi_write_into(&psi_args);
}
void
vidtv_psi_pat_table_update_sec_len(struct vidtv_psi_table_pat *pat)
{
- /* see ISO/IEC 13818-1 : 2000 p.43 */
u16 length = 0;
u32 i;
+ /* see ISO/IEC 13818-1 : 2000 p.43 */
+
/* from immediately after 'section_length' until 'last_section_number'*/
length += PAT_LEN_UNTIL_LAST_SECTION_NUMBER;
/* do not count the pointer */
- for (i = 0; i < pat->programs; ++i)
+ for (i = 0; i < pat->num_pat; ++i)
length += sizeof(struct vidtv_psi_table_pat_program) -
sizeof(struct vidtv_psi_table_pat_program *);
void vidtv_psi_pmt_table_update_sec_len(struct vidtv_psi_table_pmt *pmt)
{
- /* see ISO/IEC 13818-1 : 2000 p.46 */
- u16 length = 0;
struct vidtv_psi_table_pmt_stream *s = pmt->stream;
u16 desc_loop_len;
+ u16 length = 0;
+
+ /* see ISO/IEC 13818-1 : 2000 p.46 */
/* from immediately after 'section_length' until 'program_info_length'*/
length += PMT_LEN_UNTIL_PROGRAM_INFO_LENGTH;
void vidtv_psi_sdt_table_update_sec_len(struct vidtv_psi_table_sdt *sdt)
{
- /* see ETSI EN 300 468 V 1.10.1 p.24 */
- u16 length = 0;
struct vidtv_psi_table_sdt_service *s = sdt->service;
u16 desc_loop_len;
+ u16 length = 0;
+
+ /* see ETSI EN 300 468 V 1.10.1 p.24 */
/*
* from immediately after 'section_length' until
}
length += CRC_SIZE_IN_BYTES;
-
vidtv_psi_set_sec_len(&sdt->header, length);
}
const u16 RESERVED = 0x07;
program = kzalloc(sizeof(*program), GFP_KERNEL);
+ if (!program)
+ return NULL;
program->service_id = cpu_to_be16(service_id);
void
vidtv_psi_pat_program_destroy(struct vidtv_psi_table_pat_program *p)
{
- struct vidtv_psi_table_pat_program *curr = p;
struct vidtv_psi_table_pat_program *tmp = NULL;
+ struct vidtv_psi_table_pat_program *curr = p;
while (curr) {
tmp = curr;
}
}
+/* This function transfers ownership of p to the table */
void
vidtv_psi_pat_program_assign(struct vidtv_psi_table_pat *pat,
struct vidtv_psi_table_pat_program *p)
{
- /* This function transfers ownership of p to the table */
+ struct vidtv_psi_table_pat_program *program;
+ u16 program_count;
- u16 program_count = 0;
- struct vidtv_psi_table_pat_program *program = p;
+ do {
+ program_count = 0;
+ program = p;
- if (p == pat->program)
- return;
+ if (p == pat->program)
+ return;
- while (program) {
- ++program_count;
- program = program->next;
- }
+ while (program) {
+ ++program_count;
+ program = program->next;
+ }
- pat->programs = program_count;
- pat->program = p;
+ pat->num_pat = program_count;
+ pat->program = p;
- /* Recompute section length */
- vidtv_psi_pat_table_update_sec_len(pat);
+ /* Recompute section length */
+ vidtv_psi_pat_table_update_sec_len(pat);
- if (vidtv_psi_get_sec_len(&pat->header) > MAX_SECTION_LEN)
- vidtv_psi_pat_program_assign(pat, NULL);
+ p = NULL;
+ } while (vidtv_psi_get_sec_len(&pat->header) > MAX_SECTION_LEN);
vidtv_psi_update_version_num(&pat->header);
}
struct vidtv_psi_table_pat *vidtv_psi_pat_table_init(u16 transport_stream_id)
{
- struct vidtv_psi_table_pat *pat = kzalloc(sizeof(*pat), GFP_KERNEL);
+ struct vidtv_psi_table_pat *pat;
const u16 SYNTAX = 0x1;
const u16 ZERO = 0x0;
const u16 ONES = 0x03;
+ pat = kzalloc(sizeof(*pat), GFP_KERNEL);
+ if (!pat)
+ return NULL;
+
pat->header.table_id = 0x0;
pat->header.bitfield = cpu_to_be16((SYNTAX << 15) | (ZERO << 14) | (ONES << 12));
pat->header.section_id = 0x0;
pat->header.last_section = 0x0;
- pat->programs = 0;
-
vidtv_psi_pat_table_update_sec_len(pat);
return pat;
}
-u32 vidtv_psi_pat_write_into(struct vidtv_psi_pat_write_args args)
+u32 vidtv_psi_pat_write_into(struct vidtv_psi_pat_write_args *args)
{
- /* the number of bytes written by this function */
+ struct vidtv_psi_table_pat_program *p = args->pat->program;
+ struct header_write_args h_args = {
+ .dest_buf = args->buf,
+ .dest_offset = args->offset,
+ .pid = VIDTV_PAT_PID,
+ .h = &args->pat->header,
+ .continuity_counter = args->continuity_counter,
+ .dest_buf_sz = args->buf_sz,
+ };
+ struct psi_write_args psi_args = {
+ .dest_buf = args->buf,
+ .pid = VIDTV_PAT_PID,
+ .new_psi_section = false,
+ .continuity_counter = args->continuity_counter,
+ .is_crc = false,
+ .dest_buf_sz = args->buf_sz,
+ };
+ struct crc32_write_args c_args = {
+ .dest_buf = args->buf,
+ .pid = VIDTV_PAT_PID,
+ .dest_buf_sz = args->buf_sz,
+ };
+ u32 crc = INITIAL_CRC;
u32 nbytes = 0;
- const u16 pat_pid = VIDTV_PAT_PID;
- u32 crc = 0xffffffff;
-
- struct vidtv_psi_table_pat_program *p = args.pat->program;
- struct header_write_args h_args = {};
- struct psi_write_args psi_args = {};
- struct crc32_write_args c_args = {};
+ vidtv_psi_pat_table_update_sec_len(args->pat);
- vidtv_psi_pat_table_update_sec_len(args.pat);
-
- h_args.dest_buf = args.buf;
- h_args.dest_offset = args.offset;
- h_args.h = &args.pat->header;
- h_args.pid = pat_pid;
- h_args.continuity_counter = args.continuity_counter;
- h_args.dest_buf_sz = args.buf_sz;
h_args.crc = &crc;
- nbytes += vidtv_psi_table_header_write_into(h_args);
+ nbytes += vidtv_psi_table_header_write_into(&h_args);
/* note that the field 'u16 programs' is not really part of the PAT */
- psi_args.dest_buf = args.buf;
- psi_args.pid = pat_pid;
- psi_args.new_psi_section = false;
- psi_args.continuity_counter = args.continuity_counter;
- psi_args.is_crc = false;
- psi_args.dest_buf_sz = args.buf_sz;
- psi_args.crc = &crc;
+ psi_args.crc = &crc;
while (p) {
/* copy the PAT programs */
psi_args.from = p;
/* skip the pointer */
psi_args.len = sizeof(*p) -
- sizeof(struct vidtv_psi_table_pat_program *);
- psi_args.dest_offset = args.offset + nbytes;
+ sizeof(struct vidtv_psi_table_pat_program *);
+ psi_args.dest_offset = args->offset + nbytes;
+ psi_args.continuity_counter = args->continuity_counter;
- nbytes += vidtv_psi_ts_psi_write_into(psi_args);
+ nbytes += vidtv_psi_ts_psi_write_into(&psi_args);
p = p->next;
}
- c_args.dest_buf = args.buf;
- c_args.dest_offset = args.offset + nbytes;
+ c_args.dest_offset = args->offset + nbytes;
+ c_args.continuity_counter = args->continuity_counter;
c_args.crc = cpu_to_be32(crc);
- c_args.pid = pat_pid;
- c_args.continuity_counter = args.continuity_counter;
- c_args.dest_buf_sz = args.buf_sz;
/* Write the CRC32 at the end */
- nbytes += table_section_crc32_write_into(c_args);
+ nbytes += table_section_crc32_write_into(&c_args);
return nbytes;
}
u16 desc_loop_len;
stream = kzalloc(sizeof(*stream), GFP_KERNEL);
+ if (!stream)
+ return NULL;
stream->type = stream_type;
void vidtv_psi_pmt_stream_destroy(struct vidtv_psi_table_pmt_stream *s)
{
- struct vidtv_psi_table_pmt_stream *curr_stream = s;
struct vidtv_psi_table_pmt_stream *tmp_stream = NULL;
+ struct vidtv_psi_table_pmt_stream *curr_stream = s;
while (curr_stream) {
tmp_stream = curr_stream;
void vidtv_psi_pmt_stream_assign(struct vidtv_psi_table_pmt *pmt,
struct vidtv_psi_table_pmt_stream *s)
{
- /* This function transfers ownership of s to the table */
- if (s == pmt->stream)
- return;
+ do {
+ /* This function transfers ownership of s to the table */
+ if (s == pmt->stream)
+ return;
- pmt->stream = s;
- vidtv_psi_pmt_table_update_sec_len(pmt);
+ pmt->stream = s;
+ vidtv_psi_pmt_table_update_sec_len(pmt);
- if (vidtv_psi_get_sec_len(&pmt->header) > MAX_SECTION_LEN)
- vidtv_psi_pmt_stream_assign(pmt, NULL);
+ s = NULL;
+ } while (vidtv_psi_get_sec_len(&pmt->header) > MAX_SECTION_LEN);
vidtv_psi_update_version_num(&pmt->header);
}
struct vidtv_psi_table_pmt *vidtv_psi_pmt_table_init(u16 program_number,
u16 pcr_pid)
{
- struct vidtv_psi_table_pmt *pmt = kzalloc(sizeof(*pmt), GFP_KERNEL);
- const u16 SYNTAX = 0x1;
- const u16 ZERO = 0x0;
- const u16 ONES = 0x03;
+ struct vidtv_psi_table_pmt *pmt;
const u16 RESERVED1 = 0x07;
const u16 RESERVED2 = 0x0f;
+ const u16 SYNTAX = 0x1;
+ const u16 ONES = 0x03;
+ const u16 ZERO = 0x0;
u16 desc_loop_len;
+ pmt = kzalloc(sizeof(*pmt), GFP_KERNEL);
+ if (!pmt)
+ return NULL;
+
if (!pcr_pid)
pcr_pid = 0x1fff;
return pmt;
}
-u32 vidtv_psi_pmt_write_into(struct vidtv_psi_pmt_write_args args)
+u32 vidtv_psi_pmt_write_into(struct vidtv_psi_pmt_write_args *args)
{
- /* the number of bytes written by this function */
+ struct vidtv_psi_desc *table_descriptor = args->pmt->descriptor;
+ struct vidtv_psi_table_pmt_stream *stream = args->pmt->stream;
+ struct vidtv_psi_desc *stream_descriptor;
+ struct header_write_args h_args = {
+ .dest_buf = args->buf,
+ .dest_offset = args->offset,
+ .h = &args->pmt->header,
+ .pid = args->pid,
+ .continuity_counter = args->continuity_counter,
+ .dest_buf_sz = args->buf_sz,
+ };
+ struct psi_write_args psi_args = {
+ .dest_buf = args->buf,
+ .from = &args->pmt->bitfield,
+ .len = sizeof_field(struct vidtv_psi_table_pmt, bitfield) +
+ sizeof_field(struct vidtv_psi_table_pmt, bitfield2),
+ .pid = args->pid,
+ .new_psi_section = false,
+ .is_crc = false,
+ .dest_buf_sz = args->buf_sz,
+ };
+ struct desc_write_args d_args = {
+ .dest_buf = args->buf,
+ .desc = table_descriptor,
+ .pid = args->pid,
+ .dest_buf_sz = args->buf_sz,
+ };
+ struct crc32_write_args c_args = {
+ .dest_buf = args->buf,
+ .pid = args->pid,
+ .dest_buf_sz = args->buf_sz,
+ };
+ u32 crc = INITIAL_CRC;
u32 nbytes = 0;
- u32 crc = 0xffffffff;
-
- struct vidtv_psi_desc *table_descriptor = args.pmt->descriptor;
- struct vidtv_psi_table_pmt_stream *stream = args.pmt->stream;
- struct vidtv_psi_desc *stream_descriptor = (stream) ?
- args.pmt->stream->descriptor :
- NULL;
-
- struct header_write_args h_args = {};
- struct psi_write_args psi_args = {};
- struct desc_write_args d_args = {};
- struct crc32_write_args c_args = {};
-
- vidtv_psi_pmt_table_update_sec_len(args.pmt);
-
- h_args.dest_buf = args.buf;
- h_args.dest_offset = args.offset;
- h_args.h = &args.pmt->header;
- h_args.pid = args.pid;
- h_args.continuity_counter = args.continuity_counter;
- h_args.dest_buf_sz = args.buf_sz;
+
+ vidtv_psi_pmt_table_update_sec_len(args->pmt);
+
h_args.crc = &crc;
- nbytes += vidtv_psi_table_header_write_into(h_args);
+ nbytes += vidtv_psi_table_header_write_into(&h_args);
/* write the two bitfields */
- psi_args.dest_buf = args.buf;
- psi_args.from = &args.pmt->bitfield;
- psi_args.len = sizeof_field(struct vidtv_psi_table_pmt, bitfield) +
- sizeof_field(struct vidtv_psi_table_pmt, bitfield2);
-
- psi_args.dest_offset = args.offset + nbytes;
- psi_args.pid = args.pid;
- psi_args.new_psi_section = false;
- psi_args.continuity_counter = args.continuity_counter;
- psi_args.is_crc = false;
- psi_args.dest_buf_sz = args.buf_sz;
- psi_args.crc = &crc;
-
- nbytes += vidtv_psi_ts_psi_write_into(psi_args);
+ psi_args.dest_offset = args->offset + nbytes;
+ psi_args.continuity_counter = args->continuity_counter;
+ nbytes += vidtv_psi_ts_psi_write_into(&psi_args);
while (table_descriptor) {
/* write the descriptors, if any */
- d_args.dest_buf = args.buf;
- d_args.dest_offset = args.offset + nbytes;
- d_args.desc = table_descriptor;
- d_args.pid = args.pid;
- d_args.continuity_counter = args.continuity_counter;
- d_args.dest_buf_sz = args.buf_sz;
+ d_args.dest_offset = args->offset + nbytes;
+ d_args.continuity_counter = args->continuity_counter;
d_args.crc = &crc;
- nbytes += vidtv_psi_desc_write_into(d_args);
+ nbytes += vidtv_psi_desc_write_into(&d_args);
table_descriptor = table_descriptor->next;
}
+ psi_args.len += sizeof_field(struct vidtv_psi_table_pmt_stream, type);
while (stream) {
/* write the streams, if any */
psi_args.from = stream;
- psi_args.len = sizeof_field(struct vidtv_psi_table_pmt_stream, type) +
- sizeof_field(struct vidtv_psi_table_pmt_stream, bitfield) +
- sizeof_field(struct vidtv_psi_table_pmt_stream, bitfield2);
- psi_args.dest_offset = args.offset + nbytes;
+ psi_args.dest_offset = args->offset + nbytes;
+ psi_args.continuity_counter = args->continuity_counter;
+
+ nbytes += vidtv_psi_ts_psi_write_into(&psi_args);
- nbytes += vidtv_psi_ts_psi_write_into(psi_args);
+ stream_descriptor = stream->descriptor;
while (stream_descriptor) {
/* write the stream descriptors, if any */
- d_args.dest_buf = args.buf;
- d_args.dest_offset = args.offset + nbytes;
+ d_args.dest_offset = args->offset + nbytes;
d_args.desc = stream_descriptor;
- d_args.pid = args.pid;
- d_args.continuity_counter = args.continuity_counter;
- d_args.dest_buf_sz = args.buf_sz;
+ d_args.continuity_counter = args->continuity_counter;
d_args.crc = &crc;
- nbytes += vidtv_psi_desc_write_into(d_args);
+ nbytes += vidtv_psi_desc_write_into(&d_args);
stream_descriptor = stream_descriptor->next;
}
stream = stream->next;
}
- c_args.dest_buf = args.buf;
- c_args.dest_offset = args.offset + nbytes;
+ c_args.dest_offset = args->offset + nbytes;
c_args.crc = cpu_to_be32(crc);
- c_args.pid = args.pid;
- c_args.continuity_counter = args.continuity_counter;
- c_args.dest_buf_sz = args.buf_sz;
+ c_args.continuity_counter = args->continuity_counter;
/* Write the CRC32 at the end */
- nbytes += table_section_crc32_write_into(c_args);
+ nbytes += table_section_crc32_write_into(&c_args);
return nbytes;
}
kfree(pmt);
}
-struct vidtv_psi_table_sdt *vidtv_psi_sdt_table_init(u16 transport_stream_id)
+struct vidtv_psi_table_sdt *vidtv_psi_sdt_table_init(u16 network_id,
+ u16 transport_stream_id)
{
- struct vidtv_psi_table_sdt *sdt = kzalloc(sizeof(*sdt), GFP_KERNEL);
+ struct vidtv_psi_table_sdt *sdt;
+ const u16 RESERVED = 0xff;
const u16 SYNTAX = 0x1;
- const u16 ONE = 0x1;
const u16 ONES = 0x03;
- const u16 RESERVED = 0xff;
+ const u16 ONE = 0x1;
- sdt->header.table_id = 0x42;
+ sdt = kzalloc(sizeof(*sdt), GFP_KERNEL);
+ if (!sdt)
+ return NULL;
+ sdt->header.table_id = 0x42;
sdt->header.bitfield = cpu_to_be16((SYNTAX << 15) | (ONE << 14) | (ONES << 12));
/*
* This can be changed to something more useful, when support for
* NIT gets added
*/
- sdt->network_id = cpu_to_be16(0xff01);
+ sdt->network_id = cpu_to_be16(network_id);
sdt->reserved = RESERVED;
vidtv_psi_sdt_table_update_sec_len(sdt);
return sdt;
}
-u32 vidtv_psi_sdt_write_into(struct vidtv_psi_sdt_write_args args)
+u32 vidtv_psi_sdt_write_into(struct vidtv_psi_sdt_write_args *args)
{
+ struct header_write_args h_args = {
+ .dest_buf = args->buf,
+ .dest_offset = args->offset,
+ .h = &args->sdt->header,
+ .pid = VIDTV_SDT_PID,
+ .dest_buf_sz = args->buf_sz,
+ };
+ struct psi_write_args psi_args = {
+ .dest_buf = args->buf,
+ .len = sizeof_field(struct vidtv_psi_table_sdt, network_id) +
+ sizeof_field(struct vidtv_psi_table_sdt, reserved),
+ .pid = VIDTV_SDT_PID,
+ .new_psi_section = false,
+ .is_crc = false,
+ .dest_buf_sz = args->buf_sz,
+ };
+ struct desc_write_args d_args = {
+ .dest_buf = args->buf,
+ .pid = VIDTV_SDT_PID,
+ .dest_buf_sz = args->buf_sz,
+ };
+ struct crc32_write_args c_args = {
+ .dest_buf = args->buf,
+ .pid = VIDTV_SDT_PID,
+ .dest_buf_sz = args->buf_sz,
+ };
+ struct vidtv_psi_table_sdt_service *service = args->sdt->service;
+ struct vidtv_psi_desc *service_desc;
u32 nbytes = 0;
- u16 sdt_pid = VIDTV_SDT_PID; /* see ETSI EN 300 468 v1.15.1 p. 11 */
+ u32 crc = INITIAL_CRC;
- u32 crc = 0xffffffff;
+ /* see ETSI EN 300 468 v1.15.1 p. 11 */
- struct vidtv_psi_table_sdt_service *service = args.sdt->service;
- struct vidtv_psi_desc *service_desc = (args.sdt->service) ?
- args.sdt->service->descriptor :
- NULL;
+ vidtv_psi_sdt_table_update_sec_len(args->sdt);
- struct header_write_args h_args = {};
- struct psi_write_args psi_args = {};
- struct desc_write_args d_args = {};
- struct crc32_write_args c_args = {};
-
- vidtv_psi_sdt_table_update_sec_len(args.sdt);
-
- h_args.dest_buf = args.buf;
- h_args.dest_offset = args.offset;
- h_args.h = &args.sdt->header;
- h_args.pid = sdt_pid;
- h_args.continuity_counter = args.continuity_counter;
- h_args.dest_buf_sz = args.buf_sz;
+ h_args.continuity_counter = args->continuity_counter;
h_args.crc = &crc;
- nbytes += vidtv_psi_table_header_write_into(h_args);
-
- psi_args.dest_buf = args.buf;
- psi_args.from = &args.sdt->network_id;
+ nbytes += vidtv_psi_table_header_write_into(&h_args);
- psi_args.len = sizeof_field(struct vidtv_psi_table_sdt, network_id) +
- sizeof_field(struct vidtv_psi_table_sdt, reserved);
-
- psi_args.dest_offset = args.offset + nbytes;
- psi_args.pid = sdt_pid;
- psi_args.new_psi_section = false;
- psi_args.continuity_counter = args.continuity_counter;
- psi_args.is_crc = false;
- psi_args.dest_buf_sz = args.buf_sz;
+ psi_args.from = &args->sdt->network_id;
+ psi_args.dest_offset = args->offset + nbytes;
+ psi_args.continuity_counter = args->continuity_counter;
psi_args.crc = &crc;
/* copy u16 network_id + u8 reserved)*/
- nbytes += vidtv_psi_ts_psi_write_into(psi_args);
+ nbytes += vidtv_psi_ts_psi_write_into(&psi_args);
+
+ /* skip both pointers at the end */
+ psi_args.len = sizeof(struct vidtv_psi_table_sdt_service) -
+ sizeof(struct vidtv_psi_desc *) -
+ sizeof(struct vidtv_psi_table_sdt_service *);
while (service) {
/* copy the services, if any */
psi_args.from = service;
- /* skip both pointers at the end */
- psi_args.len = sizeof(struct vidtv_psi_table_sdt_service) -
- sizeof(struct vidtv_psi_desc *) -
- sizeof(struct vidtv_psi_table_sdt_service *);
- psi_args.dest_offset = args.offset + nbytes;
+ psi_args.dest_offset = args->offset + nbytes;
+ psi_args.continuity_counter = args->continuity_counter;
+
+ nbytes += vidtv_psi_ts_psi_write_into(&psi_args);
- nbytes += vidtv_psi_ts_psi_write_into(psi_args);
+ service_desc = service->descriptor;
while (service_desc) {
/* copy the service descriptors, if any */
- d_args.dest_buf = args.buf;
- d_args.dest_offset = args.offset + nbytes;
+ d_args.dest_offset = args->offset + nbytes;
d_args.desc = service_desc;
- d_args.pid = sdt_pid;
- d_args.continuity_counter = args.continuity_counter;
- d_args.dest_buf_sz = args.buf_sz;
+ d_args.continuity_counter = args->continuity_counter;
d_args.crc = &crc;
- nbytes += vidtv_psi_desc_write_into(d_args);
+ nbytes += vidtv_psi_desc_write_into(&d_args);
service_desc = service_desc->next;
}
service = service->next;
}
- c_args.dest_buf = args.buf;
- c_args.dest_offset = args.offset + nbytes;
+ c_args.dest_offset = args->offset + nbytes;
c_args.crc = cpu_to_be32(crc);
- c_args.pid = sdt_pid;
- c_args.continuity_counter = args.continuity_counter;
- c_args.dest_buf_sz = args.buf_sz;
+ c_args.continuity_counter = args->continuity_counter;
/* Write the CRC at the end */
- nbytes += table_section_crc32_write_into(c_args);
+ nbytes += table_section_crc32_write_into(&c_args);
return nbytes;
}
struct vidtv_psi_table_sdt_service
*vidtv_psi_sdt_service_init(struct vidtv_psi_table_sdt_service *head,
- u16 service_id)
+ u16 service_id,
+ bool eit_schedule,
+ bool eit_present_following)
{
struct vidtv_psi_table_sdt_service *service;
service = kzalloc(sizeof(*service), GFP_KERNEL);
+ if (!service)
+ return NULL;
/*
* ETSI 300 468: this is a 16bit field which serves as a label to
* corresponding program_map_section
*/
service->service_id = cpu_to_be16(service_id);
- service->EIT_schedule = 0x0;
- service->EIT_present_following = 0x0;
+ service->EIT_schedule = eit_schedule;
+ service->EIT_present_following = eit_present_following;
service->reserved = 0x3f;
service->bitfield = cpu_to_be16(RUNNING << 13);
vidtv_psi_sdt_service_assign(struct vidtv_psi_table_sdt *sdt,
struct vidtv_psi_table_sdt_service *service)
{
- if (service == sdt->service)
- return;
+ do {
+ if (service == sdt->service)
+ return;
- sdt->service = service;
+ sdt->service = service;
- /* recompute section length */
- vidtv_psi_sdt_table_update_sec_len(sdt);
+ /* recompute section length */
+ vidtv_psi_sdt_table_update_sec_len(sdt);
- if (vidtv_psi_get_sec_len(&sdt->header) > MAX_SECTION_LEN)
- vidtv_psi_sdt_service_assign(sdt, NULL);
+ service = NULL;
+ } while (vidtv_psi_get_sec_len(&sdt->header) > MAX_SECTION_LEN);
vidtv_psi_update_version_num(&sdt->header);
}
+/*
+ * PMTs contain information about programs. For each program,
+ * there is one PMT section. This function will create a section
+ * for each program found in the PAT
+ */
struct vidtv_psi_table_pmt**
-vidtv_psi_pmt_create_sec_for_each_pat_entry(struct vidtv_psi_table_pat *pat, u16 pcr_pid)
+vidtv_psi_pmt_create_sec_for_each_pat_entry(struct vidtv_psi_table_pat *pat,
+ u16 pcr_pid)
{
+ struct vidtv_psi_table_pat_program *program;
+ struct vidtv_psi_table_pmt **pmt_secs;
+ u32 i = 0, num_pmt = 0;
+
/*
- * PMTs contain information about programs. For each program,
- * there is one PMT section. This function will create a section
- * for each program found in the PAT
+ * The number of PMT entries is the number of PAT entries
+ * that contain service_id. That exclude special tables, like NIT
*/
- struct vidtv_psi_table_pat_program *program = pat->program;
- struct vidtv_psi_table_pmt **pmt_secs;
- u32 i = 0;
+ program = pat->program;
+ while (program) {
+ if (program->service_id)
+ num_pmt++;
+ program = program->next;
+ }
- /* a section for each program_id */
- pmt_secs = kcalloc(pat->programs,
+ pmt_secs = kcalloc(num_pmt,
sizeof(struct vidtv_psi_table_pmt *),
GFP_KERNEL);
-
- while (program) {
- pmt_secs[i] = vidtv_psi_pmt_table_init(be16_to_cpu(program->service_id), pcr_pid);
- ++i;
- program = program->next;
+ if (!pmt_secs)
+ return NULL;
+
+ for (program = pat->program; program; program = program->next) {
+ if (!program->service_id)
+ continue;
+ pmt_secs[i] = vidtv_psi_pmt_table_init(be16_to_cpu(program->service_id),
+ pcr_pid);
+
+ if (!pmt_secs[i]) {
+ while (i > 0) {
+ i--;
+ vidtv_psi_pmt_table_destroy(pmt_secs[i]);
+ }
+ return NULL;
+ }
+ i++;
}
+ pat->num_pmt = num_pmt;
return pmt_secs;
}
+/* find the PMT section associated with 'program_num' */
struct vidtv_psi_table_pmt
*vidtv_psi_find_pmt_sec(struct vidtv_psi_table_pmt **pmt_sections,
u16 nsections,
u16 program_num)
{
- /* find the PMT section associated with 'program_num' */
struct vidtv_psi_table_pmt *sec = NULL;
u32 i;
return NULL; /* not found */
}
+
+static void vidtv_psi_nit_table_update_sec_len(struct vidtv_psi_table_nit *nit)
+{
+ u16 length = 0;
+ struct vidtv_psi_table_transport *t = nit->transport;
+ u16 desc_loop_len;
+ u16 transport_loop_len = 0;
+
+ /*
+ * from immediately after 'section_length' until
+ * 'network_descriptor_length'
+ */
+ length += NIT_LEN_UNTIL_NETWORK_DESCRIPTOR_LEN;
+
+ desc_loop_len = vidtv_psi_desc_comp_loop_len(nit->descriptor);
+ vidtv_psi_set_desc_loop_len(&nit->bitfield, desc_loop_len, 12);
+
+ length += desc_loop_len;
+
+ length += sizeof_field(struct vidtv_psi_table_nit, bitfield2);
+
+ while (t) {
+ /* skip both pointers at the end */
+ transport_loop_len += sizeof(struct vidtv_psi_table_transport) -
+ sizeof(struct vidtv_psi_desc *) -
+ sizeof(struct vidtv_psi_table_transport *);
+
+ length += transport_loop_len;
+
+ desc_loop_len = vidtv_psi_desc_comp_loop_len(t->descriptor);
+ vidtv_psi_set_desc_loop_len(&t->bitfield, desc_loop_len, 12);
+
+ length += desc_loop_len;
+
+ t = t->next;
+ }
+
+ // Actually sets the transport stream loop len, maybe rename this function later
+ vidtv_psi_set_desc_loop_len(&nit->bitfield2, transport_loop_len, 12);
+ length += CRC_SIZE_IN_BYTES;
+
+ vidtv_psi_set_sec_len(&nit->header, length);
+}
+
+struct vidtv_psi_table_nit
+*vidtv_psi_nit_table_init(u16 network_id,
+ u16 transport_stream_id,
+ char *network_name,
+ struct vidtv_psi_desc_service_list_entry *service_list)
+{
+ struct vidtv_psi_table_transport *transport;
+ struct vidtv_psi_table_nit *nit;
+ const u16 SYNTAX = 0x1;
+ const u16 ONES = 0x03;
+ const u16 ONE = 0x1;
+
+ nit = kzalloc(sizeof(*nit), GFP_KERNEL);
+ if (!nit)
+ return NULL;
+
+ transport = kzalloc(sizeof(*transport), GFP_KERNEL);
+ if (!transport)
+ goto free_nit;
+
+ nit->header.table_id = 0x40; // ACTUAL_NETWORK
+
+ nit->header.bitfield = cpu_to_be16((SYNTAX << 15) | (ONE << 14) | (ONES << 12));
+
+ nit->header.id = cpu_to_be16(network_id);
+ nit->header.current_next = ONE;
+
+ nit->header.version = 0x1f;
+
+ nit->header.one2 = ONES;
+ nit->header.section_id = 0;
+ nit->header.last_section = 0;
+
+ nit->bitfield = cpu_to_be16(0xf);
+ nit->bitfield2 = cpu_to_be16(0xf);
+
+ nit->descriptor = (struct vidtv_psi_desc *)
+ vidtv_psi_network_name_desc_init(NULL, network_name);
+ if (!nit->descriptor)
+ goto free_transport;
+
+ transport->transport_id = cpu_to_be16(transport_stream_id);
+ transport->network_id = cpu_to_be16(network_id);
+ transport->bitfield = cpu_to_be16(0xf);
+ transport->descriptor = (struct vidtv_psi_desc *)
+ vidtv_psi_service_list_desc_init(NULL, service_list);
+ if (!transport->descriptor)
+ goto free_nit_desc;
+
+ nit->transport = transport;
+
+ vidtv_psi_nit_table_update_sec_len(nit);
+
+ return nit;
+
+free_nit_desc:
+ vidtv_psi_desc_destroy((struct vidtv_psi_desc *)nit->descriptor);
+
+free_transport:
+ kfree(transport);
+free_nit:
+ kfree(nit);
+ return NULL;
+}
+
+u32 vidtv_psi_nit_write_into(struct vidtv_psi_nit_write_args *args)
+{
+ struct header_write_args h_args = {
+ .dest_buf = args->buf,
+ .dest_offset = args->offset,
+ .h = &args->nit->header,
+ .pid = VIDTV_NIT_PID,
+ .dest_buf_sz = args->buf_sz,
+ };
+ struct psi_write_args psi_args = {
+ .dest_buf = args->buf,
+ .from = &args->nit->bitfield,
+ .len = sizeof_field(struct vidtv_psi_table_nit, bitfield),
+ .pid = VIDTV_NIT_PID,
+ .new_psi_section = false,
+ .is_crc = false,
+ .dest_buf_sz = args->buf_sz,
+ };
+ struct desc_write_args d_args = {
+ .dest_buf = args->buf,
+ .pid = VIDTV_NIT_PID,
+ .dest_buf_sz = args->buf_sz,
+ };
+ struct crc32_write_args c_args = {
+ .dest_buf = args->buf,
+ .pid = VIDTV_NIT_PID,
+ .dest_buf_sz = args->buf_sz,
+ };
+ struct vidtv_psi_desc *table_descriptor = args->nit->descriptor;
+ struct vidtv_psi_table_transport *transport = args->nit->transport;
+ struct vidtv_psi_desc *transport_descriptor;
+ u32 crc = INITIAL_CRC;
+ u32 nbytes = 0;
+
+ vidtv_psi_nit_table_update_sec_len(args->nit);
+
+ h_args.continuity_counter = args->continuity_counter;
+ h_args.crc = &crc;
+
+ nbytes += vidtv_psi_table_header_write_into(&h_args);
+
+ /* write the bitfield */
+
+ psi_args.dest_offset = args->offset + nbytes;
+ psi_args.continuity_counter = args->continuity_counter;
+ psi_args.crc = &crc;
+
+ nbytes += vidtv_psi_ts_psi_write_into(&psi_args);
+
+ while (table_descriptor) {
+ /* write the descriptors, if any */
+ d_args.dest_offset = args->offset + nbytes;
+ d_args.desc = table_descriptor;
+ d_args.continuity_counter = args->continuity_counter;
+ d_args.crc = &crc;
+
+ nbytes += vidtv_psi_desc_write_into(&d_args);
+
+ table_descriptor = table_descriptor->next;
+ }
+
+ /* write the second bitfield */
+ psi_args.from = &args->nit->bitfield2;
+ psi_args.len = sizeof_field(struct vidtv_psi_table_nit, bitfield2);
+ psi_args.dest_offset = args->offset + nbytes;
+
+ nbytes += vidtv_psi_ts_psi_write_into(&psi_args);
+
+ psi_args.len = sizeof_field(struct vidtv_psi_table_transport, transport_id) +
+ sizeof_field(struct vidtv_psi_table_transport, network_id) +
+ sizeof_field(struct vidtv_psi_table_transport, bitfield);
+ while (transport) {
+ /* write the transport sections, if any */
+ psi_args.from = transport;
+ psi_args.dest_offset = args->offset + nbytes;
+
+ nbytes += vidtv_psi_ts_psi_write_into(&psi_args);
+
+ transport_descriptor = transport->descriptor;
+
+ while (transport_descriptor) {
+ /* write the transport descriptors, if any */
+ d_args.dest_offset = args->offset + nbytes;
+ d_args.desc = transport_descriptor;
+ d_args.continuity_counter = args->continuity_counter;
+ d_args.crc = &crc;
+
+ nbytes += vidtv_psi_desc_write_into(&d_args);
+
+ transport_descriptor = transport_descriptor->next;
+ }
+
+ transport = transport->next;
+ }
+
+ c_args.dest_offset = args->offset + nbytes;
+ c_args.crc = cpu_to_be32(crc);
+ c_args.continuity_counter = args->continuity_counter;
+
+ /* Write the CRC32 at the end */
+ nbytes += table_section_crc32_write_into(&c_args);
+
+ return nbytes;
+}
+
+static void vidtv_psi_transport_destroy(struct vidtv_psi_table_transport *t)
+{
+ struct vidtv_psi_table_transport *tmp_t = NULL;
+ struct vidtv_psi_table_transport *curr_t = t;
+
+ while (curr_t) {
+ tmp_t = curr_t;
+ curr_t = curr_t->next;
+ vidtv_psi_desc_destroy(tmp_t->descriptor);
+ kfree(tmp_t);
+ }
+}
+
+void vidtv_psi_nit_table_destroy(struct vidtv_psi_table_nit *nit)
+{
+ vidtv_psi_desc_destroy(nit->descriptor);
+ vidtv_psi_transport_destroy(nit->transport);
+ kfree(nit);
+}
+
+void vidtv_psi_eit_table_update_sec_len(struct vidtv_psi_table_eit *eit)
+{
+ struct vidtv_psi_table_eit_event *e = eit->event;
+ u16 desc_loop_len;
+ u16 length = 0;
+
+ /*
+ * from immediately after 'section_length' until
+ * 'last_table_id'
+ */
+ length += EIT_LEN_UNTIL_LAST_TABLE_ID;
+
+ while (e) {
+ /* skip both pointers at the end */
+ length += sizeof(struct vidtv_psi_table_eit_event) -
+ sizeof(struct vidtv_psi_desc *) -
+ sizeof(struct vidtv_psi_table_eit_event *);
+
+ desc_loop_len = vidtv_psi_desc_comp_loop_len(e->descriptor);
+ vidtv_psi_set_desc_loop_len(&e->bitfield, desc_loop_len, 12);
+
+ length += desc_loop_len;
+
+ e = e->next;
+ }
+
+ length += CRC_SIZE_IN_BYTES;
+
+ vidtv_psi_set_sec_len(&eit->header, length);
+}
+
+void vidtv_psi_eit_event_assign(struct vidtv_psi_table_eit *eit,
+ struct vidtv_psi_table_eit_event *e)
+{
+ do {
+ if (e == eit->event)
+ return;
+
+ eit->event = e;
+ vidtv_psi_eit_table_update_sec_len(eit);
+
+ e = NULL;
+ } while (vidtv_psi_get_sec_len(&eit->header) > EIT_MAX_SECTION_LEN);
+
+ vidtv_psi_update_version_num(&eit->header);
+}
+
+struct vidtv_psi_table_eit
+*vidtv_psi_eit_table_init(u16 network_id,
+ u16 transport_stream_id,
+ __be16 service_id)
+{
+ struct vidtv_psi_table_eit *eit;
+ const u16 SYNTAX = 0x1;
+ const u16 ONE = 0x1;
+ const u16 ONES = 0x03;
+
+ eit = kzalloc(sizeof(*eit), GFP_KERNEL);
+ if (!eit)
+ return NULL;
+
+ eit->header.table_id = 0x4e; //actual_transport_stream: present/following
+
+ eit->header.bitfield = cpu_to_be16((SYNTAX << 15) | (ONE << 14) | (ONES << 12));
+
+ eit->header.id = service_id;
+ eit->header.current_next = ONE;
+
+ eit->header.version = 0x1f;
+
+ eit->header.one2 = ONES;
+ eit->header.section_id = 0;
+ eit->header.last_section = 0;
+
+ eit->transport_id = cpu_to_be16(transport_stream_id);
+ eit->network_id = cpu_to_be16(network_id);
+
+ eit->last_segment = eit->header.last_section; /* not implemented */
+ eit->last_table_id = eit->header.table_id; /* not implemented */
+
+ vidtv_psi_eit_table_update_sec_len(eit);
+
+ return eit;
+}
+
+u32 vidtv_psi_eit_write_into(struct vidtv_psi_eit_write_args *args)
+{
+ struct header_write_args h_args = {
+ .dest_buf = args->buf,
+ .dest_offset = args->offset,
+ .h = &args->eit->header,
+ .pid = VIDTV_EIT_PID,
+ .dest_buf_sz = args->buf_sz,
+ };
+ struct psi_write_args psi_args = {
+ .dest_buf = args->buf,
+ .len = sizeof_field(struct vidtv_psi_table_eit, transport_id) +
+ sizeof_field(struct vidtv_psi_table_eit, network_id) +
+ sizeof_field(struct vidtv_psi_table_eit, last_segment) +
+ sizeof_field(struct vidtv_psi_table_eit, last_table_id),
+ .pid = VIDTV_EIT_PID,
+ .new_psi_section = false,
+ .is_crc = false,
+ .dest_buf_sz = args->buf_sz,
+ };
+ struct desc_write_args d_args = {
+ .dest_buf = args->buf,
+ .pid = VIDTV_EIT_PID,
+ .dest_buf_sz = args->buf_sz,
+ };
+ struct crc32_write_args c_args = {
+ .dest_buf = args->buf,
+ .pid = VIDTV_EIT_PID,
+ .dest_buf_sz = args->buf_sz,
+ };
+ struct vidtv_psi_table_eit_event *event = args->eit->event;
+ struct vidtv_psi_desc *event_descriptor;
+ u32 crc = INITIAL_CRC;
+ u32 nbytes = 0;
+
+ vidtv_psi_eit_table_update_sec_len(args->eit);
+
+ h_args.continuity_counter = args->continuity_counter;
+ h_args.crc = &crc;
+
+ nbytes += vidtv_psi_table_header_write_into(&h_args);
+
+ psi_args.from = &args->eit->transport_id;
+ psi_args.dest_offset = args->offset + nbytes;
+ psi_args.continuity_counter = args->continuity_counter;
+ psi_args.crc = &crc;
+
+ nbytes += vidtv_psi_ts_psi_write_into(&psi_args);
+
+ /* skip both pointers at the end */
+ psi_args.len = sizeof(struct vidtv_psi_table_eit_event) -
+ sizeof(struct vidtv_psi_desc *) -
+ sizeof(struct vidtv_psi_table_eit_event *);
+ while (event) {
+ /* copy the events, if any */
+ psi_args.from = event;
+ psi_args.dest_offset = args->offset + nbytes;
+
+ nbytes += vidtv_psi_ts_psi_write_into(&psi_args);
+
+ event_descriptor = event->descriptor;
+
+ while (event_descriptor) {
+ /* copy the event descriptors, if any */
+ d_args.dest_offset = args->offset + nbytes;
+ d_args.desc = event_descriptor;
+ d_args.continuity_counter = args->continuity_counter;
+ d_args.crc = &crc;
+
+ nbytes += vidtv_psi_desc_write_into(&d_args);
+
+ event_descriptor = event_descriptor->next;
+ }
+
+ event = event->next;
+ }
+
+ c_args.dest_offset = args->offset + nbytes;
+ c_args.crc = cpu_to_be32(crc);
+ c_args.continuity_counter = args->continuity_counter;
+
+ /* Write the CRC at the end */
+ nbytes += table_section_crc32_write_into(&c_args);
+
+ return nbytes;
+}
+
+struct vidtv_psi_table_eit_event
+*vidtv_psi_eit_event_init(struct vidtv_psi_table_eit_event *head, u16 event_id)
+{
+ const u8 DURATION[] = {0x23, 0x59, 0x59}; /* BCD encoded */
+ struct vidtv_psi_table_eit_event *e;
+ struct timespec64 ts;
+ struct tm time;
+ int mjd, l;
+ __be16 mjd_be;
+
+ e = kzalloc(sizeof(*e), GFP_KERNEL);
+ if (!e)
+ return NULL;
+
+ e->event_id = cpu_to_be16(event_id);
+
+ ts = ktime_to_timespec64(ktime_get_real());
+ time64_to_tm(ts.tv_sec, 0, &time);
+
+ /* Convert date to Modified Julian Date - per EN 300 468 Annex C */
+ if (time.tm_mon < 2)
+ l = 1;
+ else
+ l = 0;
+
+ mjd = 14956 + time.tm_mday;
+ mjd += (time.tm_year - l) * 36525 / 100;
+ mjd += (time.tm_mon + 2 + l * 12) * 306001 / 10000;
+ mjd_be = cpu_to_be16(mjd);
+
+ /*
+ * Store MJD and hour/min/sec to the event.
+ *
+ * Let's make the event to start on a full hour
+ */
+ memcpy(e->start_time, &mjd_be, sizeof(mjd_be));
+ e->start_time[2] = bin2bcd(time.tm_hour);
+ e->start_time[3] = 0;
+ e->start_time[4] = 0;
+
+ /*
+ * TODO: for now, the event will last for a day. Should be
+ * enough for testing purposes, but if one runs the driver
+ * for more than that, the current event will become invalid.
+ * So, we need a better code here in order to change the start
+ * time once the event expires.
+ */
+ memcpy(e->duration, DURATION, sizeof(e->duration));
+
+ e->bitfield = cpu_to_be16(RUNNING << 13);
+
+ if (head) {
+ while (head->next)
+ head = head->next;
+
+ head->next = e;
+ }
+
+ return e;
+}
+
+void vidtv_psi_eit_event_destroy(struct vidtv_psi_table_eit_event *e)
+{
+ struct vidtv_psi_table_eit_event *tmp_e = NULL;
+ struct vidtv_psi_table_eit_event *curr_e = e;
+
+ while (curr_e) {
+ tmp_e = curr_e;
+ curr_e = curr_e->next;
+ vidtv_psi_desc_destroy(tmp_e->descriptor);
+ kfree(tmp_e);
+ }
+}
+
+void vidtv_psi_eit_table_destroy(struct vidtv_psi_table_eit *eit)
+{
+ vidtv_psi_eit_event_destroy(eit->event);
+ kfree(eit);
+}
* technically be broken into one or more sections, we do not do this here,
* hence 'table' and 'section' are interchangeable for vidtv.
*
- * This code currently supports three tables: PAT, PMT and SDT. These are the
- * bare minimum to get userspace to recognize our MPEG transport stream. It can
- * be extended to support more PSI tables in the future.
- *
* Copyright (C) 2020 Daniel W. S. Almeida
*/
#define VIDTV_PSI_H
#include <linux/types.h>
-#include <asm/byteorder.h>
/*
* all section lengths start immediately after the 'section_length' field
#define PAT_LEN_UNTIL_LAST_SECTION_NUMBER 5
#define PMT_LEN_UNTIL_PROGRAM_INFO_LENGTH 9
#define SDT_LEN_UNTIL_RESERVED_FOR_FUTURE_USE 8
+#define NIT_LEN_UNTIL_NETWORK_DESCRIPTOR_LEN 7
+#define EIT_LEN_UNTIL_LAST_TABLE_ID 11
#define MAX_SECTION_LEN 1021
+#define EIT_MAX_SECTION_LEN 4093 /* see ETSI 300 468 v.1.10.1 p. 26 */
#define VIDTV_PAT_PID 0 /* mandated by the specs */
#define VIDTV_SDT_PID 0x0011 /* mandated by the specs */
+#define VIDTV_NIT_PID 0x0010 /* mandated by the specs */
+#define VIDTV_EIT_PID 0x0012 /*mandated by the specs */
enum vidtv_psi_descriptors {
REGISTRATION_DESCRIPTOR = 0x05, /* See ISO/IEC 13818-1 section 2.6.8 */
+ NETWORK_NAME_DESCRIPTOR = 0x40, /* See ETSI EN 300 468 section 6.2.27 */
+ SERVICE_LIST_DESCRIPTOR = 0x41, /* See ETSI EN 300 468 section 6.2.35 */
SERVICE_DESCRIPTOR = 0x48, /* See ETSI EN 300 468 section 6.2.33 */
+ SHORT_EVENT_DESCRIPTOR = 0x4d, /* See ETSI EN 300 468 section 6.2.37 */
};
enum vidtv_psi_stream_types {
STREAM_PRIVATE_DATA = 0x06, /* see ISO/IEC 13818-1 2000 p. 48 */
};
-/**
+/*
* struct vidtv_psi_desc - A generic PSI descriptor type.
* The descriptor length is an 8-bit field specifying the total number of bytes of the data portion
* of the descriptor following the byte defining the value of this field.
u8 data[];
} __packed;
-/**
+/*
* struct vidtv_psi_desc_service - Service descriptor.
* See ETSI EN 300 468 section 6.2.33.
*/
char *service_name;
} __packed;
-/**
+/*
* struct vidtv_psi_desc_registration - A registration descriptor.
* See ISO/IEC 13818-1 section 2.6.8
*/
u8 additional_identification_info[];
} __packed;
-/**
+/*
+ * struct vidtv_psi_desc_network_name - A network name descriptor
+ * see ETSI EN 300 468 v1.15.1 section 6.2.27
+ */
+struct vidtv_psi_desc_network_name {
+ struct vidtv_psi_desc *next;
+ u8 type;
+ u8 length;
+ char *network_name;
+} __packed;
+
+struct vidtv_psi_desc_service_list_entry {
+ __be16 service_id;
+ u8 service_type;
+ struct vidtv_psi_desc_service_list_entry *next;
+} __packed;
+
+/*
+ * struct vidtv_psi_desc_service_list - A service list descriptor
+ * see ETSI EN 300 468 v1.15.1 section 6.2.35
+ */
+struct vidtv_psi_desc_service_list {
+ struct vidtv_psi_desc *next;
+ u8 type;
+ u8 length;
+ struct vidtv_psi_desc_service_list_entry *service_list;
+} __packed;
+
+/*
+ * struct vidtv_psi_desc_short_event - A short event descriptor
+ * see ETSI EN 300 468 v1.15.1 section 6.2.37
+ */
+struct vidtv_psi_desc_short_event {
+ struct vidtv_psi_desc *next;
+ u8 type;
+ u8 length;
+ char *iso_language_code;
+ u8 event_name_len;
+ char *event_name;
+ u8 text_len;
+ char *text;
+} __packed;
+
+struct vidtv_psi_desc_short_event
+*vidtv_psi_short_event_desc_init(struct vidtv_psi_desc *head,
+ char *iso_language_code,
+ char *event_name,
+ char *text);
+
+/*
* struct vidtv_psi_table_header - A header that is present for all PSI tables.
*/
struct vidtv_psi_table_header {
u8 last_section; /* last_section_number */
} __packed;
-/**
+/*
* struct vidtv_psi_table_pat_program - A single program in the PAT
* See ISO/IEC 13818-1 : 2000 p.43
*/
struct vidtv_psi_table_pat_program *next;
} __packed;
-/**
+/*
* struct vidtv_psi_table_pat - The Program Allocation Table (PAT)
* See ISO/IEC 13818-1 : 2000 p.43
*/
struct vidtv_psi_table_pat {
struct vidtv_psi_table_header header;
- u16 programs; /* Included by libdvbv5, not part of the table and not actually serialized */
+ u16 num_pat;
+ u16 num_pmt;
struct vidtv_psi_table_pat_program *program;
} __packed;
-/**
+/*
* struct vidtv_psi_table_sdt_service - Represents a service in the SDT.
* see ETSI EN 300 468 v1.15.1 section 5.2.3.
*/
struct vidtv_psi_table_sdt_service *next;
} __packed;
-/**
+/*
* struct vidtv_psi_table_sdt - Represents the Service Description Table
* see ETSI EN 300 468 v1.15.1 section 5.2.3.
*/
struct vidtv_psi_table_sdt_service *service;
} __packed;
-/**
+/*
* enum service_running_status - Status of a SDT service.
* see ETSI EN 300 468 v1.15.1 section 5.2.3 table 6.
*/
RUNNING = 0x4,
};
-/**
+/*
* enum service_type - The type of a SDT service.
* see ETSI EN 300 468 v1.15.1 section 6.2.33, table 81.
*/
enum service_type {
/* see ETSI EN 300 468 v1.15.1 p. 77 */
DIGITAL_TELEVISION_SERVICE = 0x1,
+ DIGITAL_RADIO_SOUND_SERVICE = 0X2,
};
-/**
+/*
* struct vidtv_psi_table_pmt_stream - A single stream in the PMT.
* See ISO/IEC 13818-1 : 2000 p.46.
*/
struct vidtv_psi_table_pmt_stream *next;
} __packed;
-/**
+/*
* struct vidtv_psi_table_pmt - The Program Map Table (PMT).
* See ISO/IEC 13818-1 : 2000 p.46.
*/
u8 *additional_ident_info,
u32 additional_info_len);
+struct vidtv_psi_desc_network_name
+*vidtv_psi_network_name_desc_init(struct vidtv_psi_desc *head, char *network_name);
+
+struct vidtv_psi_desc_service_list
+*vidtv_psi_service_list_desc_init(struct vidtv_psi_desc *head,
+ struct vidtv_psi_desc_service_list_entry *entry);
+
struct vidtv_psi_table_pat_program
*vidtv_psi_pat_program_init(struct vidtv_psi_table_pat_program *head,
u16 service_id,
struct vidtv_psi_table_pmt *vidtv_psi_pmt_table_init(u16 program_number,
u16 pcr_pid);
-struct vidtv_psi_table_sdt *vidtv_psi_sdt_table_init(u16 transport_stream_id);
+struct vidtv_psi_table_sdt *vidtv_psi_sdt_table_init(u16 network_id,
+ u16 transport_stream_id);
struct vidtv_psi_table_sdt_service*
vidtv_psi_sdt_service_init(struct vidtv_psi_table_sdt_service *head,
- u16 service_id);
+ u16 service_id,
+ bool eit_schedule,
+ bool eit_present_following);
void
vidtv_psi_desc_destroy(struct vidtv_psi_desc *desc);
* vidtv_psi_create_sec_for_each_pat_entry - Create a PMT section for each
* program found in the PAT
* @pat: The PAT to look for programs.
- * @s: The stream loop (one or more streams)
* @pcr_pid: packet ID for the PCR to be used for the program described in this
* PMT section
*/
* equal to the size of the PAT, since more space is needed for TS headers during TS
* encapsulation.
*/
-u32 vidtv_psi_pat_write_into(struct vidtv_psi_pat_write_args args);
+u32 vidtv_psi_pat_write_into(struct vidtv_psi_pat_write_args *args);
/**
* struct vidtv_psi_sdt_write_args - Arguments for writing a SDT table
* equal to the size of the SDT, since more space is needed for TS headers during TS
* encapsulation.
*/
-u32 vidtv_psi_sdt_write_into(struct vidtv_psi_sdt_write_args args);
+u32 vidtv_psi_sdt_write_into(struct vidtv_psi_sdt_write_args *args);
/**
* struct vidtv_psi_pmt_write_args - Arguments for writing a PMT section
* @buf: The destination buffer.
* @offset: The offset into the destination buffer.
* @pmt: A pointer to the PMT.
+ * @pid: Program ID
* @buf_sz: The size of the destination buffer.
* @continuity_counter: A pointer to the CC. Incremented on every new packet.
- *
+ * @pcr_pid: The TS PID used for the PSI packets. All channels will share the
+ * same PCR.
*/
struct vidtv_psi_pmt_write_args {
char *buf;
* equal to the size of the PMT section, since more space is needed for TS headers
* during TS encapsulation.
*/
-u32 vidtv_psi_pmt_write_into(struct vidtv_psi_pmt_write_args args);
+u32 vidtv_psi_pmt_write_into(struct vidtv_psi_pmt_write_args *args);
/**
* vidtv_psi_find_pmt_sec - Finds the PMT section for 'program_num'
u16 vidtv_psi_get_pat_program_pid(struct vidtv_psi_table_pat_program *p);
u16 vidtv_psi_pmt_stream_get_elem_pid(struct vidtv_psi_table_pmt_stream *s);
+/**
+ * struct vidtv_psi_table_transport - A entry in the TS loop for the NIT and/or other tables.
+ * See ETSI 300 468 section 5.2.1
+ * @transport_id: The TS ID being described
+ * @network_id: The network_id that contains the TS ID
+ * @bitfield: Contains the descriptor loop length
+ * @descriptor: A descriptor loop
+ * @next: Pointer to the next entry
+ *
+ */
+struct vidtv_psi_table_transport {
+ __be16 transport_id;
+ __be16 network_id;
+ __be16 bitfield; /* desc_len: 12, reserved: 4 */
+ struct vidtv_psi_desc *descriptor;
+ struct vidtv_psi_table_transport *next;
+} __packed;
+
+/**
+ * struct vidtv_psi_table_nit - A Network Information Table (NIT). See ETSI 300
+ * 468 section 5.2.1
+ * @header: A PSI table header
+ * @bitfield: Contains the network descriptor length
+ * @descriptor: A descriptor loop describing the network
+ * @bitfield2: Contains the transport stream loop length
+ * @transport: The transport stream loop
+ *
+ */
+struct vidtv_psi_table_nit {
+ struct vidtv_psi_table_header header;
+ __be16 bitfield; /* network_desc_len: 12, reserved:4 */
+ struct vidtv_psi_desc *descriptor;
+ __be16 bitfield2; /* ts_loop_len: 12, reserved: 4 */
+ struct vidtv_psi_table_transport *transport;
+} __packed;
+
+struct vidtv_psi_table_nit
+*vidtv_psi_nit_table_init(u16 network_id,
+ u16 transport_stream_id,
+ char *network_name,
+ struct vidtv_psi_desc_service_list_entry *service_list);
+
+/**
+ * struct vidtv_psi_nit_write_args - Arguments for writing a NIT section
+ * @buf: The destination buffer.
+ * @offset: The offset into the destination buffer.
+ * @nit: A pointer to the NIT
+ * @buf_sz: The size of the destination buffer.
+ * @continuity_counter: A pointer to the CC. Incremented on every new packet.
+ *
+ */
+struct vidtv_psi_nit_write_args {
+ char *buf;
+ u32 offset;
+ struct vidtv_psi_table_nit *nit;
+ u32 buf_sz;
+ u8 *continuity_counter;
+};
+
+/**
+ * vidtv_psi_nit_write_into - Write NIT as MPEG-TS packets into a buffer.
+ * @args: an instance of struct vidtv_psi_nit_write_args
+ *
+ * This function writes the MPEG TS packets for a NIT table into a buffer.
+ * Calling code will usually generate the NIT via a call to its init function
+ * and thus is responsible for freeing it.
+ *
+ * Return: The number of bytes written into the buffer. This is NOT
+ * equal to the size of the NIT, since more space is needed for TS headers during TS
+ * encapsulation.
+ */
+u32 vidtv_psi_nit_write_into(struct vidtv_psi_nit_write_args *args);
+
+void vidtv_psi_nit_table_destroy(struct vidtv_psi_table_nit *nit);
+
+/*
+ * struct vidtv_psi_desc_short_event - A short event descriptor
+ * see ETSI EN 300 468 v1.15.1 section 6.2.37
+ */
+struct vidtv_psi_table_eit_event {
+ __be16 event_id;
+ u8 start_time[5];
+ u8 duration[3];
+ __be16 bitfield; /* desc_length: 12, free_CA_mode: 1, running_status: 1 */
+ struct vidtv_psi_desc *descriptor;
+ struct vidtv_psi_table_eit_event *next;
+} __packed;
+
+/*
+ * struct vidtv_psi_table_eit - A Event Information Table (EIT)
+ * See ETSI 300 468 section 5.2.4
+ */
+struct vidtv_psi_table_eit {
+ struct vidtv_psi_table_header header;
+ __be16 transport_id;
+ __be16 network_id;
+ u8 last_segment;
+ u8 last_table_id;
+ struct vidtv_psi_table_eit_event *event;
+} __packed;
+
+struct vidtv_psi_table_eit
+*vidtv_psi_eit_table_init(u16 network_id,
+ u16 transport_stream_id,
+ u16 service_id);
+
+/**
+ * struct vidtv_psi_eit_write_args - Arguments for writing an EIT section
+ * @buf: The destination buffer.
+ * @offset: The offset into the destination buffer.
+ * @eit: A pointer to the EIT
+ * @buf_sz: The size of the destination buffer.
+ * @continuity_counter: A pointer to the CC. Incremented on every new packet.
+ *
+ */
+struct vidtv_psi_eit_write_args {
+ char *buf;
+ u32 offset;
+ struct vidtv_psi_table_eit *eit;
+ u32 buf_sz;
+ u8 *continuity_counter;
+};
+
+/**
+ * vidtv_psi_eit_write_into - Write EIT as MPEG-TS packets into a buffer.
+ * @args: an instance of struct vidtv_psi_nit_write_args
+ *
+ * This function writes the MPEG TS packets for a EIT table into a buffer.
+ * Calling code will usually generate the EIT via a call to its init function
+ * and thus is responsible for freeing it.
+ *
+ * Return: The number of bytes written into the buffer. This is NOT
+ * equal to the size of the EIT, since more space is needed for TS headers during TS
+ * encapsulation.
+ */
+u32 vidtv_psi_eit_write_into(struct vidtv_psi_eit_write_args *args);
+
+void vidtv_psi_eit_table_destroy(struct vidtv_psi_table_eit *eit);
+
+/**
+ * vidtv_psi_eit_table_update_sec_len - Recompute and update the EIT section length.
+ * @eit: The EIT whose length is to be updated.
+ *
+ * This will traverse the table and accumulate the length of its components,
+ * which is then used to replace the 'section_length' field.
+ *
+ * If section_length > EIT_MAX_SECTION_LEN, the operation fails.
+ */
+void vidtv_psi_eit_table_update_sec_len(struct vidtv_psi_table_eit *eit);
+
+/**
+ * vidtv_psi_eit_event_assign - Assigns the event loop to the EIT.
+ * @eit: The EIT to assign to.
+ * @e: The event loop
+ *
+ * This will free the previous event loop in the table.
+ * This will assign ownership of the stream loop to the table, i.e. the table
+ * will free this stream loop when a call to its destroy function is made.
+ */
+void vidtv_psi_eit_event_assign(struct vidtv_psi_table_eit *eit,
+ struct vidtv_psi_table_eit_event *e);
+
+struct vidtv_psi_table_eit_event
+*vidtv_psi_eit_event_init(struct vidtv_psi_table_eit_event *head, u16 event_id);
+
+void vidtv_psi_eit_event_destroy(struct vidtv_psi_table_eit_event *e);
+
#endif // VIDTV_PSI_H
#define pr_fmt(fmt) KBUILD_MODNAME ":%s, %d: " fmt, __func__, __LINE__
-#include <linux/types.h>
-#include <linux/slab.h>
+#include <linux/bug.h>
#include <linux/crc32.h>
-#include <linux/vmalloc.h>
-#include <linux/string.h>
-#include <linux/kernel.h>
+#include <linux/fixp-arith.h>
#include <linux/jiffies.h>
+#include <linux/kernel.h>
+#include <linux/math64.h>
#include <linux/printk.h>
#include <linux/ratelimit.h>
-#include <linux/fixp-arith.h>
-
-#include <linux/math64.h>
-#include <asm/byteorder.h>
+#include <linux/slab.h>
+#include <linux/string.h>
+#include <linux/types.h>
+#include <linux/vmalloc.h>
-#include "vidtv_s302m.h"
-#include "vidtv_encoder.h"
#include "vidtv_common.h"
+#include "vidtv_encoder.h"
+#include "vidtv_s302m.h"
#define S302M_SAMPLING_RATE_HZ 48000
#define PES_PRIVATE_STREAM_1 0xbd /* PES: private_stream_1 */
int duration;
};
-#define COMPASS 120 /* beats per minute (Allegro) */
-static const struct tone_duration beethoven_5th_symphony[] = {
+#define COMPASS 100 /* beats per minute */
+static const struct tone_duration beethoven_fur_elise[] = {
+ { NOTE_SILENT, 512},
{ NOTE_E_6, 128}, { NOTE_DS_6, 128}, { NOTE_E_6, 128},
{ NOTE_DS_6, 128}, { NOTE_E_6, 128}, { NOTE_B_5, 128},
{ NOTE_D_6, 128}, { NOTE_C_6, 128}, { NOTE_A_3, 128},
{ NOTE_E_5, 128}, { NOTE_D_5, 128}, { NOTE_A_3, 128},
{ NOTE_E_4, 128}, { NOTE_A_4, 128}, { NOTE_E_4, 128},
{ NOTE_D_5, 128}, { NOTE_C_5, 128}, { NOTE_E_3, 128},
- { NOTE_E_4, 128}, { NOTE_E_5, 255}, { NOTE_E_6, 128},
- { NOTE_E_5, 128}, { NOTE_E_6, 128}, { NOTE_E_5, 255},
+ { NOTE_E_4, 128}, { NOTE_E_5, 128}, { NOTE_E_5, 128},
+ { NOTE_E_6, 128}, { NOTE_E_5, 128}, { NOTE_E_6, 128},
+ { NOTE_E_5, 128}, { NOTE_E_5, 128}, { NOTE_DS_5, 128},
+ { NOTE_E_5, 128}, { NOTE_DS_6, 128}, { NOTE_E_6, 128},
{ NOTE_DS_5, 128}, { NOTE_E_5, 128}, { NOTE_DS_6, 128},
- { NOTE_E_6, 128}, { NOTE_DS_5, 128}, { NOTE_E_5, 128},
- { NOTE_DS_6, 128}, { NOTE_E_6, 128}, { NOTE_DS_6, 128},
{ NOTE_E_6, 128}, { NOTE_DS_6, 128}, { NOTE_E_6, 128},
- { NOTE_B_5, 128}, { NOTE_D_6, 128}, { NOTE_C_6, 128},
- { NOTE_A_3, 128}, { NOTE_E_4, 128}, { NOTE_A_4, 128},
- { NOTE_C_5, 128}, { NOTE_E_5, 128}, { NOTE_A_5, 128},
- { NOTE_E_3, 128}, { NOTE_E_4, 128}, { NOTE_GS_4, 128},
- { NOTE_E_5, 128}, { NOTE_GS_5, 128}, { NOTE_B_5, 128},
- { NOTE_A_3, 128}, { NOTE_E_4, 128}, { NOTE_A_4, 128},
- { NOTE_E_5, 128}, { NOTE_E_6, 128}, { NOTE_DS_6, 128},
+ { NOTE_DS_6, 128}, { NOTE_E_6, 128}, { NOTE_B_5, 128},
+ { NOTE_D_6, 128}, { NOTE_C_6, 128}, { NOTE_A_3, 128},
+ { NOTE_E_4, 128}, { NOTE_A_4, 128}, { NOTE_C_5, 128},
+ { NOTE_E_5, 128}, { NOTE_A_5, 128}, { NOTE_E_3, 128},
+ { NOTE_E_4, 128}, { NOTE_GS_4, 128}, { NOTE_E_5, 128},
+ { NOTE_GS_5, 128}, { NOTE_B_5, 128}, { NOTE_A_3, 128},
+ { NOTE_E_4, 128}, { NOTE_A_4, 128}, { NOTE_E_5, 128},
{ NOTE_E_6, 128}, { NOTE_DS_6, 128}, { NOTE_E_6, 128},
- { NOTE_B_5, 128}, { NOTE_D_6, 128}, { NOTE_C_6, 128},
- { NOTE_A_3, 128}, { NOTE_E_4, 128}, { NOTE_A_4, 128},
- { NOTE_C_5, 128}, { NOTE_E_5, 128}, { NOTE_A_5, 128},
- { NOTE_E_3, 128}, { NOTE_E_4, 128}, { NOTE_GS_4, 128},
- { NOTE_E_5, 128}, { NOTE_C_6, 128}, { NOTE_B_5, 128},
- { NOTE_C_5, 255}, { NOTE_C_5, 255}, { NOTE_SILENT, 512},
+ { NOTE_DS_6, 128}, { NOTE_E_6, 128}, { NOTE_B_5, 128},
+ { NOTE_D_6, 128}, { NOTE_C_6, 128}, { NOTE_A_3, 128},
+ { NOTE_E_4, 128}, { NOTE_A_4, 128}, { NOTE_C_5, 128},
+ { NOTE_E_5, 128}, { NOTE_A_5, 128}, { NOTE_E_3, 128},
+ { NOTE_E_4, 128}, { NOTE_GS_4, 128}, { NOTE_E_5, 128},
+ { NOTE_C_6, 128}, { NOTE_B_5, 128}, { NOTE_A_5, 512},
+ { NOTE_SILENT, 256},
};
static struct vidtv_access_unit *vidtv_s302m_access_unit_init(struct vidtv_access_unit *head)
{
- struct vidtv_access_unit *au = kzalloc(sizeof(*au), GFP_KERNEL);
+ struct vidtv_access_unit *au;
+
+ au = kzalloc(sizeof(*au), GFP_KERNEL);
+ if (!au)
+ return NULL;
if (head) {
while (head->next)
static void
vidtv_s302m_compute_sample_count_from_video(struct vidtv_encoder *e)
{
- struct vidtv_access_unit *au = e->access_units;
struct vidtv_access_unit *sync_au = e->sync->access_units;
- u32 vau_duration_usecs;
+ struct vidtv_access_unit *au = e->access_units;
u32 sample_duration_usecs;
+ u32 vau_duration_usecs;
u32 s;
vau_duration_usecs = USEC_PER_SEC / e->sync->sampling_rate_hz;
{
u16 sample;
int pos;
+ struct vidtv_s302m_ctx *ctx = e->ctx;
if (!e->src_buf) {
/*
* Simple tone generator: play the tones at the
- * beethoven_5th_symphony array.
+ * beethoven_fur_elise array.
*/
- if (e->last_duration <= 0) {
- if (e->src_buf_offset >= ARRAY_SIZE(beethoven_5th_symphony))
+ if (ctx->last_duration <= 0) {
+ if (e->src_buf_offset >= ARRAY_SIZE(beethoven_fur_elise))
e->src_buf_offset = 0;
- e->last_tone = beethoven_5th_symphony[e->src_buf_offset].note;
- e->last_duration = beethoven_5th_symphony[e->src_buf_offset].duration * S302M_SAMPLING_RATE_HZ / COMPASS / 5;
+ ctx->last_tone = beethoven_fur_elise[e->src_buf_offset].note;
+ ctx->last_duration = beethoven_fur_elise[e->src_buf_offset].duration *
+ S302M_SAMPLING_RATE_HZ / COMPASS / 5;
e->src_buf_offset++;
- e->note_offset = 0;
+ ctx->note_offset = 0;
} else {
- e->last_duration--;
+ ctx->last_duration--;
}
- /* Handle silent */
- if (!e->last_tone) {
- e->src_buf_offset = 0;
+ /* Handle pause notes */
+ if (!ctx->last_tone)
return 0x8000;
- }
- pos = (2 * PI * e->note_offset * e->last_tone / S302M_SAMPLING_RATE_HZ);
-
- if (pos == 360)
- e->note_offset = 0;
- else
- e->note_offset++;
+ pos = (2 * PI * ctx->note_offset * ctx->last_tone) / S302M_SAMPLING_RATE_HZ;
+ ctx->note_offset++;
return (fixp_sin32(pos % (2 * PI)) >> 16) + 0x8000;
}
static u32 vidtv_s302m_write_frame(struct vidtv_encoder *e,
u16 sample)
{
- u32 nbytes = 0;
- struct vidtv_s302m_frame_16 f = {};
struct vidtv_s302m_ctx *ctx = e->ctx;
+ struct vidtv_s302m_frame_16 f = {};
+ u32 nbytes = 0;
/* from ffmpeg: see s302enc.c */
static void *vidtv_s302m_encode(struct vidtv_encoder *e)
{
+ struct vidtv_s302m_ctx *ctx = e->ctx;
+
/*
* According to SMPTE 302M, an audio access unit is specified as those
* AES3 words that are associated with a corresponding video frame.
* ffmpeg
*/
- struct vidtv_s302m_ctx *ctx = e->ctx;
-
vidtv_s302m_access_unit_destroy(e);
vidtv_s302m_alloc_au(e);
struct vidtv_encoder
*vidtv_s302m_encoder_init(struct vidtv_s302m_encoder_init_args args)
{
- struct vidtv_encoder *e = kzalloc(sizeof(*e), GFP_KERNEL);
u32 priv_sz = sizeof(struct vidtv_s302m_ctx);
+ struct vidtv_s302m_ctx *ctx;
+ struct vidtv_encoder *e;
+
+ e = kzalloc(sizeof(*e), GFP_KERNEL);
+ if (!e)
+ return NULL;
e->id = S302M;
e->encoder_buf_offset = 0;
e->sample_count = 0;
- e->last_duration = 0;
e->src_buf = (args.src_buf) ? args.src_buf : NULL;
e->src_buf_sz = (args.src_buf) ? args.src_buf_sz : 0;
e->src_buf_offset = 0;
e->is_video_encoder = false;
- e->ctx = kzalloc(priv_sz, GFP_KERNEL);
+
+ ctx = kzalloc(priv_sz, GFP_KERNEL);
+ if (!ctx)
+ return NULL;
+
+ e->ctx = ctx;
+ ctx->last_duration = 0;
e->encode = vidtv_s302m_encode;
e->clear = vidtv_s302m_clear;
#define VIDTV_S302M_H
#include <linux/types.h>
-#include <asm/byteorder.h>
#include "vidtv_encoder.h"
* @enc: A pointer to the containing encoder structure.
* @frame_index: The current frame in a block
* @au_count: The total number of access units encoded up to now
+ * @last_duration: Duration of the tone currently being played
+ * @note_offset: Position at the music tone array
+ * @last_tone: Tone currently being played
*/
struct vidtv_s302m_ctx {
struct vidtv_encoder *enc;
u32 frame_index;
u32 au_count;
+ int last_duration;
+ unsigned int note_offset;
+ enum musical_notes last_tone;
};
-/**
+/*
* struct vidtv_smpte_s302m_es - s302m MPEG Elementary Stream header.
*
* See SMPTE 302M 2007 table 1.
#define pr_fmt(fmt) KBUILD_MODNAME ":%s, %d: " fmt, __func__, __LINE__
+#include <linux/math64.h>
#include <linux/printk.h>
#include <linux/ratelimit.h>
#include <linux/types.h>
-#include <linux/math64.h>
-#include <asm/byteorder.h>
-#include "vidtv_ts.h"
#include "vidtv_common.h"
+#include "vidtv_ts.h"
static u32 vidtv_ts_write_pcr_bits(u8 *to, u32 to_offset, u64 pcr)
{
#define VIDTV_TS_H
#include <linux/types.h>
-#include <asm/byteorder.h>
#define TS_SYNC_BYTE 0x47
#define TS_PACKET_LEN 188
* @dest_offset: The byte offset into the buffer.
* @pid: The TS PID for the PCR packets.
* @buf_sz: The size of the buffer in bytes.
- * @countinuity_counter: The TS continuity_counter.
+ * @continuity_counter: The TS continuity_counter.
* @pcr: A sample from the system clock.
*/
struct pcr_write_args {
* @dest_buf: The buffer to write into.
* @dest_offset: The byte offset into the buffer.
* @buf_sz: The size of the buffer in bytes.
- * @countinuity_counter: The TS continuity_counter.
+ * @continuity_counter: The TS continuity_counter.
*/
struct null_packet_write_args {
void *dest_buf;
#include <linux/errno.h>
#include <linux/i2c.h>
#include <linux/module.h>
+#include <linux/printk.h>
+#include <linux/ratelimit.h>
#include <linux/slab.h>
#include <linux/types.h>
+
#include <media/dvb_frontend.h>
-#include <linux/printk.h>
-#include <linux/ratelimit.h>
#include "vidtv_tuner.h"
#define VIDTV_TUNER_H
#include <linux/types.h>
+
#include <media/dvb_frontend.h>
#define NUM_VALID_TUNER_FREQS 8
{
if (cb->is_internal)
gen_pool_free(hdev->internal_cb_pool,
- cb->kernel_address, cb->size);
+ (uintptr_t)cb->kernel_address, cb->size);
else
hdev->asic_funcs->asic_dma_free_coherent(hdev, cb->size,
- (void *) (uintptr_t) cb->kernel_address,
- cb->bus_address);
+ cb->kernel_address, cb->bus_address);
kfree(cb);
}
return NULL;
}
- cb->kernel_address = (u64) (uintptr_t) p;
+ cb->kernel_address = p;
cb->size = cb_size;
return cb;
vma->vm_private_data = cb;
- rc = hdev->asic_funcs->cb_mmap(hdev, vma, (void *) cb->kernel_address,
+ rc = hdev->asic_funcs->cb_mmap(hdev, vma, cb->kernel_address,
cb->bus_address, cb->size);
if (rc) {
spin_lock(&cb->lock);
struct list_head pool_list;
struct list_head va_block_list;
u64 id;
- u64 kernel_address;
+ void *kernel_address;
dma_addr_t bus_address;
u32 mmap_size;
u32 size;
struct hl_hw_sob hw_sob[HL_RSVD_SOBS];
struct hl_cs_job **shadow_queue;
enum hl_queue_type queue_type;
- u64 kernel_address;
+ void *kernel_address;
dma_addr_t bus_address;
u32 pi;
atomic_t ci;
*/
struct hl_cq {
struct hl_device *hdev;
- u64 kernel_address;
+ void *kernel_address;
dma_addr_t bus_address;
u32 cq_idx;
u32 hw_queue_id;
*/
struct hl_eq {
struct hl_device *hdev;
- u64 kernel_address;
+ void *kernel_address;
dma_addr_t bus_address;
u32 ci;
};
u32 (*get_dma_desc_list_size)(struct hl_device *hdev,
struct sg_table *sgt);
void (*add_end_of_cb_packets)(struct hl_device *hdev,
- u64 kernel_address, u32 len,
+ void *kernel_address, u32 len,
u64 cq_addr, u32 cq_val, u32 msix_num,
bool eb);
void (*update_eq_ci)(struct hl_device *hdev, u32 val);
for (;;) { \
/* Verify we read updates done by other cores or by device */ \
mb(); \
- (val) = *((u32 *) (uintptr_t) (addr)); \
+ (val) = *((u32 *)(addr)); \
if (mem_written_by_device) \
(val) = le32_to_cpu(*(__le32 *) &(val)); \
if (cond) \
break; \
if (timeout_us && ktime_compare(ktime_get(), __timeout) > 0) { \
- (val) = *((u32 *) (uintptr_t) (addr)); \
+ (val) = *((u32 *)(addr)); \
if (mem_written_by_device) \
(val) = le32_to_cpu(*(__le32 *) &(val)); \
break; \
{
struct hl_bd *bd;
- bd = (struct hl_bd *) (uintptr_t) q->kernel_address;
+ bd = q->kernel_address;
bd += hl_pi_2_offset(q->pi);
bd->ctl = cpu_to_le32(ctl);
bd->len = cpu_to_le32(len);
bd.len = cpu_to_le32(job->job_cb_size);
bd.ptr = cpu_to_le64((u64) (uintptr_t) job->user_cb);
- pi = (__le64 *) (uintptr_t) (q->kernel_address +
- ((q->pi & (q->int_queue_len - 1)) * sizeof(bd)));
+ pi = q->kernel_address + (q->pi & (q->int_queue_len - 1)) * sizeof(bd);
q->pi++;
q->pi &= ((q->int_queue_len << 1) - 1);
if (!p)
return -ENOMEM;
- q->kernel_address = (u64) (uintptr_t) p;
+ q->kernel_address = p;
q->shadow_queue = kmalloc_array(HL_QUEUE_LENGTH,
sizeof(*q->shadow_queue),
if (is_cpu_queue)
hdev->asic_funcs->cpu_accessible_dma_pool_free(hdev,
HL_QUEUE_SIZE_IN_BYTES,
- (void *) (uintptr_t) q->kernel_address);
+ q->kernel_address);
else
hdev->asic_funcs->asic_dma_free_coherent(hdev,
HL_QUEUE_SIZE_IN_BYTES,
- (void *) (uintptr_t) q->kernel_address,
+ q->kernel_address,
q->bus_address);
return rc;
return -EFAULT;
}
- q->kernel_address = (u64) (uintptr_t) p;
+ q->kernel_address = p;
q->pi = 0;
atomic_set(&q->ci, 0);
if (!p)
return -ENOMEM;
- q->kernel_address = (u64) (uintptr_t) p;
+ q->kernel_address = p;
/* Make sure read/write pointers are initialized to start of queue */
atomic_set(&q->ci, 0);
if (q->queue_type == QUEUE_TYPE_CPU)
hdev->asic_funcs->cpu_accessible_dma_pool_free(hdev,
HL_QUEUE_SIZE_IN_BYTES,
- (void *) (uintptr_t) q->kernel_address);
+ q->kernel_address);
else
hdev->asic_funcs->asic_dma_free_coherent(hdev,
HL_QUEUE_SIZE_IN_BYTES,
- (void *) (uintptr_t) q->kernel_address,
+ q->kernel_address,
q->bus_address);
}
return IRQ_HANDLED;
}
- cq_base = (struct hl_cq_entry *) (uintptr_t) cq->kernel_address;
+ cq_base = cq->kernel_address;
while (1) {
bool entry_ready = ((le32_to_cpu(cq_base[cq->ci].data) &
struct hl_eq_entry *eq_base;
struct hl_eqe_work *handle_eqe_work;
- eq_base = (struct hl_eq_entry *) (uintptr_t) eq->kernel_address;
+ eq_base = eq->kernel_address;
while (1) {
bool entry_ready =
return -ENOMEM;
q->hdev = hdev;
- q->kernel_address = (u64) (uintptr_t) p;
+ q->kernel_address = p;
q->hw_queue_id = hw_queue_id;
q->ci = 0;
q->pi = 0;
void hl_cq_fini(struct hl_device *hdev, struct hl_cq *q)
{
hdev->asic_funcs->asic_dma_free_coherent(hdev, HL_CQ_SIZE_IN_BYTES,
- (void *) (uintptr_t) q->kernel_address, q->bus_address);
+ q->kernel_address,
+ q->bus_address);
}
void hl_cq_reset(struct hl_device *hdev, struct hl_cq *q)
* when the device is operational again
*/
- memset((void *) (uintptr_t) q->kernel_address, 0, HL_CQ_SIZE_IN_BYTES);
+ memset(q->kernel_address, 0, HL_CQ_SIZE_IN_BYTES);
}
/**
return -ENOMEM;
q->hdev = hdev;
- q->kernel_address = (u64) (uintptr_t) p;
+ q->kernel_address = p;
q->ci = 0;
return 0;
hdev->asic_funcs->cpu_accessible_dma_pool_free(hdev,
HL_EQ_SIZE_IN_BYTES,
- (void *) (uintptr_t) q->kernel_address);
+ q->kernel_address);
}
void hl_eq_reset(struct hl_device *hdev, struct hl_eq *q)
* when the device is operational again
*/
- memset((void *) (uintptr_t) q->kernel_address, 0, HL_EQ_SIZE_IN_BYTES);
+ memset(q->kernel_address, 0, HL_EQ_SIZE_IN_BYTES);
}
if (!cb)
return -EFAULT;
- init_tpc_mem_pkt = (struct packet_lin_dma *) (uintptr_t)
- cb->kernel_address;
+ init_tpc_mem_pkt = cb->kernel_address;
cb_size = sizeof(*init_tpc_mem_pkt);
memset(init_tpc_mem_pkt, 0, cb_size);
u16 pkt_size;
struct gaudi_packet *user_pkt;
- user_pkt = (struct gaudi_packet *) (uintptr_t)
- (parser->user_cb->kernel_address + cb_parsed_length);
+ user_pkt = parser->user_cb->kernel_address + cb_parsed_length;
pkt_id = (enum packet_id) (
(le64_to_cpu(user_pkt->header) &
u32 new_pkt_size = 0;
struct gaudi_packet *user_pkt, *kernel_pkt;
- user_pkt = (struct gaudi_packet *) (uintptr_t)
- (parser->user_cb->kernel_address + cb_parsed_length);
- kernel_pkt = (struct gaudi_packet *) (uintptr_t)
- (parser->patched_cb->kernel_address +
- cb_patched_cur_length);
+ user_pkt = parser->user_cb->kernel_address + cb_parsed_length;
+ kernel_pkt = parser->patched_cb->kernel_address +
+ cb_patched_cur_length;
pkt_id = (enum packet_id) (
(le64_to_cpu(user_pkt->header) &
* The check that parser->user_cb_size <= parser->user_cb->size was done
* in validate_queue_index().
*/
- memcpy((void *) (uintptr_t) parser->patched_cb->kernel_address,
- (void *) (uintptr_t) parser->user_cb->kernel_address,
+ memcpy(parser->patched_cb->kernel_address,
+ parser->user_cb->kernel_address,
parser->user_cb_size);
patched_cb_size = parser->patched_cb_size;
}
static void gaudi_add_end_of_cb_packets(struct hl_device *hdev,
- u64 kernel_address, u32 len,
+ void *kernel_address, u32 len,
u64 cq_addr, u32 cq_val, u32 msi_vec,
bool eb)
{
struct packet_msg_prot *cq_pkt;
u32 tmp;
- cq_pkt = (struct packet_msg_prot *) (uintptr_t)
- (kernel_address + len - (sizeof(struct packet_msg_prot) * 2));
+ cq_pkt = kernel_address + len - (sizeof(struct packet_msg_prot) * 2);
tmp = FIELD_PREP(GAUDI_PKT_CTL_OPCODE_MASK, PACKET_MSG_PROT);
tmp |= FIELD_PREP(GAUDI_PKT_CTL_MB_MASK, 1);
if (!cb)
return -EFAULT;
- lin_dma_pkt = (struct packet_lin_dma *) (uintptr_t) cb->kernel_address;
+ lin_dma_pkt = cb->kernel_address;
memset(lin_dma_pkt, 0, sizeof(*lin_dma_pkt));
cb_size = sizeof(*lin_dma_pkt);
(addr - gaudi->hbm_bar_cur_addr));
}
-static void gaudi_mmu_prepare_reg(struct hl_device *hdev, u64 reg, u32 asid)
+void gaudi_mmu_prepare_reg(struct hl_device *hdev, u64 reg, u32 asid)
{
/* mask to zero the MMBP and ASID bits */
WREG32_AND(reg, ~0x7FF);
gaudi_mmu_prepare_reg(hdev, mmMME2_ACC_WBC, asid);
gaudi_mmu_prepare_reg(hdev, mmMME3_ACC_WBC, asid);
- gaudi_mmu_prepare_reg(hdev, mmPSOC_GLOBAL_CONF_TRACE_ARUSER, asid);
- gaudi_mmu_prepare_reg(hdev, mmPSOC_GLOBAL_CONF_TRACE_AWUSER, asid);
-
hdev->asic_funcs->set_clock_gating(hdev);
mutex_unlock(&gaudi->clk_gate_mutex);
cb = job->patched_cb;
- fence_pkt = (struct packet_msg_prot *) (uintptr_t) (cb->kernel_address +
- job->job_cb_size - sizeof(struct packet_msg_prot));
+ fence_pkt = cb->kernel_address +
+ job->job_cb_size - sizeof(struct packet_msg_prot);
tmp = FIELD_PREP(GAUDI_PKT_CTL_OPCODE_MASK, PACKET_MSG_PROT);
tmp |= FIELD_PREP(GAUDI_PKT_CTL_EB_MASK, 1);
params.num_memories = 33;
params.derr = true;
params.disable_clock_gating = true;
+ extract_info_from_fw = false;
+ break;
default:
return;
}
struct packet_msg_short *pkt;
u32 value, ctl;
- pkt = (struct packet_msg_short *) (uintptr_t) cb->kernel_address;
+ pkt = cb->kernel_address;
memset(pkt, 0, sizeof(*pkt));
/* Inc by 1, Mode ADD */
u16 sob_val, u16 mon_id, u32 q_idx)
{
struct hl_cb *cb = (struct hl_cb *) data;
- void *buf = (void *) (uintptr_t) cb->kernel_address;
+ void *buf = cb->kernel_address;
u64 monitor_base, fence_addr = 0;
u32 size = 0;
u16 msg_addr_offset;
int gaudi_debug_coresight(struct hl_device *hdev, void *data);
void gaudi_halt_coresight(struct hl_device *hdev);
int gaudi_get_clk_rate(struct hl_device *hdev, u32 *cur_clk, u32 *max_clk);
+void gaudi_mmu_prepare_reg(struct hl_device *hdev, u64 reg, u32 asid);
#endif /* GAUDIP_H_ */
return -EINVAL;
}
+ gaudi_mmu_prepare_reg(hdev, mmPSOC_GLOBAL_CONF_TRACE_ARUSER,
+ hdev->compute_ctx->asid);
+ gaudi_mmu_prepare_reg(hdev, mmPSOC_GLOBAL_CONF_TRACE_AWUSER,
+ hdev->compute_ctx->asid);
+
msb = upper_32_bits(input->buffer_address) >> 8;
msb &= PSOC_GLOBAL_CONF_TRACE_ADDR_MSB_MASK;
WREG32(mmPSOC_GLOBAL_CONF_TRACE_ADDR, msb);
cb = job->patched_cb;
- fence_pkt = (struct packet_msg_prot *) (uintptr_t) (cb->kernel_address +
- job->job_cb_size - sizeof(struct packet_msg_prot));
+ fence_pkt = cb->kernel_address +
+ job->job_cb_size - sizeof(struct packet_msg_prot);
tmp = (PACKET_MSG_PROT << GOYA_PKT_CTL_OPCODE_SHIFT) |
(1 << GOYA_PKT_CTL_EB_SHIFT) |
u16 pkt_size;
struct goya_packet *user_pkt;
- user_pkt = (struct goya_packet *) (uintptr_t)
- (parser->user_cb->kernel_address + cb_parsed_length);
+ user_pkt = parser->user_cb->kernel_address + cb_parsed_length;
pkt_id = (enum packet_id) (
(le64_to_cpu(user_pkt->header) &
u32 new_pkt_size = 0;
struct goya_packet *user_pkt, *kernel_pkt;
- user_pkt = (struct goya_packet *) (uintptr_t)
- (parser->user_cb->kernel_address + cb_parsed_length);
- kernel_pkt = (struct goya_packet *) (uintptr_t)
- (parser->patched_cb->kernel_address +
- cb_patched_cur_length);
+ user_pkt = parser->user_cb->kernel_address + cb_parsed_length;
+ kernel_pkt = parser->patched_cb->kernel_address +
+ cb_patched_cur_length;
pkt_id = (enum packet_id) (
(le64_to_cpu(user_pkt->header) &
* The check that parser->user_cb_size <= parser->user_cb->size was done
* in validate_queue_index().
*/
- memcpy((void *) (uintptr_t) parser->patched_cb->kernel_address,
- (void *) (uintptr_t) parser->user_cb->kernel_address,
+ memcpy(parser->patched_cb->kernel_address,
+ parser->user_cb->kernel_address,
parser->user_cb_size);
patched_cb_size = parser->patched_cb_size;
return goya_parse_cb_no_mmu(hdev, parser);
}
-void goya_add_end_of_cb_packets(struct hl_device *hdev, u64 kernel_address,
+void goya_add_end_of_cb_packets(struct hl_device *hdev, void *kernel_address,
u32 len, u64 cq_addr, u32 cq_val, u32 msix_vec,
bool eb)
{
struct packet_msg_prot *cq_pkt;
u32 tmp;
- cq_pkt = (struct packet_msg_prot *) (uintptr_t)
- (kernel_address + len - (sizeof(struct packet_msg_prot) * 2));
+ cq_pkt = kernel_address + len - (sizeof(struct packet_msg_prot) * 2);
tmp = (PACKET_MSG_PROT << GOYA_PKT_CTL_OPCODE_SHIFT) |
(1 << GOYA_PKT_CTL_EB_SHIFT) |
if (!cb)
return -ENOMEM;
- lin_dma_pkt = (struct packet_lin_dma *) (uintptr_t) cb->kernel_address;
+ lin_dma_pkt = cb->kernel_address;
do {
memset(lin_dma_pkt, 0, sizeof(*lin_dma_pkt));
void goya_handle_eqe(struct hl_device *hdev, struct hl_eq_entry *eq_entry);
void *goya_get_events_stat(struct hl_device *hdev, bool aggregate, u32 *size);
-void goya_add_end_of_cb_packets(struct hl_device *hdev, u64 kernel_address,
+void goya_add_end_of_cb_packets(struct hl_device *hdev, void *kernel_address,
u32 len, u64 cq_addr, u32 cq_val, u32 msix_vec,
bool eb);
int goya_cs_parser(struct hl_device *hdev, struct hl_cs_parser *parser);
#define QM_ARB_ERR_MSG_EN_MASK (\
QM_ARB_ERR_MSG_EN_CHOISE_OVF_MASK |\
- QM_ARB_ERR_MSG_EN_CHOISE_WDT_MASK |\
QM_ARB_ERR_MSG_EN_AXI_LBW_ERR_MASK)
#define PCIE_AUX_FLR_CTRL_HW_CTRL_MASK 0x1
*
* @cl: host client
*
- * Return: mtu
+ * Return: mtu or 0 if client is not connected
*/
static inline size_t mei_cl_mtu(const struct mei_cl *cl)
{
- return cl->me_cl->props.max_msg_length;
+ return cl->me_cl ? cl->me_cl->props.max_msg_length : 0;
}
/**
TMIO_MASK_INIT_RCAR2);
}
-/*
- * This is a temporary workaround! This driver used 'hw_reset' wrongly and the
- * fix for that showed a regression. So, we mimic the old behaviour until the
- * proper solution is found.
- */
-static void renesas_sdhi_hw_reset(struct mmc_host *mmc)
-{
- struct tmio_mmc_host *host = mmc_priv(mmc);
- renesas_sdhi_reset(host);
-}
-
#define SH_MOBILE_SDHI_MIN_TAP_ROW 3
static int renesas_sdhi_select_tuning(struct tmio_mmc_host *host)
if (of_data && of_data->scc_offset) {
priv->scc_ctl = host->ctl + of_data->scc_offset;
host->reset = renesas_sdhi_reset;
- host->ops.hw_reset = renesas_sdhi_hw_reset;
- host->mmc->caps |= MMC_CAP_HW_RESET;
}
}
tmio_mmc_host_remove(host);
renesas_sdhi_clk_disable(host);
+ tmio_mmc_host_free(host);
return 0;
}
#define SDHCI_ARASAN_VENDOR_REGISTER 0x78
#define SDHCI_ARASAN_ITAPDLY_REGISTER 0xF0F8
+#define SDHCI_ARASAN_ITAPDLY_SEL_MASK 0xFF
+
#define SDHCI_ARASAN_OTAPDLY_REGISTER 0xF0FC
+#define SDHCI_ARASAN_OTAPDLY_SEL_MASK 0x3F
#define SDHCI_ARASAN_CQE_BASE_ADDR 0x200
#define VENDOR_ENHANCED_STROBE BIT(0)
u8 tap_delay, tap_max = 0;
int ret;
- /*
- * This is applicable for SDHCI_SPEC_300 and above
- * ZynqMP does not set phase for <=25MHz clock.
- * If degrees is zero, no need to do anything.
- */
- if (host->version < SDHCI_SPEC_300 ||
- host->timing == MMC_TIMING_LEGACY ||
- host->timing == MMC_TIMING_UHS_SDR12 || !degrees)
+ /* This is applicable for SDHCI_SPEC_300 and above */
+ if (host->version < SDHCI_SPEC_300)
return 0;
switch (host->timing) {
if (ret)
pr_err("Error setting Output Tap Delay\n");
+ /* Release DLL Reset */
+ zynqmp_pm_sd_dll_reset(node_id, PM_DLL_RESET_RELEASE);
+
return ret;
}
u8 tap_delay, tap_max = 0;
int ret;
- /*
- * This is applicable for SDHCI_SPEC_300 and above
- * ZynqMP does not set phase for <=25MHz clock.
- * If degrees is zero, no need to do anything.
- */
- if (host->version < SDHCI_SPEC_300 ||
- host->timing == MMC_TIMING_LEGACY ||
- host->timing == MMC_TIMING_UHS_SDR12 || !degrees)
+ /* This is applicable for SDHCI_SPEC_300 and above */
+ if (host->version < SDHCI_SPEC_300)
return 0;
+ /* Assert DLL Reset */
+ zynqmp_pm_sd_dll_reset(node_id, PM_DLL_RESET_ASSERT);
+
switch (host->timing) {
case MMC_TIMING_MMC_HS:
case MMC_TIMING_SD_HS:
struct sdhci_host *host = sdhci_arasan->host;
u8 tap_delay, tap_max = 0;
- /*
- * This is applicable for SDHCI_SPEC_300 and above
- * Versal does not set phase for <=25MHz clock.
- * If degrees is zero, no need to do anything.
- */
- if (host->version < SDHCI_SPEC_300 ||
- host->timing == MMC_TIMING_LEGACY ||
- host->timing == MMC_TIMING_UHS_SDR12 || !degrees)
+ /* This is applicable for SDHCI_SPEC_300 and above */
+ if (host->version < SDHCI_SPEC_300)
return 0;
switch (host->timing) {
regval = sdhci_readl(host, SDHCI_ARASAN_OTAPDLY_REGISTER);
regval |= SDHCI_OTAPDLY_ENABLE;
sdhci_writel(host, regval, SDHCI_ARASAN_OTAPDLY_REGISTER);
+ regval &= ~SDHCI_ARASAN_OTAPDLY_SEL_MASK;
regval |= tap_delay;
sdhci_writel(host, regval, SDHCI_ARASAN_OTAPDLY_REGISTER);
}
struct sdhci_host *host = sdhci_arasan->host;
u8 tap_delay, tap_max = 0;
- /*
- * This is applicable for SDHCI_SPEC_300 and above
- * Versal does not set phase for <=25MHz clock.
- * If degrees is zero, no need to do anything.
- */
- if (host->version < SDHCI_SPEC_300 ||
- host->timing == MMC_TIMING_LEGACY ||
- host->timing == MMC_TIMING_UHS_SDR12 || !degrees)
+ /* This is applicable for SDHCI_SPEC_300 and above */
+ if (host->version < SDHCI_SPEC_300)
return 0;
switch (host->timing) {
sdhci_writel(host, regval, SDHCI_ARASAN_ITAPDLY_REGISTER);
regval |= SDHCI_ITAPDLY_ENABLE;
sdhci_writel(host, regval, SDHCI_ARASAN_ITAPDLY_REGISTER);
+ regval &= ~SDHCI_ARASAN_ITAPDLY_SEL_MASK;
regval |= tap_delay;
sdhci_writel(host, regval, SDHCI_ARASAN_ITAPDLY_REGISTER);
regval &= ~SDHCI_ITAPDLY_CHGWIN;
static struct soc_device_attribute soc_unreliable_pulse_detection[] = {
{ .family = "QorIQ LX2160A", .revision = "1.0", },
+ { .family = "QorIQ LX2160A", .revision = "2.0", },
+ { .family = "QorIQ LS1028A", .revision = "1.0", },
{ },
};
}
}
+static void sdhci_intel_set_uhs_signaling(struct sdhci_host *host,
+ unsigned int timing)
+{
+ /* Set UHS timing to SDR25 for High Speed mode */
+ if (timing == MMC_TIMING_MMC_HS || timing == MMC_TIMING_SD_HS)
+ timing = MMC_TIMING_UHS_SDR25;
+ sdhci_set_uhs_signaling(host, timing);
+}
+
#define INTEL_HS400_ES_REG 0x78
#define INTEL_HS400_ES_BIT BIT(0)
.enable_dma = sdhci_pci_enable_dma,
.set_bus_width = sdhci_set_bus_width,
.reset = sdhci_reset,
- .set_uhs_signaling = sdhci_set_uhs_signaling,
+ .set_uhs_signaling = sdhci_intel_set_uhs_signaling,
.hw_reset = sdhci_pci_hw_reset,
};
.enable_dma = sdhci_pci_enable_dma,
.set_bus_width = sdhci_set_bus_width,
.reset = sdhci_cqhci_reset,
- .set_uhs_signaling = sdhci_set_uhs_signaling,
+ .set_uhs_signaling = sdhci_intel_set_uhs_signaling,
.hw_reset = sdhci_pci_hw_reset,
.irq = sdhci_cqhci_irq,
};
if (host->reset)
host->reset(host);
+ tmio_mmc_abort_dma(host);
+
if (host->pdata->flags & TMIO_MMC_SDIO_IRQ) {
sd_ctrl_write16(host, CTL_SDIO_IRQ_MASK, host->sdio_irq_mask);
sd_ctrl_write16(host, CTL_TRANSACTION_CTL, 0x0001);
/* Ready for new calls */
host->mrq = NULL;
-
- tmio_mmc_abort_dma(host);
mmc_request_done(host->mmc, mrq);
}
switch (ios->power_mode) {
case MMC_POWER_OFF:
tmio_mmc_power_off(host);
+ /* Downgrade ensures a sane state for tuning HW (e.g. SCC) */
+ if (host->mmc->ops->hs400_downgrade)
+ host->mmc->ops->hs400_downgrade(host->mmc);
host->set_clock(host, 0);
break;
case MMC_POWER_UP:
return 0;
}
+static int gpio_nand_attach_chip(struct nand_chip *chip)
+{
+ chip->ecc.engine_type = NAND_ECC_ENGINE_TYPE_SOFT;
+ chip->ecc.algo = NAND_ECC_ALGO_HAMMING;
+
+ return 0;
+}
+
static const struct nand_controller_ops gpio_nand_ops = {
.exec_op = gpio_nand_exec_op,
+ .attach_chip = gpio_nand_attach_chip,
.setup_interface = gpio_nand_setup_interface,
};
return err;
}
- this->ecc.engine_type = NAND_ECC_ENGINE_TYPE_SOFT;
- this->ecc.algo = NAND_ECC_ALGO_HAMMING;
-
platform_set_drvdata(pdev, priv);
/* Set chip enabled but write protected */
return ret;
}
+static int au1550nd_attach_chip(struct nand_chip *chip)
+{
+ chip->ecc.engine_type = NAND_ECC_ENGINE_TYPE_SOFT;
+ chip->ecc.algo = NAND_ECC_ALGO_HAMMING;
+
+ return 0;
+}
+
static const struct nand_controller_ops au1550nd_ops = {
.exec_op = au1550nd_exec_op,
+ .attach_chip = au1550nd_attach_chip,
};
static int au1550nd_probe(struct platform_device *pdev)
nand_controller_init(&ctx->controller);
ctx->controller.ops = &au1550nd_ops;
this->controller = &ctx->controller;
- this->ecc.engine_type = NAND_ECC_ENGINE_TYPE_SOFT;
- this->ecc.algo = NAND_ECC_ALGO_HAMMING;
if (pd->devwidth)
this->options |= NAND_BUSWIDTH_16;
static struct cs553x_nand_controller *controllers[4];
+static int cs553x_attach_chip(struct nand_chip *chip)
+{
+ if (chip->ecc.engine_type != NAND_ECC_ENGINE_TYPE_ON_HOST)
+ return 0;
+
+ chip->ecc.size = 256;
+ chip->ecc.bytes = 3;
+ chip->ecc.hwctl = cs_enable_hwecc;
+ chip->ecc.calculate = cs_calculate_ecc;
+ chip->ecc.correct = nand_correct_data;
+ chip->ecc.strength = 1;
+
+ return 0;
+}
+
static const struct nand_controller_ops cs553x_nand_controller_ops = {
.exec_op = cs553x_exec_op,
+ .attach_chip = cs553x_attach_chip,
};
static int __init cs553x_init_one(int cs, int mmio, unsigned long adr)
goto out_mtd;
}
- this->ecc.engine_type = NAND_ECC_ENGINE_TYPE_ON_HOST;
- this->ecc.size = 256;
- this->ecc.bytes = 3;
- this->ecc.hwctl = cs_enable_hwecc;
- this->ecc.calculate = cs_calculate_ecc;
- this->ecc.correct = nand_correct_data;
- this->ecc.strength = 1;
-
/* Enable the following for a flash based bad block table */
this->bbt_options = NAND_BBT_USE_FLASH;
if (IS_ERR(pdata))
return PTR_ERR(pdata);
+ /* Use board-specific ECC config */
+ info->chip.ecc.engine_type = pdata->engine_type;
+ info->chip.ecc.placement = pdata->ecc_placement;
+
switch (info->chip.ecc.engine_type) {
case NAND_ECC_ENGINE_TYPE_NONE:
pdata->ecc_bits = 0;
info->mask_ale = pdata->mask_ale ? : MASK_ALE;
info->mask_cle = pdata->mask_cle ? : MASK_CLE;
- /* Use board-specific ECC config */
- info->chip.ecc.engine_type = pdata->engine_type;
- info->chip.ecc.placement = pdata->ecc_placement;
-
spin_lock_irq(&davinci_nand_lock);
/* put CSxNAND into NAND mode */
return 1;
}
+static int doc200x_attach_chip(struct nand_chip *chip)
+{
+ if (chip->ecc.engine_type != NAND_ECC_ENGINE_TYPE_ON_HOST)
+ return 0;
+
+ chip->ecc.placement = NAND_ECC_PLACEMENT_INTERLEAVED;
+ chip->ecc.size = 512;
+ chip->ecc.bytes = 6;
+ chip->ecc.strength = 2;
+ chip->ecc.options = NAND_ECC_GENERIC_ERASED_CHECK;
+ chip->ecc.hwctl = doc200x_enable_hwecc;
+ chip->ecc.calculate = doc200x_calculate_ecc;
+ chip->ecc.correct = doc200x_correct_data;
+
+ return 0;
+}
+
static const struct nand_controller_ops doc200x_ops = {
.exec_op = doc200x_exec_op,
+ .attach_chip = doc200x_attach_chip,
};
static const struct nand_controller_ops doc2001plus_ops = {
.exec_op = doc2001plus_exec_op,
+ .attach_chip = doc200x_attach_chip,
};
static int __init doc_probe(unsigned long physadr)
nand->controller = &doc->base;
nand_set_controller_data(nand, doc);
- nand->ecc.hwctl = doc200x_enable_hwecc;
- nand->ecc.calculate = doc200x_calculate_ecc;
- nand->ecc.correct = doc200x_correct_data;
-
- nand->ecc.engine_type = NAND_ECC_ENGINE_TYPE_ON_HOST;
- nand->ecc.placement = NAND_ECC_PLACEMENT_INTERLEAVED;
- nand->ecc.size = 512;
- nand->ecc.bytes = 6;
- nand->ecc.strength = 2;
- nand->ecc.options = NAND_ECC_GENERIC_ERASED_CHECK;
nand->bbt_options = NAND_BBT_USE_FLASH;
/* Skip the automatic BBT scan so we can run it manually */
nand->options |= NAND_SKIP_BBTSCAN | NAND_NO_BBM_QUIRK;
struct mtd_info *mtd = nand_to_mtd(nand);
struct fsmc_nand_data *host = nand_to_fsmc(nand);
+ if (nand->ecc.engine_type == NAND_ECC_ENGINE_TYPE_INVALID)
+ nand->ecc.engine_type = NAND_ECC_ENGINE_TYPE_ON_HOST;
+
+ if (!nand->ecc.size)
+ nand->ecc.size = 512;
+
+ if (AMBA_REV_BITS(host->pid) >= 8) {
+ nand->ecc.read_page = fsmc_read_page_hwecc;
+ nand->ecc.calculate = fsmc_read_hwecc_ecc4;
+ nand->ecc.correct = fsmc_bch8_correct_data;
+ nand->ecc.bytes = 13;
+ nand->ecc.strength = 8;
+ }
+
if (AMBA_REV_BITS(host->pid) >= 8) {
switch (mtd->oobsize) {
case 16:
dev_info(host->dev, "Using 1-bit HW ECC scheme\n");
nand->ecc.calculate = fsmc_read_hwecc_ecc1;
nand->ecc.correct = nand_correct_data;
+ nand->ecc.hwctl = fsmc_enable_hwecc;
nand->ecc.bytes = 3;
nand->ecc.strength = 1;
nand->ecc.options |= NAND_ECC_SOFT_HAMMING_SM_ORDER;
mtd->dev.parent = &pdev->dev;
- /*
- * Setup default ECC mode. nand_dt_init() called from nand_scan_ident()
- * can overwrite this value if the DT provides a different value.
- */
- nand->ecc.engine_type = NAND_ECC_ENGINE_TYPE_ON_HOST;
- nand->ecc.hwctl = fsmc_enable_hwecc;
- nand->ecc.size = 512;
nand->badblockbits = 7;
if (host->mode == USE_DMA_ACCESS) {
nand->options |= NAND_KEEP_TIMINGS;
}
- if (AMBA_REV_BITS(host->pid) >= 8) {
- nand->ecc.read_page = fsmc_read_page_hwecc;
- nand->ecc.calculate = fsmc_read_hwecc_ecc4;
- nand->ecc.correct = fsmc_bch8_correct_data;
- nand->ecc.bytes = 13;
- nand->ecc.strength = 8;
- }
-
nand_controller_init(&host->base);
host->base.ops = &fsmc_nand_controller_ops;
nand->controller = &host->base;
return ret;
}
+static int gpio_nand_attach_chip(struct nand_chip *chip)
+{
+ chip->ecc.engine_type = NAND_ECC_ENGINE_TYPE_SOFT;
+ chip->ecc.algo = NAND_ECC_ALGO_HAMMING;
+
+ return 0;
+}
+
static const struct nand_controller_ops gpio_nand_ops = {
.exec_op = gpio_nand_exec_op,
+ .attach_chip = gpio_nand_attach_chip,
};
#ifdef CONFIG_OF
gpiomtd->base.ops = &gpio_nand_ops;
nand_set_flash_node(chip, pdev->dev.of_node);
- chip->ecc.engine_type = NAND_ECC_ENGINE_TYPE_SOFT;
- chip->ecc.algo = NAND_ECC_ALGO_HAMMING;
chip->options = gpiomtd->plat.options;
chip->controller = &gpiomtd->base;
struct lpc32xx_nand_host *host = nand_get_controller_data(chip);
struct device *dev = &host->pdev->dev;
+ if (chip->ecc.engine_type != NAND_ECC_ENGINE_TYPE_ON_HOST)
+ return 0;
+
host->dma_buf = devm_kzalloc(dev, mtd->writesize, GFP_KERNEL);
if (!host->dma_buf)
return -ENOMEM;
if (!host->dummy_buf)
return -ENOMEM;
- chip->ecc.engine_type = NAND_ECC_ENGINE_TYPE_ON_HOST;
chip->ecc.size = 512;
+ chip->ecc.hwctl = lpc32xx_ecc_enable;
+ chip->ecc.read_page_raw = lpc32xx_read_page;
+ chip->ecc.read_page = lpc32xx_read_page;
+ chip->ecc.write_page_raw = lpc32xx_write_page_lowlevel;
+ chip->ecc.write_page = lpc32xx_write_page_lowlevel;
+ chip->ecc.write_oob = lpc32xx_write_oob;
+ chip->ecc.read_oob = lpc32xx_read_oob;
+ chip->ecc.strength = 4;
+ chip->ecc.bytes = 10;
+
mtd_set_ooblayout(mtd, &lpc32xx_ooblayout_ops);
host->mlcsubpages = mtd->writesize / 512;
platform_set_drvdata(pdev, host);
/* Initialize function pointers */
- nand_chip->ecc.hwctl = lpc32xx_ecc_enable;
- nand_chip->ecc.read_page_raw = lpc32xx_read_page;
- nand_chip->ecc.read_page = lpc32xx_read_page;
- nand_chip->ecc.write_page_raw = lpc32xx_write_page_lowlevel;
- nand_chip->ecc.write_page = lpc32xx_write_page_lowlevel;
- nand_chip->ecc.write_oob = lpc32xx_write_oob;
- nand_chip->ecc.read_oob = lpc32xx_read_oob;
- nand_chip->ecc.strength = 4;
- nand_chip->ecc.bytes = 10;
nand_chip->legacy.waitfunc = lpc32xx_waitfunc;
nand_chip->options = NAND_NO_SUBPAGE_WRITE;
struct mtd_info *mtd = nand_to_mtd(chip);
struct lpc32xx_nand_host *host = nand_get_controller_data(chip);
+ if (chip->ecc.engine_type != NAND_ECC_ENGINE_TYPE_ON_HOST)
+ return 0;
+
/* OOB and ECC CPU and DMA work areas */
host->ecc_buf = (uint32_t *)(host->data_buf + LPC32XX_DMA_DATA_SIZE);
if (mtd->writesize <= 512)
mtd_set_ooblayout(mtd, &lpc32xx_ooblayout_ops);
+ chip->ecc.placement = NAND_ECC_PLACEMENT_INTERLEAVED;
/* These sizes remain the same regardless of page size */
chip->ecc.size = 256;
+ chip->ecc.strength = 1;
chip->ecc.bytes = LPC32XX_SLC_DEV_ECC_BYTES;
chip->ecc.prepad = 0;
chip->ecc.postpad = 0;
+ chip->ecc.read_page_raw = lpc32xx_nand_read_page_raw_syndrome;
+ chip->ecc.read_page = lpc32xx_nand_read_page_syndrome;
+ chip->ecc.write_page_raw = lpc32xx_nand_write_page_raw_syndrome;
+ chip->ecc.write_page = lpc32xx_nand_write_page_syndrome;
+ chip->ecc.write_oob = lpc32xx_nand_write_oob_syndrome;
+ chip->ecc.read_oob = lpc32xx_nand_read_oob_syndrome;
+ chip->ecc.calculate = lpc32xx_nand_ecc_calculate;
+ chip->ecc.correct = nand_correct_data;
+ chip->ecc.hwctl = lpc32xx_nand_ecc_enable;
/*
* Use a custom BBT marker setup for small page FLASH that
platform_set_drvdata(pdev, host);
/* NAND callbacks for LPC32xx SLC hardware */
- chip->ecc.engine_type = NAND_ECC_ENGINE_TYPE_ON_HOST;
- chip->ecc.placement = NAND_ECC_PLACEMENT_INTERLEAVED;
chip->legacy.read_byte = lpc32xx_nand_read_byte;
chip->legacy.read_buf = lpc32xx_nand_read_buf;
chip->legacy.write_buf = lpc32xx_nand_write_buf;
- chip->ecc.read_page_raw = lpc32xx_nand_read_page_raw_syndrome;
- chip->ecc.read_page = lpc32xx_nand_read_page_syndrome;
- chip->ecc.write_page_raw = lpc32xx_nand_write_page_raw_syndrome;
- chip->ecc.write_page = lpc32xx_nand_write_page_syndrome;
- chip->ecc.write_oob = lpc32xx_nand_write_oob_syndrome;
- chip->ecc.read_oob = lpc32xx_nand_read_oob_syndrome;
- chip->ecc.calculate = lpc32xx_nand_ecc_calculate;
- chip->ecc.correct = nand_correct_data;
- chip->ecc.strength = 1;
- chip->ecc.hwctl = lpc32xx_nand_ecc_enable;
/*
* Allocate a large enough buffer for a single huge page plus
#define NFC_TIMEOUT (HZ / 10) /* 1/10 s */
struct mpc5121_nfc_prv {
+ struct nand_controller controller;
struct nand_chip chip;
int irq;
void __iomem *regs;
iounmap(prv->csreg);
}
+static int mpc5121_nfc_attach_chip(struct nand_chip *chip)
+{
+ chip->ecc.engine_type = NAND_ECC_ENGINE_TYPE_SOFT;
+ chip->ecc.algo = NAND_ECC_ALGO_HAMMING;
+
+ return 0;
+}
+
+static const struct nand_controller_ops mpc5121_nfc_ops = {
+ .attach_chip = mpc5121_nfc_attach_chip,
+};
+
static int mpc5121_nfc_probe(struct platform_device *op)
{
struct device_node *dn = op->dev.of_node;
chip = &prv->chip;
mtd = nand_to_mtd(chip);
+ nand_controller_init(&prv->controller);
+ prv->controller.ops = &mpc5121_nfc_ops;
+ chip->controller = &prv->controller;
+
mtd->dev.parent = dev;
nand_set_controller_data(chip, prv);
nand_set_flash_node(chip, dn);
chip->legacy.set_features = nand_get_set_features_notsupp;
chip->legacy.get_features = nand_get_set_features_notsupp;
chip->bbt_options = NAND_BBT_USE_FLASH;
- chip->ecc.engine_type = NAND_ECC_ENGINE_TYPE_SOFT;
- chip->ecc.algo = NAND_ECC_ALGO_HAMMING;
/* Support external chip-select logic on ADS5121 board */
if (of_machine_is_compatible("fsl,mpc5121ads")) {
#include <linux/platform_data/mtd-orion_nand.h>
struct orion_nand_info {
+ struct nand_controller controller;
struct nand_chip chip;
struct clk *clk;
};
buf[i++] = readb(io_base);
}
+static int orion_nand_attach_chip(struct nand_chip *chip)
+{
+ chip->ecc.engine_type = NAND_ECC_ENGINE_TYPE_SOFT;
+ chip->ecc.algo = NAND_ECC_ALGO_HAMMING;
+
+ return 0;
+}
+
+static const struct nand_controller_ops orion_nand_ops = {
+ .attach_chip = orion_nand_attach_chip,
+};
+
static int __init orion_nand_probe(struct platform_device *pdev)
{
struct orion_nand_info *info;
nc = &info->chip;
mtd = nand_to_mtd(nc);
+ nand_controller_init(&info->controller);
+ info->controller.ops = &orion_nand_ops;
+ nc->controller = &info->controller;
+
res = platform_get_resource(pdev, IORESOURCE_MEM, 0);
io_base = devm_ioremap_resource(&pdev->dev, res);
nc->legacy.IO_ADDR_R = nc->legacy.IO_ADDR_W = io_base;
nc->legacy.cmd_ctrl = orion_nand_cmd_ctrl;
nc->legacy.read_buf = orion_nand_read_buf;
- nc->ecc.engine_type = NAND_ECC_ENGINE_TYPE_SOFT;
- nc->ecc.algo = NAND_ECC_ALGO_HAMMING;
if (board->chip_delay)
nc->legacy.chip_delay = board->chip_delay;
static unsigned int lpcctl;
static struct mtd_info *pasemi_nand_mtd;
+static struct nand_controller controller;
static const char driver_name[] = "pasemi-nand";
static void pasemi_read_buf(struct nand_chip *chip, u_char *buf, int len)
return !!(inl(lpcctl) & LBICTRL_LPCCTL_NR);
}
+static int pasemi_attach_chip(struct nand_chip *chip)
+{
+ chip->ecc.engine_type = NAND_ECC_ENGINE_TYPE_SOFT;
+ chip->ecc.algo = NAND_ECC_ALGO_HAMMING;
+
+ return 0;
+}
+
+static const struct nand_controller_ops pasemi_ops = {
+ .attach_chip = pasemi_attach_chip,
+};
+
static int pasemi_nand_probe(struct platform_device *ofdev)
{
struct device *dev = &ofdev->dev;
goto out;
}
+ controller.ops = &pasemi_ops;
+ nand_controller_init(&controller);
+ chip->controller = &controller;
+
pasemi_nand_mtd = nand_to_mtd(chip);
/* Link the private data with the MTD structure */
chip->legacy.read_buf = pasemi_read_buf;
chip->legacy.write_buf = pasemi_write_buf;
chip->legacy.chip_delay = 0;
- chip->ecc.engine_type = NAND_ECC_ENGINE_TYPE_SOFT;
- chip->ecc.algo = NAND_ECC_ALGO_HAMMING;
/* Enable the following for a flash based bad block table */
chip->bbt_options = NAND_BBT_USE_FLASH;
#include <linux/mtd/platnand.h>
struct plat_nand_data {
+ struct nand_controller controller;
struct nand_chip chip;
void __iomem *io_base;
};
+static int plat_nand_attach_chip(struct nand_chip *chip)
+{
+ chip->ecc.engine_type = NAND_ECC_ENGINE_TYPE_SOFT;
+ chip->ecc.algo = NAND_ECC_ALGO_HAMMING;
+
+ return 0;
+}
+
+static const struct nand_controller_ops plat_nand_ops = {
+ .attach_chip = plat_nand_attach_chip,
+};
+
/*
* Probe for the NAND device.
*/
if (!data)
return -ENOMEM;
+ data->controller.ops = &plat_nand_ops;
+ nand_controller_init(&data->controller);
+ data->chip.controller = &data->controller;
+
res = platform_get_resource(pdev, IORESOURCE_MEM, 0);
data->io_base = devm_ioremap_resource(&pdev->dev, res);
if (IS_ERR(data->io_base))
data->chip.options |= pdata->chip.options;
data->chip.bbt_options |= pdata->chip.bbt_options;
- data->chip.ecc.engine_type = NAND_ECC_ENGINE_TYPE_SOFT;
- data->chip.ecc.algo = NAND_ECC_ALGO_HAMMING;
-
platform_set_drvdata(pdev, data);
/* Handle any platform specific setup */
return ret;
}
+static int r852_attach_chip(struct nand_chip *chip)
+{
+ if (chip->ecc.engine_type != NAND_ECC_ENGINE_TYPE_ON_HOST)
+ return 0;
+
+ chip->ecc.placement = NAND_ECC_PLACEMENT_INTERLEAVED;
+ chip->ecc.size = R852_DMA_LEN;
+ chip->ecc.bytes = SM_OOB_SIZE;
+ chip->ecc.strength = 2;
+ chip->ecc.hwctl = r852_ecc_hwctl;
+ chip->ecc.calculate = r852_ecc_calculate;
+ chip->ecc.correct = r852_ecc_correct;
+
+ /* TODO: hack */
+ chip->ecc.read_oob = r852_read_oob;
+
+ return 0;
+}
+
+static const struct nand_controller_ops r852_ops = {
+ .attach_chip = r852_attach_chip,
+};
+
static int r852_probe(struct pci_dev *pci_dev, const struct pci_device_id *id)
{
int error;
chip->legacy.read_buf = r852_read_buf;
chip->legacy.write_buf = r852_write_buf;
- /* ecc */
- chip->ecc.engine_type = NAND_ECC_ENGINE_TYPE_ON_HOST;
- chip->ecc.placement = NAND_ECC_PLACEMENT_INTERLEAVED;
- chip->ecc.size = R852_DMA_LEN;
- chip->ecc.bytes = SM_OOB_SIZE;
- chip->ecc.strength = 2;
- chip->ecc.hwctl = r852_ecc_hwctl;
- chip->ecc.calculate = r852_ecc_calculate;
- chip->ecc.correct = r852_ecc_correct;
-
- /* TODO: hack */
- chip->ecc.read_oob = r852_read_oob;
-
/* init our device structure */
dev = kzalloc(sizeof(struct r852_device), GFP_KERNEL);
dev->pci_dev = pci_dev;
pci_set_drvdata(pci_dev, dev);
+ nand_controller_init(&dev->controller);
+ dev->controller.ops = &r852_ops;
+ chip->controller = &dev->controller;
+
dev->bounce_buffer = dma_alloc_coherent(&pci_dev->dev, R852_DMA_LEN,
&dev->phys_bounce_buffer, GFP_KERNEL);
#define DMA_MEMORY 1
struct r852_device {
+ struct nand_controller controller;
void __iomem *mmio; /* mmio */
struct nand_chip *chip; /* nand chip backpointer */
struct pci_dev *pci_dev; /* pci backpointer */
#include <linux/io.h>
struct sharpsl_nand {
+ struct nand_controller controller;
struct nand_chip chip;
void __iomem *io;
return readb(sharpsl->io + ECCCNTR) != 0;
}
+static int sharpsl_attach_chip(struct nand_chip *chip)
+{
+ if (chip->ecc.engine_type != NAND_ECC_ENGINE_TYPE_ON_HOST)
+ return 0;
+
+ chip->ecc.size = 256;
+ chip->ecc.bytes = 3;
+ chip->ecc.strength = 1;
+ chip->ecc.hwctl = sharpsl_nand_enable_hwecc;
+ chip->ecc.calculate = sharpsl_nand_calculate_ecc;
+ chip->ecc.correct = nand_correct_data;
+
+ return 0;
+}
+
+static const struct nand_controller_ops sharpsl_ops = {
+ .attach_chip = sharpsl_attach_chip,
+};
+
/*
* Main initialization routine
*/
/* Get pointer to private data */
this = (struct nand_chip *)(&sharpsl->chip);
+ nand_controller_init(&sharpsl->controller);
+ sharpsl->controller.ops = &sharpsl_ops;
+ this->controller = &sharpsl->controller;
+
/* Link the private data with the MTD structure */
mtd = nand_to_mtd(this);
mtd->dev.parent = &pdev->dev;
this->legacy.dev_ready = sharpsl_nand_dev_ready;
/* 15 us command delay time */
this->legacy.chip_delay = 15;
- /* set eccmode using hardware ECC */
- this->ecc.engine_type = NAND_ECC_ENGINE_TYPE_ON_HOST;
- this->ecc.size = 256;
- this->ecc.bytes = 3;
- this->ecc.strength = 1;
this->badblock_pattern = data->badblock_pattern;
- this->ecc.hwctl = sharpsl_nand_enable_hwecc;
- this->ecc.calculate = sharpsl_nand_calculate_ecc;
- this->ecc.correct = nand_correct_data;
/* Scan to find existence of the device */
err = nand_scan(this, 1);
#define FPGA_NAND_DATA_SHIFT 16
struct socrates_nand_host {
+ struct nand_controller controller;
struct nand_chip nand_chip;
void __iomem *io_base;
struct device *dev;
return 1;
}
+static int socrates_attach_chip(struct nand_chip *chip)
+{
+ chip->ecc.engine_type = NAND_ECC_ENGINE_TYPE_SOFT;
+ chip->ecc.algo = NAND_ECC_ALGO_HAMMING;
+
+ return 0;
+}
+
+static const struct nand_controller_ops socrates_ops = {
+ .attach_chip = socrates_attach_chip,
+};
+
/*
* Probe for the NAND device.
*/
mtd = nand_to_mtd(nand_chip);
host->dev = &ofdev->dev;
+ nand_controller_init(&host->controller);
+ host->controller.ops = &socrates_ops;
+ nand_chip->controller = &host->controller;
+
/* link the private data structures */
nand_set_controller_data(nand_chip, host);
nand_set_flash_node(nand_chip, ofdev->dev.of_node);
nand_chip->legacy.read_buf = socrates_nand_read_buf;
nand_chip->legacy.dev_ready = socrates_nand_device_ready;
- /* enable ECC */
- nand_chip->ecc.engine_type = NAND_ECC_ENGINE_TYPE_SOFT;
- nand_chip->ecc.algo = NAND_ECC_ALGO_HAMMING;
-
/* TODO: I have no idea what real delay is. */
nand_chip->legacy.chip_delay = 20; /* 20us command delay time */
/*--------------------------------------------------------------------------*/
struct tmio_nand {
+ struct nand_controller controller;
struct nand_chip chip;
struct completion comp;
cell->disable(dev);
}
+static int tmio_attach_chip(struct nand_chip *chip)
+{
+ if (chip->ecc.engine_type != NAND_ECC_ENGINE_TYPE_ON_HOST)
+ return 0;
+
+ chip->ecc.size = 512;
+ chip->ecc.bytes = 6;
+ chip->ecc.strength = 2;
+ chip->ecc.hwctl = tmio_nand_enable_hwecc;
+ chip->ecc.calculate = tmio_nand_calculate_ecc;
+ chip->ecc.correct = tmio_nand_correct_data;
+
+ return 0;
+}
+
+static const struct nand_controller_ops tmio_ops = {
+ .attach_chip = tmio_attach_chip,
+};
+
static int tmio_probe(struct platform_device *dev)
{
struct tmio_nand_data *data = dev_get_platdata(&dev->dev);
mtd->name = "tmio-nand";
mtd->dev.parent = &dev->dev;
+ nand_controller_init(&tmio->controller);
+ tmio->controller.ops = &tmio_ops;
+ nand_chip->controller = &tmio->controller;
+
tmio->ccr = devm_ioremap(&dev->dev, ccr->start, resource_size(ccr));
if (!tmio->ccr)
return -EIO;
nand_chip->legacy.write_buf = tmio_nand_write_buf;
nand_chip->legacy.read_buf = tmio_nand_read_buf;
- /* set eccmode using hardware ECC */
- nand_chip->ecc.engine_type = NAND_ECC_ENGINE_TYPE_ON_HOST;
- nand_chip->ecc.size = 512;
- nand_chip->ecc.bytes = 6;
- nand_chip->ecc.strength = 2;
- nand_chip->ecc.hwctl = tmio_nand_enable_hwecc;
- nand_chip->ecc.calculate = tmio_nand_calculate_ecc;
- nand_chip->ecc.correct = tmio_nand_correct_data;
-
if (data)
nand_chip->badblock_pattern = data->badblock_pattern;
{
struct mtd_info *mtd = nand_to_mtd(chip);
+ if (chip->ecc.engine_type != NAND_ECC_ENGINE_TYPE_ON_HOST)
+ return 0;
+
+ chip->ecc.strength = 1;
+
if (mtd->writesize >= 512) {
chip->ecc.size = 512;
chip->ecc.bytes = 6;
chip->ecc.bytes = 3;
}
+ chip->ecc.calculate = txx9ndfmc_calculate_ecc;
+ chip->ecc.correct = txx9ndfmc_correct_data;
+ chip->ecc.hwctl = txx9ndfmc_enable_hwecc;
+
return 0;
}
chip->legacy.write_buf = txx9ndfmc_write_buf;
chip->legacy.cmd_ctrl = txx9ndfmc_cmd_ctrl;
chip->legacy.dev_ready = txx9ndfmc_dev_ready;
- chip->ecc.calculate = txx9ndfmc_calculate_ecc;
- chip->ecc.correct = txx9ndfmc_correct_data;
- chip->ecc.hwctl = txx9ndfmc_enable_hwecc;
- chip->ecc.engine_type = NAND_ECC_ENGINE_TYPE_ON_HOST;
- chip->ecc.strength = 1;
chip->legacy.chip_delay = 100;
chip->controller = &drvdata->controller;
#define NAND_CON_NANDM 1
struct xway_nand_data {
+ struct nand_controller controller;
struct nand_chip chip;
unsigned long csflags;
void __iomem *nandaddr;
xway_writeb(nand_to_mtd(chip), NAND_WRITE_DATA, buf[i]);
}
+static int xway_attach_chip(struct nand_chip *chip)
+{
+ chip->ecc.engine_type = NAND_ECC_ENGINE_TYPE_SOFT;
+ chip->ecc.algo = NAND_ECC_ALGO_HAMMING;
+
+ return 0;
+}
+
+static const struct nand_controller_ops xway_nand_ops = {
+ .attach_chip = xway_attach_chip,
+};
+
/*
* Probe for the NAND device.
*/
data->chip.legacy.read_byte = xway_read_byte;
data->chip.legacy.chip_delay = 30;
- data->chip.ecc.engine_type = NAND_ECC_ENGINE_TYPE_SOFT;
- data->chip.ecc.algo = NAND_ECC_ALGO_HAMMING;
+ nand_controller_init(&data->controller);
+ data->controller.ops = &xway_nand_ops;
+ data->chip.controller = &data->controller;
platform_set_drvdata(pdev, data);
nand_set_controller_data(&data->chip, data);
};
/* Info for udev, that this is a virtual tunnel endpoint */
-static struct device_type bareudp_type = {
+static const struct device_type bareudp_type = {
.name = "bareudp",
};
slave->dev->flags &= ~IFF_SLAVE;
}
-static struct slave *bond_alloc_slave(struct bonding *bond)
+static void slave_kobj_release(struct kobject *kobj)
+{
+ struct slave *slave = to_slave(kobj);
+ struct bonding *bond = bond_get_bond_by_slave(slave);
+
+ cancel_delayed_work_sync(&slave->notify_work);
+ if (BOND_MODE(bond) == BOND_MODE_8023AD)
+ kfree(SLAVE_AD_INFO(slave));
+
+ kfree(slave);
+}
+
+static struct kobj_type slave_ktype = {
+ .release = slave_kobj_release,
+#ifdef CONFIG_SYSFS
+ .sysfs_ops = &slave_sysfs_ops,
+#endif
+};
+
+static int bond_kobj_init(struct slave *slave)
+{
+ int err;
+
+ err = kobject_init_and_add(&slave->kobj, &slave_ktype,
+ &(slave->dev->dev.kobj), "bonding_slave");
+ if (err)
+ kobject_put(&slave->kobj);
+
+ return err;
+}
+
+static struct slave *bond_alloc_slave(struct bonding *bond,
+ struct net_device *slave_dev)
{
struct slave *slave = NULL;
if (!slave)
return NULL;
+ slave->bond = bond;
+ slave->dev = slave_dev;
+
+ if (bond_kobj_init(slave))
+ return NULL;
+
if (BOND_MODE(bond) == BOND_MODE_8023AD) {
SLAVE_AD_INFO(slave) = kzalloc(sizeof(struct ad_slave_info),
GFP_KERNEL);
if (!SLAVE_AD_INFO(slave)) {
- kfree(slave);
+ kobject_put(&slave->kobj);
return NULL;
}
}
return slave;
}
-static void bond_free_slave(struct slave *slave)
-{
- struct bonding *bond = bond_get_bond_by_slave(slave);
-
- cancel_delayed_work_sync(&slave->notify_work);
- if (BOND_MODE(bond) == BOND_MODE_8023AD)
- kfree(SLAVE_AD_INFO(slave));
-
- kfree(slave);
-}
-
static void bond_fill_ifbond(struct bonding *bond, struct ifbond *info)
{
info->bond_mode = BOND_MODE(bond);
goto err_undo_flags;
}
- new_slave = bond_alloc_slave(bond);
+ new_slave = bond_alloc_slave(bond, slave_dev);
if (!new_slave) {
res = -ENOMEM;
goto err_undo_flags;
}
- new_slave->bond = bond;
- new_slave->dev = slave_dev;
/* Set the new_slave's queue_id to be zero. Queue ID mapping
* is set via sysfs or module option if desired.
*/
dev_set_mtu(slave_dev, new_slave->original_mtu);
err_free:
- bond_free_slave(new_slave);
+ kobject_put(&new_slave->kobj);
err_undo_flags:
/* Enslave of first slave has failed and we need to fix master's mac */
if (!netif_is_bond_master(slave_dev))
slave_dev->priv_flags &= ~IFF_BONDING;
- bond_free_slave(slave);
+ kobject_put(&slave->kobj);
return 0;
}
// SPDX-License-Identifier: GPL-2.0
#include <linux/proc_fs.h>
+#include <linux/ethtool.h>
#include <linux/export.h>
#include <net/net_namespace.h>
#include <net/netns/generic.h>
};
#define to_slave_attr(_at) container_of(_at, struct slave_attribute, attr)
-#define to_slave(obj) container_of(obj, struct slave, kobj)
static ssize_t slave_show(struct kobject *kobj,
struct attribute *attr, char *buf)
return slave_attr->show(slave, buf);
}
-static const struct sysfs_ops slave_sysfs_ops = {
+const struct sysfs_ops slave_sysfs_ops = {
.show = slave_show,
};
-static struct kobj_type slave_ktype = {
-#ifdef CONFIG_SYSFS
- .sysfs_ops = &slave_sysfs_ops,
-#endif
-};
-
int bond_sysfs_slave_add(struct slave *slave)
{
const struct slave_attribute **a;
int err;
- err = kobject_init_and_add(&slave->kobj, &slave_ktype,
- &(slave->dev->dev.kobj), "bonding_slave");
- if (err) {
- kobject_put(&slave->kobj);
- return err;
- }
-
for (a = slave_attrs; *a; ++a) {
err = sysfs_create_file(&slave->kobj, &((*a)->attr));
if (err) {
for (a = slave_attrs; *a; ++a)
sysfs_remove_file(&slave->kobj, &((*a)->attr));
-
- kobject_put(&slave->kobj);
}
}
reg_mid = at91_can_id_to_reg_mid(cf->can_id);
reg_mcr = ((cf->can_id & CAN_RTR_FLAG) ? AT91_MCR_MRTR : 0) |
- (cf->can_dlc << 16) | AT91_MCR_MTCR;
+ (cf->len << 16) | AT91_MCR_MTCR;
/* disable MB while writing ID (see datasheet) */
set_mb_mode(priv, mb, AT91_MB_MODE_DISABLED);
/* This triggers transmission */
at91_write(priv, AT91_MCR(mb), reg_mcr);
- stats->tx_bytes += cf->can_dlc;
+ stats->tx_bytes += cf->len;
/* _NOTE_: subtract AT91_MB_TX_FIRST offset from mb! */
can_put_echo_skb(skb, dev, mb - get_mb_tx_first(priv));
cf->data[1] = CAN_ERR_CRTL_RX_OVERFLOW;
stats->rx_packets++;
- stats->rx_bytes += cf->can_dlc;
+ stats->rx_bytes += cf->len;
netif_receive_skb(skb);
}
cf->can_id = (reg_mid >> 18) & CAN_SFF_MASK;
reg_msr = at91_read(priv, AT91_MSR(mb));
- cf->can_dlc = get_can_dlc((reg_msr >> 16) & 0xf);
+ cf->len = can_cc_dlc2len((reg_msr >> 16) & 0xf);
if (reg_msr & AT91_MSR_MRTR)
cf->can_id |= CAN_RTR_FLAG;
at91_read_mb(dev, mb, cf);
stats->rx_packets++;
- stats->rx_bytes += cf->can_dlc;
+ stats->rx_bytes += cf->len;
netif_receive_skb(skb);
can_led_event(dev, CAN_LED_EVENT_RX);
at91_poll_err_frame(dev, cf, reg_sr);
dev->stats.rx_packets++;
- dev->stats.rx_bytes += cf->can_dlc;
+ dev->stats.rx_bytes += cf->len;
netif_receive_skb(skb);
return 1;
at91_irq_err_state(dev, cf, new_state);
dev->stats.rx_packets++;
- dev->stats.rx_bytes += cf->can_dlc;
+ dev->stats.rx_bytes += cf->len;
netif_rx(skb);
priv->can.state = new_state;
struct can_frame *frame, int idx)
{
struct c_can_priv *priv = netdev_priv(dev);
- u16 ctrl = IF_MCONT_TX | frame->can_dlc;
+ u16 ctrl = IF_MCONT_TX | frame->len;
bool rtr = frame->can_id & CAN_RTR_FLAG;
u32 arb = IF_ARB_MSGVAL;
int i;
if (priv->type == BOSCH_D_CAN) {
u32 data = 0, dreg = C_CAN_IFACE(DATA1_REG, iface);
- for (i = 0; i < frame->can_dlc; i += 4, dreg += 2) {
+ for (i = 0; i < frame->len; i += 4, dreg += 2) {
data = (u32)frame->data[i];
data |= (u32)frame->data[i + 1] << 8;
data |= (u32)frame->data[i + 2] << 16;
priv->write_reg32(priv, dreg, data);
}
} else {
- for (i = 0; i < frame->can_dlc; i += 2) {
+ for (i = 0; i < frame->len; i += 2) {
priv->write_reg(priv,
C_CAN_IFACE(DATA1_REG, iface) + i / 2,
frame->data[i] |
return -ENOMEM;
}
- frame->can_dlc = get_can_dlc(ctrl & 0x0F);
+ frame->len = can_cc_dlc2len(ctrl & 0x0F);
arb = priv->read_reg32(priv, C_CAN_IFACE(ARB1_REG, iface));
int i, dreg = C_CAN_IFACE(DATA1_REG, iface);
if (priv->type == BOSCH_D_CAN) {
- for (i = 0; i < frame->can_dlc; i += 4, dreg += 2) {
+ for (i = 0; i < frame->len; i += 4, dreg += 2) {
data = priv->read_reg32(priv, dreg);
frame->data[i] = data;
frame->data[i + 1] = data >> 8;
frame->data[i + 3] = data >> 24;
}
} else {
- for (i = 0; i < frame->can_dlc; i += 2, dreg++) {
+ for (i = 0; i < frame->len; i += 2, dreg++) {
data = priv->read_reg(priv, dreg);
frame->data[i] = data;
frame->data[i + 1] = data >> 8;
}
stats->rx_packets++;
- stats->rx_bytes += frame->can_dlc;
+ stats->rx_bytes += frame->len;
netif_receive_skb(skb);
return 0;
* transmit as we might race against do_tx().
*/
c_can_setup_tx_object(dev, IF_TX, frame, idx);
- priv->dlc[idx] = frame->can_dlc;
+ priv->dlc[idx] = frame->len;
can_put_echo_skb(skb, dev, idx);
/* Update the active bits */
}
stats->rx_packets++;
- stats->rx_bytes += cf->can_dlc;
+ stats->rx_bytes += cf->len;
netif_receive_skb(skb);
return 1;
}
stats->rx_packets++;
- stats->rx_bytes += cf->can_dlc;
+ stats->rx_bytes += cf->len;
netif_receive_skb(skb);
return 1;
}
time_after(time_out, jiffies))
cpu_relax();
- if (time_after(jiffies, time_out))
- return -ETIMEDOUT;
+ if (time_after(jiffies, time_out)) {
+ ret = -ETIMEDOUT;
+ goto err_out;
+ }
ret = c_can_start(dev);
- if (!ret)
- c_can_irq_control(priv, true);
+ if (ret)
+ goto err_out;
+
+ c_can_irq_control(priv, true);
+
+ return 0;
+
+err_out:
+ c_can_reset_ram(priv, false);
+ c_can_pm_runtime_put_sync(priv);
return ret;
}
u32 id;
int i;
- dlc = cf->can_dlc;
+ dlc = cf->len;
id = cf->can_id;
rtr = cf->can_id & CAN_RTR_FLAG ? 0 : MSGCFG_DIR;
cf->can_id = CAN_RTR_FLAG;
if (config & MSGCFG_XTD)
cf->can_id |= CAN_EFF_FLAG;
- cf->can_dlc = 0;
+ cf->len = 0;
} else {
if (config & MSGCFG_XTD) {
id = cc770_read_reg(priv, msgobj[mo].id[3]);
}
cf->can_id = id;
- cf->can_dlc = get_can_dlc((config & 0xf0) >> 4);
- for (i = 0; i < cf->can_dlc; i++)
+ cf->len = can_cc_dlc2len((config & 0xf0) >> 4);
+ for (i = 0; i < cf->len; i++)
cf->data[i] = cc770_read_reg(priv, msgobj[mo].data[i]);
}
stats->rx_packets++;
- stats->rx_bytes += cf->can_dlc;
+ stats->rx_bytes += cf->len;
netif_rx(skb);
}
stats->rx_packets++;
- stats->rx_bytes += cf->can_dlc;
+ stats->rx_bytes += cf->len;
netif_rx(skb);
return 0;
}
cf = (struct can_frame *)priv->tx_skb->data;
- stats->tx_bytes += cf->can_dlc;
+ stats->tx_bytes += cf->len;
stats->tx_packets++;
can_put_echo_skb(priv->tx_skb, dev, 0);
static const u8 dlc2len[] = {0, 1, 2, 3, 4, 5, 6, 7,
8, 12, 16, 20, 24, 32, 48, 64};
-/* get data length from can_dlc with sanitized can_dlc */
-u8 can_dlc2len(u8 can_dlc)
+/* get data length from raw data length code (DLC) */
+u8 can_fd_dlc2len(u8 dlc)
{
- return dlc2len[can_dlc & 0x0F];
+ return dlc2len[dlc & 0x0F];
}
-EXPORT_SYMBOL_GPL(can_dlc2len);
+EXPORT_SYMBOL_GPL(can_fd_dlc2len);
static const u8 len2dlc[] = {0, 1, 2, 3, 4, 5, 6, 7, 8, /* 0 - 8 */
9, 9, 9, 9, /* 9 - 12 */
15, 15, 15, 15, 15, 15, 15, 15}; /* 57 - 64 */
/* map the sanitized data length to an appropriate data length code */
-u8 can_len2dlc(u8 len)
+u8 can_fd_len2dlc(u8 len)
{
if (unlikely(len > 64))
return 0xF;
return len2dlc[len];
}
-EXPORT_SYMBOL_GPL(can_len2dlc);
+EXPORT_SYMBOL_GPL(can_fd_len2dlc);
#ifdef CONFIG_CAN_CALC_BITTIMING
#define CAN_CALC_MAX_ERROR 50 /* in one-tenth of a percent */
cf->can_id |= CAN_ERR_RESTARTED;
- netif_rx(skb);
+ netif_rx_ni(skb);
stats->rx_packets++;
- stats->rx_bytes += cf->can_dlc;
+ stats->rx_bytes += cf->len;
restart:
netdev_dbg(dev, "restarted\n");
return NULL;
(*cf)->can_id = CAN_ERR_FLAG;
- (*cf)->can_dlc = CAN_ERR_DLC;
+ (*cf)->len = CAN_ERR_DLC;
return skb;
}
#define FLEXCAN_QUIRK_BROKEN_PERR_STATE BIT(6)
/* default to BE register access */
#define FLEXCAN_QUIRK_DEFAULT_BIG_ENDIAN BIT(7)
-/* Setup stop mode to support wakeup */
-#define FLEXCAN_QUIRK_SETUP_STOP_MODE BIT(8)
+/* Setup stop mode with GPR to support wakeup */
+#define FLEXCAN_QUIRK_SETUP_STOP_MODE_GPR BIT(8)
/* Support CAN-FD mode */
#define FLEXCAN_QUIRK_SUPPORT_FD BIT(9)
/* support memory detection and correction */
static const struct flexcan_devtype_data fsl_imx6q_devtype_data = {
.quirks = FLEXCAN_QUIRK_DISABLE_RXFG | FLEXCAN_QUIRK_ENABLE_EACEN_RRS |
FLEXCAN_QUIRK_USE_OFF_TIMESTAMP | FLEXCAN_QUIRK_BROKEN_PERR_STATE |
- FLEXCAN_QUIRK_SETUP_STOP_MODE,
+ FLEXCAN_QUIRK_SETUP_STOP_MODE_GPR,
};
static const struct flexcan_devtype_data fsl_imx8qm_devtype_data = {
static struct flexcan_devtype_data fsl_imx8mp_devtype_data = {
.quirks = FLEXCAN_QUIRK_DISABLE_RXFG | FLEXCAN_QUIRK_ENABLE_EACEN_RRS |
FLEXCAN_QUIRK_DISABLE_MECR | FLEXCAN_QUIRK_USE_OFF_TIMESTAMP |
- FLEXCAN_QUIRK_BROKEN_PERR_STATE | FLEXCAN_QUIRK_SETUP_STOP_MODE |
+ FLEXCAN_QUIRK_BROKEN_PERR_STATE | FLEXCAN_QUIRK_SETUP_STOP_MODE_GPR |
FLEXCAN_QUIRK_SUPPORT_FD | FLEXCAN_QUIRK_SUPPORT_ECC,
};
int err;
err = pm_runtime_get_sync(priv->dev);
- if (err < 0)
+ if (err < 0) {
+ pm_runtime_put_noidle(priv->dev);
return err;
+ }
err = __flexcan_get_berr_counter(dev, bec);
struct canfd_frame *cfd = (struct canfd_frame *)skb->data;
u32 can_id;
u32 data;
- u32 ctrl = FLEXCAN_MB_CODE_TX_DATA | ((can_len2dlc(cfd->len)) << 16);
+ u32 ctrl = FLEXCAN_MB_CODE_TX_DATA | ((can_fd_len2dlc(cfd->len)) << 16);
int i;
if (can_dropped_invalid_skb(dev, skb))
cfd->can_id = (reg_id >> 18) & CAN_SFF_MASK;
if (reg_ctrl & FLEXCAN_MB_CNT_EDL) {
- cfd->len = can_dlc2len(get_canfd_dlc((reg_ctrl >> 16) & 0xf));
+ cfd->len = can_fd_dlc2len((reg_ctrl >> 16) & 0xf);
if (reg_ctrl & FLEXCAN_MB_CNT_BRS)
cfd->flags |= CANFD_BRS;
} else {
- cfd->len = get_can_dlc((reg_ctrl >> 16) & 0xf);
+ cfd->len = can_cc_dlc2len((reg_ctrl >> 16) & 0xf);
if (reg_ctrl & FLEXCAN_MB_CNT_RTR)
cfd->can_id |= CAN_RTR_FLAG;
priv->write(reg_ctrl2, ®s->ctrl2);
}
+static int flexcan_rx_offload_setup(struct net_device *dev)
+{
+ struct flexcan_priv *priv = netdev_priv(dev);
+ int err;
+
+ if (priv->can.ctrlmode & CAN_CTRLMODE_FD)
+ priv->mb_size = sizeof(struct flexcan_mb) + CANFD_MAX_DLEN;
+ else
+ priv->mb_size = sizeof(struct flexcan_mb) + CAN_MAX_DLEN;
+ priv->mb_count = (sizeof(priv->regs->mb[0]) / priv->mb_size) +
+ (sizeof(priv->regs->mb[1]) / priv->mb_size);
+
+ if (priv->devtype_data->quirks & FLEXCAN_QUIRK_USE_OFF_TIMESTAMP)
+ priv->tx_mb_reserved =
+ flexcan_get_mb(priv, FLEXCAN_TX_MB_RESERVED_OFF_TIMESTAMP);
+ else
+ priv->tx_mb_reserved =
+ flexcan_get_mb(priv, FLEXCAN_TX_MB_RESERVED_OFF_FIFO);
+ priv->tx_mb_idx = priv->mb_count - 1;
+ priv->tx_mb = flexcan_get_mb(priv, priv->tx_mb_idx);
+ priv->tx_mask = FLEXCAN_IFLAG_MB(priv->tx_mb_idx);
+
+ priv->offload.mailbox_read = flexcan_mailbox_read;
+
+ if (priv->devtype_data->quirks & FLEXCAN_QUIRK_USE_OFF_TIMESTAMP) {
+ priv->offload.mb_first = FLEXCAN_RX_MB_OFF_TIMESTAMP_FIRST;
+ priv->offload.mb_last = priv->mb_count - 2;
+
+ priv->rx_mask = GENMASK_ULL(priv->offload.mb_last,
+ priv->offload.mb_first);
+ err = can_rx_offload_add_timestamp(dev, &priv->offload);
+ } else {
+ priv->rx_mask = FLEXCAN_IFLAG_RX_FIFO_OVERFLOW |
+ FLEXCAN_IFLAG_RX_FIFO_AVAILABLE;
+ err = can_rx_offload_add_fifo(dev, &priv->offload,
+ FLEXCAN_NAPI_WEIGHT);
+ }
+
+ return err;
+}
+
+static void flexcan_chip_interrupts_enable(const struct net_device *dev)
+{
+ const struct flexcan_priv *priv = netdev_priv(dev);
+ struct flexcan_regs __iomem *regs = priv->regs;
+ u64 reg_imask;
+
+ disable_irq(dev->irq);
+ priv->write(priv->reg_ctrl_default, ®s->ctrl);
+ reg_imask = priv->rx_mask | priv->tx_mask;
+ priv->write(upper_32_bits(reg_imask), ®s->imask2);
+ priv->write(lower_32_bits(reg_imask), ®s->imask1);
+ enable_irq(dev->irq);
+}
+
+static void flexcan_chip_interrupts_disable(const struct net_device *dev)
+{
+ const struct flexcan_priv *priv = netdev_priv(dev);
+ struct flexcan_regs __iomem *regs = priv->regs;
+
+ priv->write(0, ®s->imask2);
+ priv->write(0, ®s->imask1);
+ priv->write(priv->reg_ctrl_default & ~FLEXCAN_CTRL_ERR_ALL,
+ ®s->ctrl);
+}
+
/* flexcan_chip_start
*
* this functions is entered with clocks enabled
struct flexcan_priv *priv = netdev_priv(dev);
struct flexcan_regs __iomem *regs = priv->regs;
u32 reg_mcr, reg_ctrl, reg_ctrl2, reg_mecr;
- u64 reg_imask;
int err, i;
struct flexcan_mb __iomem *mb;
priv->write(reg_ctrl2, ®s->ctrl2);
}
- err = flexcan_transceiver_enable(priv);
- if (err)
- goto out_chip_disable;
-
/* synchronize with the can bus */
err = flexcan_chip_unfreeze(priv);
if (err)
- goto out_transceiver_disable;
+ goto out_chip_disable;
priv->can.state = CAN_STATE_ERROR_ACTIVE;
- /* enable interrupts atomically */
- disable_irq(dev->irq);
- priv->write(priv->reg_ctrl_default, ®s->ctrl);
- reg_imask = priv->rx_mask | priv->tx_mask;
- priv->write(upper_32_bits(reg_imask), ®s->imask2);
- priv->write(lower_32_bits(reg_imask), ®s->imask1);
- enable_irq(dev->irq);
-
/* print chip status */
netdev_dbg(dev, "%s: reading mcr=0x%08x ctrl=0x%08x\n", __func__,
priv->read(®s->mcr), priv->read(®s->ctrl));
return 0;
- out_transceiver_disable:
- flexcan_transceiver_disable(priv);
out_chip_disable:
flexcan_chip_disable(priv);
return err;
static int __flexcan_chip_stop(struct net_device *dev, bool disable_on_error)
{
struct flexcan_priv *priv = netdev_priv(dev);
- struct flexcan_regs __iomem *regs = priv->regs;
int err;
/* freeze + disable module */
if (err && !disable_on_error)
goto out_chip_unfreeze;
- /* Disable all interrupts */
- priv->write(0, ®s->imask2);
- priv->write(0, ®s->imask1);
- priv->write(priv->reg_ctrl_default & ~FLEXCAN_CTRL_ERR_ALL,
- ®s->ctrl);
-
- flexcan_transceiver_disable(priv);
priv->can.state = CAN_STATE_STOPPED;
return 0;
}
err = pm_runtime_get_sync(priv->dev);
- if (err < 0)
+ if (err < 0) {
+ pm_runtime_put_noidle(priv->dev);
return err;
+ }
err = open_candev(dev);
if (err)
goto out_runtime_put;
- err = request_irq(dev->irq, flexcan_irq, IRQF_SHARED, dev->name, dev);
+ err = flexcan_transceiver_enable(priv);
if (err)
goto out_close;
- if (priv->can.ctrlmode & CAN_CTRLMODE_FD)
- priv->mb_size = sizeof(struct flexcan_mb) + CANFD_MAX_DLEN;
- else
- priv->mb_size = sizeof(struct flexcan_mb) + CAN_MAX_DLEN;
- priv->mb_count = (sizeof(priv->regs->mb[0]) / priv->mb_size) +
- (sizeof(priv->regs->mb[1]) / priv->mb_size);
-
- if (priv->devtype_data->quirks & FLEXCAN_QUIRK_USE_OFF_TIMESTAMP)
- priv->tx_mb_reserved =
- flexcan_get_mb(priv, FLEXCAN_TX_MB_RESERVED_OFF_TIMESTAMP);
- else
- priv->tx_mb_reserved =
- flexcan_get_mb(priv, FLEXCAN_TX_MB_RESERVED_OFF_FIFO);
- priv->tx_mb_idx = priv->mb_count - 1;
- priv->tx_mb = flexcan_get_mb(priv, priv->tx_mb_idx);
- priv->tx_mask = FLEXCAN_IFLAG_MB(priv->tx_mb_idx);
+ err = flexcan_rx_offload_setup(dev);
+ if (err)
+ goto out_transceiver_disable;
- priv->offload.mailbox_read = flexcan_mailbox_read;
+ err = flexcan_chip_start(dev);
+ if (err)
+ goto out_can_rx_offload_del;
- if (priv->devtype_data->quirks & FLEXCAN_QUIRK_USE_OFF_TIMESTAMP) {
- priv->offload.mb_first = FLEXCAN_RX_MB_OFF_TIMESTAMP_FIRST;
- priv->offload.mb_last = priv->mb_count - 2;
+ can_rx_offload_enable(&priv->offload);
- priv->rx_mask = GENMASK_ULL(priv->offload.mb_last,
- priv->offload.mb_first);
- err = can_rx_offload_add_timestamp(dev, &priv->offload);
- } else {
- priv->rx_mask = FLEXCAN_IFLAG_RX_FIFO_OVERFLOW |
- FLEXCAN_IFLAG_RX_FIFO_AVAILABLE;
- err = can_rx_offload_add_fifo(dev, &priv->offload,
- FLEXCAN_NAPI_WEIGHT);
- }
+ err = request_irq(dev->irq, flexcan_irq, IRQF_SHARED, dev->name, dev);
if (err)
- goto out_free_irq;
+ goto out_can_rx_offload_disable;
- /* start chip and queuing */
- err = flexcan_chip_start(dev);
- if (err)
- goto out_offload_del;
+ flexcan_chip_interrupts_enable(dev);
can_led_event(dev, CAN_LED_EVENT_OPEN);
- can_rx_offload_enable(&priv->offload);
netif_start_queue(dev);
return 0;
- out_offload_del:
+ out_can_rx_offload_disable:
+ can_rx_offload_disable(&priv->offload);
+ flexcan_chip_stop(dev);
+ out_can_rx_offload_del:
can_rx_offload_del(&priv->offload);
- out_free_irq:
- free_irq(dev->irq, dev);
+ out_transceiver_disable:
+ flexcan_transceiver_disable(priv);
out_close:
close_candev(dev);
out_runtime_put:
struct flexcan_priv *priv = netdev_priv(dev);
netif_stop_queue(dev);
+ flexcan_chip_interrupts_disable(dev);
+ free_irq(dev->irq, dev);
can_rx_offload_disable(&priv->offload);
flexcan_chip_stop_disable_on_error(dev);
can_rx_offload_del(&priv->offload);
- free_irq(dev->irq, dev);
-
+ flexcan_transceiver_disable(priv);
close_candev(dev);
+
pm_runtime_put(priv->dev);
can_led_event(dev, CAN_LED_EVENT_STOP);
if (err)
return err;
+ flexcan_chip_interrupts_enable(dev);
+
netif_wake_queue(dev);
break;
return -EINVAL;
/* stop mode property format is:
- * <&gpr req_gpr>.
+ * <&gpr req_gpr req_bit>.
*/
ret = of_property_read_u32_array(np, "fsl,stop-mode", out_val,
ARRAY_SIZE(out_val));
of_can_transceiver(dev);
devm_can_led_init(dev);
- if (priv->devtype_data->quirks & FLEXCAN_QUIRK_SETUP_STOP_MODE) {
+ if (priv->devtype_data->quirks & FLEXCAN_QUIRK_SETUP_STOP_MODE_GPR) {
err = flexcan_setup_stop_mode(pdev);
if (err)
dev_dbg(&pdev->dev, "failed to setup stop-mode\n");
if (err)
return err;
+ flexcan_chip_interrupts_disable(dev);
+
err = pinctrl_pm_select_sleep_state(device);
if (err)
return err;
err = flexcan_chip_start(dev);
if (err)
return err;
+
+ flexcan_chip_interrupts_enable(dev);
}
}
cf->can_id = ((slot[0] & GRCAN_MSG_BID)
>> GRCAN_MSG_BID_BIT);
}
- cf->can_dlc = get_can_dlc((slot[1] & GRCAN_MSG_DLC)
+ cf->len = can_cc_dlc2len((slot[1] & GRCAN_MSG_DLC)
>> GRCAN_MSG_DLC_BIT);
if (rtr) {
cf->can_id |= CAN_RTR_FLAG;
} else {
- for (i = 0; i < cf->can_dlc; i++) {
+ for (i = 0; i < cf->len; i++) {
j = GRCAN_MSG_DATA_SLOT_INDEX(i);
shift = GRCAN_MSG_DATA_SHIFT(i);
cf->data[i] = (u8)(slot[j] >> shift);
/* Update statistics and read pointer */
stats->rx_packets++;
- stats->rx_bytes += cf->can_dlc;
+ stats->rx_bytes += cf->len;
netif_receive_skb(skb);
rd = grcan_ring_add(rd, GRCAN_MSG_SIZE, dma->rx.size);
eff = cf->can_id & CAN_EFF_FLAG;
rtr = cf->can_id & CAN_RTR_FLAG;
id = cf->can_id & (eff ? CAN_EFF_MASK : CAN_SFF_MASK);
- dlc = cf->can_dlc;
+ dlc = cf->len;
if (eff)
tmp = (id << GRCAN_MSG_EID_BIT) & GRCAN_MSG_EID;
else
* can_put_echo_skb would be an error unless other measures are
* taken.
*/
- priv->txdlc[slotindex] = cf->can_dlc; /* Store dlc for statistics */
+ priv->txdlc[slotindex] = cf->len; /* Store dlc for statistics */
can_put_echo_skb(skb, dev, slotindex);
/* Make sure everything is written before allowing hardware to
dlc = (rxdlc >> IFI_CANFD_RXFIFO_DLC_DLC_OFFSET) &
IFI_CANFD_RXFIFO_DLC_DLC_MASK;
if (rxdlc & IFI_CANFD_RXFIFO_DLC_EDL)
- cf->len = can_dlc2len(dlc);
+ cf->len = can_fd_dlc2len(dlc);
else
- cf->len = get_can_dlc(dlc);
+ cf->len = can_cc_dlc2len(dlc);
rxid = readl(priv->base + IFI_CANFD_RXFIFO_ID);
id = (rxid >> IFI_CANFD_RXFIFO_ID_ID_OFFSET);
writel(IFI_CANFD_ERROR_CTR_ER_ENABLE, priv->base + IFI_CANFD_ERROR_CTR);
stats->rx_packets++;
- stats->rx_bytes += cf->can_dlc;
+ stats->rx_bytes += cf->len;
netif_receive_skb(skb);
return 1;
}
stats->rx_packets++;
- stats->rx_bytes += cf->can_dlc;
+ stats->rx_bytes += cf->len;
netif_receive_skb(skb);
return 1;
txid = cf->can_id & CAN_SFF_MASK;
}
- txdlc = can_len2dlc(cf->len);
+ txdlc = can_fd_len2dlc(cf->len);
if ((priv->can.ctrlmode & CAN_CTRLMODE_FD) && can_is_canfd_skb(skb)) {
txdlc |= IFI_CANFD_TXFIFO_DLC_EDL;
if (cf->flags & CANFD_BRS)
cf->can_id |= desc->data[0] << 3;
cf->can_id |= (desc->data[1] & 0xe0) >> 5;
- cf->can_dlc = get_can_dlc(desc->data[1] & ICAN3_CAN_DLC_MASK);
- memcpy(cf->data, &desc->data[2], cf->can_dlc);
+ cf->len = can_cc_dlc2len(desc->data[1] & ICAN3_CAN_DLC_MASK);
+ memcpy(cf->data, &desc->data[2], cf->len);
} else {
- cf->can_dlc = get_can_dlc(desc->data[0] & ICAN3_CAN_DLC_MASK);
+ cf->len = can_cc_dlc2len(desc->data[0] & ICAN3_CAN_DLC_MASK);
if (desc->data[0] & ICAN3_EFF_RTR)
cf->can_id |= CAN_RTR_FLAG;
cf->can_id |= desc->data[3] >> 5; /* 2-0 */
}
- memcpy(cf->data, &desc->data[6], cf->can_dlc);
+ memcpy(cf->data, &desc->data[6], cf->len);
}
}
/* we always use the extended format, with the ECHO flag set */
desc->command = ICAN3_CAN_TYPE_EFF;
- desc->data[0] |= cf->can_dlc;
+ desc->data[0] |= cf->len;
desc->data[1] |= ICAN3_ECHO;
/* support single transmission (no retries) mode */
}
/* copy the data bits into the descriptor */
- memcpy(&desc->data[6], cf->data, cf->can_dlc);
+ memcpy(&desc->data[6], cf->data, cf->len);
}
/*
}
cf = (struct can_frame *)skb->data;
- dlc = cf->can_dlc;
+ dlc = cf->len;
/* check flag whether this packet has to be looped back */
if (skb->pkt_type != PACKET_LOOPBACK) {
if (cf->can_id != echo_cf->can_id)
return false;
- if (cf->can_dlc != echo_cf->can_dlc)
+ if (cf->len != echo_cf->len)
return false;
- return memcmp(cf->data, echo_cf->data, cf->can_dlc) == 0;
+ return memcmp(cf->data, echo_cf->data, cf->len) == 0;
}
/*
/* update statistics, receive the skb */
stats->rx_packets++;
- stats->rx_bytes += cf->can_dlc;
+ stats->rx_bytes += cf->len;
netif_receive_skb(skb);
err_noalloc:
static const struct can_bittiming_const kvaser_pciefd_bittiming_const = {
.name = KVASER_PCIEFD_DRV_NAME,
.tseg1_min = 1,
- .tseg1_max = 255,
+ .tseg1_max = 512,
.tseg2_min = 1,
.tseg2_max = 32,
.sjw_max = 16,
.brp_min = 1,
- .brp_max = 4096,
+ .brp_max = 8192,
.brp_inc = 1,
};
return err;
err = kvaser_pciefd_bus_on(can);
- if (err)
+ if (err) {
+ close_candev(netdev);
return err;
+ }
return 0;
}
p->header[0] |= KVASER_PCIEFD_RPACKET_IDE;
p->header[0] |= cf->can_id & CAN_EFF_MASK;
- p->header[1] |= can_len2dlc(cf->len) << KVASER_PCIEFD_RPACKET_DLC_SHIFT;
+ p->header[1] |= can_fd_len2dlc(cf->len) << KVASER_PCIEFD_RPACKET_DLC_SHIFT;
p->header[1] |= KVASER_PCIEFD_TPACKET_AREQ;
if (can_is_canfd_skb(skb)) {
if (p->header[0] & KVASER_PCIEFD_RPACKET_IDE)
cf->can_id |= CAN_EFF_FLAG;
- cf->len = can_dlc2len(p->header[1] >> KVASER_PCIEFD_RPACKET_DLC_SHIFT);
+ cf->len = can_fd_dlc2len(p->header[1] >> KVASER_PCIEFD_RPACKET_DLC_SHIFT);
if (p->header[0] & KVASER_PCIEFD_RPACKET_RTR)
cf->can_id |= CAN_RTR_FLAG;
cf->data[7] = bec.rxerr;
stats->rx_packets++;
- stats->rx_bytes += cf->can_dlc;
+ stats->rx_bytes += cf->len;
netif_rx(skb);
return 0;
if (skb) {
cf->can_id |= CAN_ERR_BUSERROR;
- stats->rx_bytes += cf->can_dlc;
+ stats->rx_bytes += cf->len;
stats->rx_packets++;
netif_rx(skb);
} else {
if (!(p->header[0] & KVASER_PCIEFD_RPACKET_RTR)) {
u8 data_len;
- data_len = can_dlc2len(p->header[1] >>
+ data_len = can_fd_dlc2len(p->header[1] >>
KVASER_PCIEFD_RPACKET_DLC_SHIFT);
pos += DIV_ROUND_UP(data_len, 4);
}
# SPDX-License-Identifier: GPL-2.0-only
-config CAN_M_CAN
+menuconfig CAN_M_CAN
tristate "Bosch M_CAN support"
help
Say Y here if you want support for Bosch M_CAN controller framework.
This is common support for devices that embed the Bosch M_CAN IP.
+if CAN_M_CAN
+
config CAN_M_CAN_PLATFORM
tristate "Bosch M_CAN support for io-mapped devices"
depends on HAS_IOMEM
- depends on CAN_M_CAN
help
Say Y here if you want support for IO Mapped Bosch M_CAN controller.
This support is for devices that have the Bosch M_CAN controller
IP embedded into the device and the IP is IO Mapped to the processor.
config CAN_M_CAN_TCAN4X5X
- depends on CAN_M_CAN
- depends on REGMAP_SPI
+ depends on SPI
+ select REGMAP_SPI
tristate "TCAN4X5X M_CAN device"
help
Say Y here if you want support for Texas Instruments TCAN4x5x
M_CAN controller. This device is a peripheral device that uses the
SPI bus for communication.
+
+endif
}
if (dlc & RX_BUF_FDF)
- cf->len = can_dlc2len((dlc >> 16) & 0x0F);
+ cf->len = can_fd_dlc2len((dlc >> 16) & 0x0F);
else
- cf->len = get_can_dlc((dlc >> 16) & 0x0F);
+ cf->len = can_cc_dlc2len((dlc >> 16) & 0x0F);
id = m_can_fifo_read(cdev, fgi, M_CAN_FIFO_ID);
if (id & RX_BUF_XTD)
}
stats->rx_packets++;
- stats->rx_bytes += cf->can_dlc;
+ stats->rx_bytes += cf->len;
netif_receive_skb(skb);
return 1;
unsigned int ecr;
switch (new_state) {
- case CAN_STATE_ERROR_ACTIVE:
+ case CAN_STATE_ERROR_WARNING:
/* error warning state */
cdev->can.can_stats.error_warning++;
cdev->can.state = CAN_STATE_ERROR_WARNING;
__m_can_get_berr_counter(dev, &bec);
switch (new_state) {
- case CAN_STATE_ERROR_ACTIVE:
+ case CAN_STATE_ERROR_WARNING:
/* error warning state */
cf->can_id |= CAN_ERR_CRTL;
cf->data[1] = (bec.txerr > bec.rxerr) ?
}
stats->rx_packets++;
- stats->rx_bytes += cf->can_dlc;
+ stats->rx_bytes += cf->len;
netif_receive_skb(skb);
return 1;
struct net_device_stats *stats = &dev->stats;
u32 ir;
+ if (pm_runtime_suspended(cdev->dev))
+ return IRQ_NONE;
ir = m_can_read(cdev, M_CAN_IR);
if (!ir)
return IRQ_NONE;
.name = KBUILD_MODNAME,
.tseg1_min = 2, /* Time segment 1 = prop_seg + phase_seg1 */
.tseg1_max = 256,
- .tseg2_min = 1, /* Time segment 2 = phase_seg2 */
+ .tseg2_min = 2, /* Time segment 2 = phase_seg2 */
.tseg2_max = 128,
.sjw_max = 128,
.brp_min = 1,
&m_can_data_bittiming_const_31X;
break;
case 32:
+ case 33:
+ /* Support both MCAN version v3.2.x and v3.3.0 */
m_can_dev->can.bittiming_const = m_can_dev->bit_timing ?
m_can_dev->bit_timing : &m_can_bittiming_const_31X;
/* disable all interrupts */
m_can_disable_all_interrupts(cdev);
+ /* Set init mode to disengage from the network */
+ m_can_config_endisable(cdev, true);
+
/* set the state as STOPPED */
cdev->can.state = CAN_STATE_STOPPED;
}
/* message ram configuration */
m_can_fifo_write(cdev, 0, M_CAN_FIFO_ID, id);
m_can_fifo_write(cdev, 0, M_CAN_FIFO_DLC,
- can_len2dlc(cf->len) << 16);
+ can_fd_len2dlc(cf->len) << 16);
for (i = 0; i < cf->len; i += 4)
m_can_fifo_write(cdev, 0,
m_can_fifo_write(cdev, putidx, M_CAN_FIFO_DLC,
((putidx << TX_BUF_MM_SHIFT) &
TX_BUF_MM_MASK) |
- (can_len2dlc(cf->len) << 16) |
+ (can_fd_len2dlc(cf->len) << 16) |
fdflags | TX_BUF_EFC);
for (i = 0; i < cf->len; i += 4)
INIT_WORK(&cdev->tx_work, m_can_tx_work_queue);
err = request_threaded_irq(dev->irq, NULL, m_can_isr,
- IRQF_ONESHOT | IRQF_TRIGGER_FALLING,
+ IRQF_ONESHOT,
dev->name, dev);
} else {
err = request_irq(dev->irq, m_can_isr, IRQF_SHARED, dev->name,
}
EXPORT_SYMBOL_GPL(m_can_class_allocate_dev);
+void m_can_class_free_dev(struct net_device *net)
+{
+ free_candev(net);
+}
+EXPORT_SYMBOL_GPL(m_can_class_free_dev);
+
int m_can_class_register(struct m_can_classdev *m_can_dev)
{
int ret;
if (ret) {
if (m_can_dev->pm_clock_support)
pm_runtime_disable(m_can_dev->dev);
- free_candev(m_can_dev->net);
}
return ret;
}
EXPORT_SYMBOL_GPL(m_can_class_register);
+void m_can_class_unregister(struct m_can_classdev *m_can_dev)
+{
+ unregister_candev(m_can_dev->net);
+
+ m_can_clk_stop(m_can_dev);
+}
+EXPORT_SYMBOL_GPL(m_can_class_unregister);
+
int m_can_class_suspend(struct device *dev)
{
struct net_device *ndev = dev_get_drvdata(dev);
}
EXPORT_SYMBOL_GPL(m_can_class_resume);
-void m_can_class_unregister(struct m_can_classdev *m_can_dev)
-{
- unregister_candev(m_can_dev->net);
-
- m_can_clk_stop(m_can_dev);
-
- free_candev(m_can_dev->net);
-}
-EXPORT_SYMBOL_GPL(m_can_class_unregister);
-
MODULE_AUTHOR("Dong Aisheng <b29396@freescale.com>");
MODULE_AUTHOR("Dan Murphy <dmurphy@ti.com>");
MODULE_LICENSE("GPL v2");
void *device_data;
int version;
- int freq;
u32 irqstatus;
int pm_clock_support;
};
struct m_can_classdev *m_can_class_allocate_dev(struct device *dev);
+void m_can_class_free_dev(struct net_device *net);
int m_can_class_register(struct m_can_classdev *cdev);
void m_can_class_unregister(struct m_can_classdev *cdev);
int m_can_class_get_clocks(struct m_can_classdev *cdev);
return -ENOMEM;
priv = devm_kzalloc(&pdev->dev, sizeof(*priv), GFP_KERNEL);
- if (!priv)
- return -ENOMEM;
+ if (!priv) {
+ ret = -ENOMEM;
+ goto probe_fail;
+ }
mcan_class->device_data = priv;
- m_can_class_get_clocks(mcan_class);
+ ret = m_can_class_get_clocks(mcan_class);
+ if (ret)
+ goto probe_fail;
res = platform_get_resource_byname(pdev, IORESOURCE_MEM, "m_can");
addr = devm_ioremap_resource(&pdev->dev, res);
irq = platform_get_irq_byname(pdev, "int0");
if (IS_ERR(addr) || irq < 0) {
ret = -EINVAL;
- goto failed_ret;
+ goto probe_fail;
}
/* message ram could be shared */
res = platform_get_resource_byname(pdev, IORESOURCE_MEM, "message_ram");
if (!res) {
ret = -ENODEV;
- goto failed_ret;
+ goto probe_fail;
}
mram_addr = devm_ioremap(&pdev->dev, res->start, resource_size(res));
if (!mram_addr) {
ret = -ENOMEM;
- goto failed_ret;
+ goto probe_fail;
}
priv->base = addr;
m_can_init_ram(mcan_class);
- ret = m_can_class_register(mcan_class);
+ return m_can_class_register(mcan_class);
-failed_ret:
+probe_fail:
+ m_can_class_free_dev(mcan_class->net);
return ret;
}
m_can_class_unregister(mcan_class);
- platform_set_drvdata(pdev, NULL);
+ m_can_class_free_dev(mcan_class->net);
return 0;
}
struct gpio_desc *device_wake_gpio;
struct gpio_desc *device_state_gpio;
struct regulator *power;
-
- /* Register based ip */
- int mram_start;
- int reg_offset;
};
static struct can_bittiming_const tcan4x5x_bittiming_const = {
struct tcan4x5x_priv *priv = cdev->device_data;
u32 val;
- regmap_read(priv->regmap, priv->reg_offset + reg, &val);
+ regmap_read(priv->regmap, TCAN4X5X_MCAN_OFFSET + reg, &val);
return val;
}
struct tcan4x5x_priv *priv = cdev->device_data;
u32 val;
- regmap_read(priv->regmap, priv->mram_start + addr_offset, &val);
+ regmap_read(priv->regmap, TCAN4X5X_MRAM_START + addr_offset, &val);
return val;
}
{
struct tcan4x5x_priv *priv = cdev->device_data;
- return regmap_write(priv->regmap, priv->reg_offset + reg, val);
+ return regmap_write(priv->regmap, TCAN4X5X_MCAN_OFFSET + reg, val);
}
static int tcan4x5x_write_fifo(struct m_can_classdev *cdev,
{
struct tcan4x5x_priv *priv = cdev->device_data;
- return regmap_write(priv->regmap, priv->mram_start + addr_offset, val);
+ return regmap_write(priv->regmap, TCAN4X5X_MRAM_START + addr_offset, val);
}
static int tcan4x5x_power_enable(struct regulator *reg, int enable)
if (ret)
return ret;
- ret = tcan4x5x_write_tcan_reg(cdev, TCAN4X5X_ERROR_STATUS,
- TCAN4X5X_CLEAR_ALL_INT);
- if (ret)
- return ret;
-
- return ret;
+ return tcan4x5x_write_tcan_reg(cdev, TCAN4X5X_ERROR_STATUS,
+ TCAN4X5X_CLEAR_ALL_INT);
}
static int tcan4x5x_init(struct m_can_classdev *cdev)
TCAN4X5X_DISABLE_INH_MSK, 0x01);
}
-static int tcan4x5x_parse_config(struct m_can_classdev *cdev)
+static int tcan4x5x_get_gpios(struct m_can_classdev *cdev)
{
struct tcan4x5x_priv *tcan4x5x = cdev->device_data;
int ret;
return -ENOMEM;
priv = devm_kzalloc(&spi->dev, sizeof(*priv), GFP_KERNEL);
- if (!priv)
- return -ENOMEM;
+ if (!priv) {
+ ret = -ENOMEM;
+ goto out_m_can_class_free_dev;
+ }
priv->power = devm_regulator_get_optional(&spi->dev, "vsup");
- if (PTR_ERR(priv->power) == -EPROBE_DEFER)
- return -EPROBE_DEFER;
- else
+ if (PTR_ERR(priv->power) == -EPROBE_DEFER) {
+ ret = -EPROBE_DEFER;
+ goto out_m_can_class_free_dev;
+ } else {
priv->power = NULL;
+ }
mcan_class->device_data = priv;
}
/* Sanity check */
- if (freq < 20000000 || freq > TCAN4X5X_EXT_CLK_DEF)
- return -ERANGE;
+ if (freq < 20000000 || freq > TCAN4X5X_EXT_CLK_DEF) {
+ ret = -ERANGE;
+ goto out_m_can_class_free_dev;
+ }
- priv->reg_offset = TCAN4X5X_MCAN_OFFSET;
- priv->mram_start = TCAN4X5X_MRAM_START;
priv->spi = spi;
priv->mcan_dev = mcan_class;
spi->bits_per_word = 32;
ret = spi_setup(spi);
if (ret)
- goto out_clk;
+ goto out_m_can_class_free_dev;
priv->regmap = devm_regmap_init(&spi->dev, &tcan4x5x_bus,
&spi->dev, &tcan4x5x_regmap);
+ if (IS_ERR(priv->regmap)) {
+ ret = PTR_ERR(priv->regmap);
+ goto out_m_can_class_free_dev;
+ }
ret = tcan4x5x_power_enable(priv->power, 1);
if (ret)
- goto out_clk;
+ goto out_m_can_class_free_dev;
- ret = tcan4x5x_parse_config(mcan_class);
+ ret = tcan4x5x_get_gpios(mcan_class);
if (ret)
goto out_power;
out_power:
tcan4x5x_power_enable(priv->power, 0);
-out_clk:
- if (!IS_ERR(mcan_class->cclk)) {
- clk_disable_unprepare(mcan_class->cclk);
- clk_disable_unprepare(mcan_class->hclk);
- }
-
- dev_err(&spi->dev, "Probe failed, err=%d\n", ret);
+ out_m_can_class_free_dev:
+ m_can_class_free_dev(mcan_class->net);
return ret;
}
{
struct tcan4x5x_priv *priv = spi_get_drvdata(spi);
+ m_can_class_unregister(priv->mcan_dev);
+
tcan4x5x_power_enable(priv->power, 0);
- m_can_class_unregister(priv->mcan_dev);
+ m_can_class_free_dev(priv->mcan_dev->net);
return 0;
}
void __iomem *data = ®s->tx.dsr1_0;
u16 *payload = (u16 *)frame->data;
- for (i = 0; i < frame->can_dlc / 2; i++) {
+ for (i = 0; i < frame->len / 2; i++) {
out_be16(data, *payload++);
data += 2 + _MSCAN_RESERVED_DSR_SIZE;
}
/* write remaining byte if necessary */
- if (frame->can_dlc & 1)
- out_8(data, frame->data[frame->can_dlc - 1]);
+ if (frame->len & 1)
+ out_8(data, frame->data[frame->len - 1]);
}
- out_8(®s->tx.dlr, frame->can_dlc);
+ out_8(®s->tx.dlr, frame->len);
out_8(®s->tx.tbpr, priv->cur_pri);
/* Start transmission. */
if (can_id & 1)
frame->can_id |= CAN_RTR_FLAG;
- frame->can_dlc = get_can_dlc(in_8(®s->rx.dlr) & 0xf);
+ frame->len = can_cc_dlc2len(in_8(®s->rx.dlr) & 0xf);
if (!(frame->can_id & CAN_RTR_FLAG)) {
void __iomem *data = ®s->rx.dsr1_0;
u16 *payload = (u16 *)frame->data;
- for (i = 0; i < frame->can_dlc / 2; i++) {
+ for (i = 0; i < frame->len / 2; i++) {
*payload++ = in_be16(data);
data += 2 + _MSCAN_RESERVED_DSR_SIZE;
}
/* read remaining byte if necessary */
- if (frame->can_dlc & 1)
- frame->data[frame->can_dlc - 1] = in_8(data);
+ if (frame->len & 1)
+ frame->data[frame->len - 1] = in_8(data);
}
out_8(®s->canrflg, MSCAN_RXF);
}
}
priv->shadow_statflg = canrflg & MSCAN_STAT_MSK;
- frame->can_dlc = CAN_ERR_DLC;
+ frame->len = CAN_ERR_DLC;
out_8(®s->canrflg, MSCAN_ERR_IF);
}
mscan_get_err_frame(dev, frame, canrflg);
stats->rx_packets++;
- stats->rx_bytes += frame->can_dlc;
+ stats->rx_bytes += frame->len;
work_done++;
netif_receive_skb(skb);
}
netif_receive_skb(skb);
stats->rx_packets++;
- stats->rx_bytes += cf->can_dlc;
+ stats->rx_bytes += cf->len;
}
static irqreturn_t pch_can_interrupt(int irq, void *dev_id)
if (id2 & PCH_ID2_DIR)
cf->can_id |= CAN_RTR_FLAG;
- cf->can_dlc = get_can_dlc((ioread32(&priv->regs->
+ cf->len = can_cc_dlc2len((ioread32(&priv->regs->
ifregs[0].mcont)) & 0xF);
- for (i = 0; i < cf->can_dlc; i += 2) {
+ for (i = 0; i < cf->len; i += 2) {
data_reg = ioread16(&priv->regs->ifregs[0].data[i / 2]);
cf->data[i] = data_reg;
cf->data[i + 1] = data_reg >> 8;
rcv_pkts++;
stats->rx_packets++;
quota--;
- stats->rx_bytes += cf->can_dlc;
+ stats->rx_bytes += cf->len;
pch_fifo_thresh(priv, obj_num);
obj_num++;
iowrite32(PCH_CMASK_RX_TX_GET | PCH_CMASK_CLRINTPND,
&priv->regs->ifregs[1].cmask);
pch_can_rw_msg_obj(&priv->regs->ifregs[1].creq, int_stat);
- dlc = get_can_dlc(ioread32(&priv->regs->ifregs[1].mcont) &
+ dlc = can_cc_dlc2len(ioread32(&priv->regs->ifregs[1].mcont) &
PCH_IF_MCONT_DLC);
stats->tx_bytes += dlc;
stats->tx_packets++;
iowrite32(id2, &priv->regs->ifregs[1].id2);
/* Copy data to register */
- for (i = 0; i < cf->can_dlc; i += 2) {
+ for (i = 0; i < cf->len; i += 2) {
iowrite16(cf->data[i] | (cf->data[i + 1] << 8),
&priv->regs->ifregs[1].data[i / 2]);
}
can_put_echo_skb(skb, ndev, tx_obj_no - PCH_RX_OBJ_END - 1);
/* Set the size of the data. Update if2_mcont */
- iowrite32(cf->can_dlc | PCH_IF_MCONT_NEWDAT | PCH_IF_MCONT_TXRQXT |
+ iowrite32(cf->len | PCH_IF_MCONT_NEWDAT | PCH_IF_MCONT_TXRQXT |
PCH_IF_MCONT_TXIE, &priv->regs->ifregs[1].mcont);
pch_can_rw_msg_obj(&priv->regs->ifregs[1].creq, tx_obj_no);
u8 cf_len;
if (rx_msg_flags & PUCAN_MSG_EXT_DATA_LEN)
- cf_len = can_dlc2len(get_canfd_dlc(pucan_msg_get_dlc(msg)));
+ cf_len = can_fd_dlc2len(pucan_msg_get_dlc(msg));
else
- cf_len = get_can_dlc(pucan_msg_get_dlc(msg));
+ cf_len = can_cc_dlc2len(pucan_msg_get_dlc(msg));
/* if this frame is an echo, */
if (rx_msg_flags & PUCAN_MSG_LOOPED_BACK) {
}
stats->rx_packets++;
- stats->rx_bytes += cf->can_dlc;
+ stats->rx_bytes += cf->len;
pucan_netif_rx(skb, msg->ts_low, msg->ts_high);
return 0;
cf->data[6] = priv->bec.txerr;
cf->data[7] = priv->bec.rxerr;
- stats->rx_bytes += cf->can_dlc;
+ stats->rx_bytes += cf->len;
stats->rx_packets++;
netif_rx(skb);
unsigned long flags;
bool should_stop_tx_queue;
int room_left;
- u8 can_dlc;
+ u8 len;
if (can_dropped_invalid_skb(ndev, skb))
return NETDEV_TX_OK;
if (can_is_canfd_skb(skb)) {
/* CAN FD frame format */
- can_dlc = can_len2dlc(cf->len);
+ len = can_fd_len2dlc(cf->len);
msg_flags |= PUCAN_MSG_EXT_DATA_LEN;
msg_flags |= PUCAN_MSG_ERROR_STATE_IND;
} else {
/* CAN 2.0 frame format */
- can_dlc = cf->len;
+ len = cf->len;
if (cf->can_id & CAN_RTR_FLAG)
msg_flags |= PUCAN_MSG_RTR;
msg_flags |= PUCAN_MSG_SELF_RECEIVE;
msg->flags = cpu_to_le16(msg_flags);
- msg->channel_dlc = PUCAN_MSG_CHANNEL_DLC(priv->index, can_dlc);
+ msg->channel_dlc = PUCAN_MSG_CHANNEL_DLC(priv->index, len);
memcpy(msg->d, cf->data, cf->len);
/* struct msg client field is used as an index in the echo skbs ring */
if (skb) {
stats->rx_packets++;
- stats->rx_bytes += cf->can_dlc;
+ stats->rx_bytes += cf->len;
netif_rx(skb);
}
}
if (cf->can_id & CAN_RTR_FLAG) { /* Remote transmission request */
data |= RCAR_CAN_RTR;
} else {
- for (i = 0; i < cf->can_dlc; i++)
+ for (i = 0; i < cf->len; i++)
writeb(cf->data[i],
&priv->regs->mb[RCAR_CAN_TX_FIFO_MBX].data[i]);
}
writel(data, &priv->regs->mb[RCAR_CAN_TX_FIFO_MBX].id);
- writeb(cf->can_dlc, &priv->regs->mb[RCAR_CAN_TX_FIFO_MBX].dlc);
+ writeb(cf->len, &priv->regs->mb[RCAR_CAN_TX_FIFO_MBX].dlc);
- priv->tx_dlc[priv->tx_head % RCAR_CAN_FIFO_DEPTH] = cf->can_dlc;
+ priv->tx_dlc[priv->tx_head % RCAR_CAN_FIFO_DEPTH] = cf->len;
can_put_echo_skb(skb, ndev, priv->tx_head % RCAR_CAN_FIFO_DEPTH);
priv->tx_head++;
/* Start Tx: write 0xff to the TFPCR register to increment
cf->can_id = (data >> RCAR_CAN_SID_SHIFT) & CAN_SFF_MASK;
dlc = readb(&priv->regs->mb[RCAR_CAN_RX_FIFO_MBX].dlc);
- cf->can_dlc = get_can_dlc(dlc);
+ cf->len = can_cc_dlc2len(dlc);
if (data & RCAR_CAN_RTR) {
cf->can_id |= CAN_RTR_FLAG;
} else {
- for (dlc = 0; dlc < cf->can_dlc; dlc++)
+ for (dlc = 0; dlc < cf->len; dlc++)
cf->data[dlc] =
readb(&priv->regs->mb[RCAR_CAN_RX_FIFO_MBX].data[dlc]);
}
can_led_event(priv->ndev, CAN_LED_EVENT_RX);
- stats->rx_bytes += cf->can_dlc;
+ stats->rx_bytes += cf->len;
stats->rx_packets++;
netif_receive_skb(skb);
}
rcar_canfd_write(priv->base, RCANFD_CERFL(ch),
RCANFD_CERFL_ERR(~cerfl));
stats->rx_packets++;
- stats->rx_bytes += cf->can_dlc;
+ stats->rx_bytes += cf->len;
netif_rx(skb);
}
can_change_state(ndev, cf, tx_state, rx_state);
stats->rx_packets++;
- stats->rx_bytes += cf->can_dlc;
+ stats->rx_bytes += cf->len;
netif_rx(skb);
}
}
if (cf->can_id & CAN_RTR_FLAG)
id |= RCANFD_CFID_CFRTR;
- dlc = RCANFD_CFPTR_CFDLC(can_len2dlc(cf->len));
+ dlc = RCANFD_CFPTR_CFDLC(can_fd_len2dlc(cf->len));
if (priv->can.ctrlmode & CAN_CTRLMODE_FD) {
rcar_canfd_write(priv->base,
if (priv->can.ctrlmode & CAN_CTRLMODE_FD) {
if (sts & RCANFD_RFFDSTS_RFFDF)
- cf->len = can_dlc2len(RCANFD_RFPTR_RFDLC(dlc));
+ cf->len = can_fd_dlc2len(RCANFD_RFPTR_RFDLC(dlc));
else
- cf->len = get_can_dlc(RCANFD_RFPTR_RFDLC(dlc));
+ cf->len = can_cc_dlc2len(RCANFD_RFPTR_RFDLC(dlc));
if (sts & RCANFD_RFFDSTS_RFESI) {
cf->flags |= CANFD_ESI;
rcar_canfd_get_data(priv, cf, RCANFD_F_RFDF(ridx, 0));
}
} else {
- cf->len = get_can_dlc(RCANFD_RFPTR_RFDLC(dlc));
+ cf->len = can_cc_dlc2len(RCANFD_RFPTR_RFDLC(dlc));
if (id & RCANFD_RFID_RFRTR)
cf->can_id |= CAN_RTR_FLAG;
else
work_done++;
stats->rx_packets++;
- stats->rx_bytes += cf->can_dlc;
+ stats->rx_bytes += cf->len;
netif_receive_skb(skb);
}
struct sja1000_priv *priv = netdev_priv(dev);
struct can_frame *cf = (struct can_frame *)skb->data;
uint8_t fi;
- uint8_t dlc;
canid_t id;
uint8_t dreg;
u8 cmd_reg_val = 0x00;
netif_stop_queue(dev);
- fi = dlc = cf->can_dlc;
+ fi = can_get_cc_dlc(cf, priv->can.ctrlmode);
id = cf->can_id;
if (id & CAN_RTR_FLAG)
priv->write_reg(priv, SJA1000_ID2, (id & 0x00000007) << 5);
}
- for (i = 0; i < dlc; i++)
+ for (i = 0; i < cf->len; i++)
priv->write_reg(priv, dreg++, cf->data[i]);
can_put_echo_skb(skb, dev, 0);
| (priv->read_reg(priv, SJA1000_ID2) >> 5);
}
- cf->can_dlc = get_can_dlc(fi & 0x0F);
+ can_frame_set_cc_len(cf, fi & 0x0F, priv->can.ctrlmode);
if (fi & SJA1000_FI_RTR) {
id |= CAN_RTR_FLAG;
} else {
- for (i = 0; i < cf->can_dlc; i++)
+ for (i = 0; i < cf->len; i++)
cf->data[i] = priv->read_reg(priv, dreg++);
}
sja1000_write_cmdreg(priv, CMD_RRB);
stats->rx_packets++;
- stats->rx_bytes += cf->can_dlc;
+ stats->rx_bytes += cf->len;
netif_rx(skb);
can_led_event(dev, CAN_LED_EVENT_RX);
netdev_dbg(dev, "arbitration lost interrupt\n");
alc = priv->read_reg(priv, SJA1000_ALC);
priv->can.can_stats.arbitration_lost++;
- stats->tx_errors++;
cf->can_id |= CAN_ERR_LOSTARB;
cf->data[0] = alc & 0x1f;
}
}
stats->rx_packets++;
- stats->rx_bytes += cf->can_dlc;
+ stats->rx_bytes += cf->len;
netif_rx(skb);
return 0;
CAN_CTRLMODE_3_SAMPLES |
CAN_CTRLMODE_ONE_SHOT |
CAN_CTRLMODE_BERR_REPORTING |
- CAN_CTRLMODE_PRESUME_ACK;
+ CAN_CTRLMODE_PRESUME_ACK |
+ CAN_CTRLMODE_CC_LEN8_DLC;
spin_lock_init(&priv->cmdreg_lock);
/*
* A CAN frame has a can_id (11 bit standard frame format OR 29 bit extended
- * frame format) a data length code (can_dlc) which can be from 0 to 8
- * and up to <can_dlc> data bytes as payload.
+ * frame format) a data length code (len) which can be from 0 to 8
+ * and up to <len> data bytes as payload.
* Additionally a CAN frame may become a remote transmission frame if the
* RTR-bit is set. This causes another ECU to send a CAN frame with the
* given can_id.
*
* Examples:
*
- * t1230 : can_id 0x123, can_dlc 0, no data
- * t4563112233 : can_id 0x456, can_dlc 3, data 0x11 0x22 0x33
- * T12ABCDEF2AA55 : extended can_id 0x12ABCDEF, can_dlc 2, data 0xAA 0x55
- * r1230 : can_id 0x123, can_dlc 0, no data, remote transmission request
+ * t1230 : can_id 0x123, len 0, no data
+ * t4563112233 : can_id 0x456, len 3, data 0x11 0x22 0x33
+ * T12ABCDEF2AA55 : extended can_id 0x12ABCDEF, len 2, data 0xAA 0x55
+ * r1230 : can_id 0x123, len 0, no data, remote transmission request
*
*/
fallthrough;
case 't':
/* store dlc ASCII value and terminate SFF CAN ID string */
- cf.can_dlc = sl->rbuff[SLC_CMD_LEN + SLC_SFF_ID_LEN];
+ cf.len = sl->rbuff[SLC_CMD_LEN + SLC_SFF_ID_LEN];
sl->rbuff[SLC_CMD_LEN + SLC_SFF_ID_LEN] = 0;
/* point to payload data behind the dlc */
cmd += SLC_CMD_LEN + SLC_SFF_ID_LEN + 1;
case 'T':
cf.can_id |= CAN_EFF_FLAG;
/* store dlc ASCII value and terminate EFF CAN ID string */
- cf.can_dlc = sl->rbuff[SLC_CMD_LEN + SLC_EFF_ID_LEN];
+ cf.len = sl->rbuff[SLC_CMD_LEN + SLC_EFF_ID_LEN];
sl->rbuff[SLC_CMD_LEN + SLC_EFF_ID_LEN] = 0;
/* point to payload data behind the dlc */
cmd += SLC_CMD_LEN + SLC_EFF_ID_LEN + 1;
cf.can_id |= tmpid;
- /* get can_dlc from sanitized ASCII value */
- if (cf.can_dlc >= '0' && cf.can_dlc < '9')
- cf.can_dlc -= '0';
+ /* get len from sanitized ASCII value */
+ if (cf.len >= '0' && cf.len < '9')
+ cf.len -= '0';
else
return;
/* RTR frames may have a dlc > 0 but they never have any data bytes */
if (!(cf.can_id & CAN_RTR_FLAG)) {
- for (i = 0; i < cf.can_dlc; i++) {
+ for (i = 0; i < cf.len; i++) {
tmp = hex_to_bin(*cmd++);
if (tmp < 0)
return;
skb_put_data(skb, &cf, sizeof(struct can_frame));
sl->dev->stats.rx_packets++;
- sl->dev->stats.rx_bytes += cf.can_dlc;
+ sl->dev->stats.rx_bytes += cf.len;
netif_rx_ni(skb);
}
pos += (cf->can_id & CAN_EFF_FLAG) ? SLC_EFF_ID_LEN : SLC_SFF_ID_LEN;
- *pos++ = cf->can_dlc + '0';
+ *pos++ = cf->len + '0';
/* RTR frames may have a dlc > 0 but they never have any data bytes */
if (!(cf->can_id & CAN_RTR_FLAG)) {
- for (i = 0; i < cf->can_dlc; i++)
+ for (i = 0; i < cf->len; i++)
pos = hex_byte_pack_upper(pos, cf->data[i]);
}
actual = sl->tty->ops->write(sl->tty, sl->xbuff, pos - sl->xbuff);
sl->xleft = (pos - sl->xbuff) - actual;
sl->xhead = sl->xbuff + actual;
- sl->dev->stats.tx_bytes += cf->can_dlc;
+ sl->dev->stats.tx_bytes += cf->len;
}
/* Write out any remaining transmit buffer. Scheduled when tty is writable */
*/
memset(&msg, 0, sizeof(msg));
msg.can_id = CAN_ERR_FLAG | CAN_ERR_RESTARTED;
- msg.can_dlc = CAN_ERR_DLC;
+ msg.len = CAN_ERR_DLC;
for (j = 0; j < ARRAY_SIZE(card->net); ++j) {
if (!(bus_bitmask_start & (1 << j)))
continue;
if (priv->index)
*ptr |= CMD_BUS2;
++ptr;
- *ptr++ = cf->can_dlc;
+ *ptr++ = cf->len;
*ptr++ = (cf->can_id >> 0);
*ptr++ = (cf->can_id >> 8);
if (cf->can_id & CAN_EFF_FLAG) {
ptr += 1;
}
if (!(cf->can_id & CAN_RTR_FLAG))
- memcpy(ptr, &cf->data[0], cf->can_dlc);
+ memcpy(ptr, &cf->data[0], cf->len);
memcpy_toio(&card->dpram[DPRAM_TX + DPRAM_TX_SIZE * fifo_wr],
buf, DPRAM_TX_SIZE);
if (++fifo_wr >= DPRAM_TX_CNT)
iowrite8(0, &card->dpram[DPRAM_RX_LOST]);
/* prepare msg */
msg.can_id = CAN_ERR_FLAG | CAN_ERR_CRTL;
- msg.can_dlc = CAN_ERR_DLC;
+ msg.len = CAN_ERR_DLC;
msg.data[1] = CAN_ERR_CRTL_RX_OVERFLOW;
/*
* service to all buses, we don't know which it was applicable
state = *ptr++;
msg.can_id = CAN_ERR_FLAG;
- msg.can_dlc = CAN_ERR_DLC;
+ msg.len = CAN_ERR_DLC;
if (state & SF_MASK_BUSOFF) {
can_state = CAN_STATE_BUS_OFF;
} else {
if (cmd & CMD_RTR)
msg.can_id |= CAN_RTR_FLAG;
- msg.can_dlc = get_can_dlc(*ptr++);
+ msg.len = can_cc_dlc2len(*ptr++);
if (cmd & CMD_XTD) {
msg.can_id |= CAN_EFF_FLAG;
msg.can_id |= le32_to_cpup((void *)ptr);
--card->tx.pending;
++netdev->stats.tx_packets;
if (!(msg.can_id & CAN_RTR_FLAG))
- netdev->stats.tx_bytes += msg.can_dlc;
+ netdev->stats.tx_bytes += msg.len;
} else {
int ret;
if (ret == NET_RX_SUCCESS) {
++netdev->stats.rx_packets;
if (!(msg.can_id & CAN_RTR_FLAG))
- netdev->stats.rx_bytes += msg.can_dlc;
+ netdev->stats.rx_bytes += msg.len;
} else {
++netdev->stats.rx_dropped;
}
((frame->can_id & CAN_EFF_MASK) << 1) |
((frame->can_id & CAN_RTR_FLAG) ? 1 : 0);
- buf[HI3110_FIFO_EXT_DLC_OFF] = frame->can_dlc;
+ buf[HI3110_FIFO_EXT_DLC_OFF] = frame->len;
memcpy(buf + HI3110_FIFO_EXT_DATA_OFF,
- frame->data, frame->can_dlc);
+ frame->data, frame->len);
hi3110_hw_tx_frame(spi, buf, HI3110_TX_EXT_BUF_LEN -
- (HI3110_CAN_MAX_DATA_LEN - frame->can_dlc));
+ (HI3110_CAN_MAX_DATA_LEN - frame->len));
} else {
/* Standard frame */
buf[HI3110_FIFO_ID_OFF] = (frame->can_id & CAN_SFF_MASK) >> 3;
((frame->can_id & CAN_SFF_MASK) << 5) |
((frame->can_id & CAN_RTR_FLAG) ? (1 << 4) : 0);
- buf[HI3110_FIFO_STD_DLC_OFF] = frame->can_dlc;
+ buf[HI3110_FIFO_STD_DLC_OFF] = frame->len;
memcpy(buf + HI3110_FIFO_STD_DATA_OFF,
- frame->data, frame->can_dlc);
+ frame->data, frame->len);
hi3110_hw_tx_frame(spi, buf, HI3110_TX_STD_BUF_LEN -
- (HI3110_CAN_MAX_DATA_LEN - frame->can_dlc));
+ (HI3110_CAN_MAX_DATA_LEN - frame->len));
}
}
}
/* Data length */
- frame->can_dlc = get_can_dlc(buf[HI3110_FIFO_WOTIME_DLC_OFF] & 0x0F);
+ frame->len = can_cc_dlc2len(buf[HI3110_FIFO_WOTIME_DLC_OFF] & 0x0F);
if (buf[HI3110_FIFO_WOTIME_ID_OFF + 3] & HI3110_FIFO_WOTIME_ID_RTR)
frame->can_id |= CAN_RTR_FLAG;
else
memcpy(frame->data, buf + HI3110_FIFO_WOTIME_DAT_OFF,
- frame->can_dlc);
+ frame->len);
priv->net->stats.rx_packets++;
- priv->net->stats.rx_bytes += frame->can_dlc;
+ priv->net->stats.rx_bytes += frame->len;
can_led_event(priv->net, CAN_LED_EVENT_RX);
} else {
frame = (struct can_frame *)priv->tx_skb->data;
hi3110_hw_tx(spi, frame);
- priv->tx_len = 1 + frame->can_dlc;
+ priv->tx_len = 1 + frame->len;
can_put_echo_skb(priv->tx_skb, net, 0);
priv->tx_skb = NULL;
}
((eid >> SIDL_EID_SHIFT) & SIDL_EID_MASK);
buf[TXBEID8_OFF] = GET_BYTE(eid, 1);
buf[TXBEID0_OFF] = GET_BYTE(eid, 0);
- buf[TXBDLC_OFF] = (rtr << DLC_RTR_SHIFT) | frame->can_dlc;
- memcpy(buf + TXBDAT_OFF, frame->data, frame->can_dlc);
- mcp251x_hw_tx_frame(spi, buf, frame->can_dlc, tx_buf_idx);
+ buf[TXBDLC_OFF] = (rtr << DLC_RTR_SHIFT) | frame->len;
+ memcpy(buf + TXBDAT_OFF, frame->data, frame->len);
+ mcp251x_hw_tx_frame(spi, buf, frame->len, tx_buf_idx);
/* use INSTRUCTION_RTS, to avoid "repeated frame problem" */
priv->spi_tx_buf[0] = INSTRUCTION_RTS(1 << tx_buf_idx);
for (i = 1; i < RXBDAT_OFF; i++)
buf[i] = mcp251x_read_reg(spi, RXBCTRL(buf_idx) + i);
- len = get_can_dlc(buf[RXBDLC_OFF] & RXBDLC_LEN_MASK);
+ len = can_cc_dlc2len(buf[RXBDLC_OFF] & RXBDLC_LEN_MASK);
for (; i < (RXBDAT_OFF + len); i++)
buf[i] = mcp251x_read_reg(spi, RXBCTRL(buf_idx) + i);
} else {
frame->can_id |= CAN_RTR_FLAG;
}
/* Data length */
- frame->can_dlc = get_can_dlc(buf[RXBDLC_OFF] & RXBDLC_LEN_MASK);
- memcpy(frame->data, buf + RXBDAT_OFF, frame->can_dlc);
+ frame->len = can_cc_dlc2len(buf[RXBDLC_OFF] & RXBDLC_LEN_MASK);
+ memcpy(frame->data, buf + RXBDAT_OFF, frame->len);
priv->net->stats.rx_packets++;
- priv->net->stats.rx_bytes += frame->can_dlc;
+ priv->net->stats.rx_bytes += frame->len;
can_led_event(priv->net, CAN_LED_EVENT_RX);
} else {
frame = (struct can_frame *)priv->tx_skb->data;
- if (frame->can_dlc > CAN_FRAME_MAX_DATA_LEN)
- frame->can_dlc = CAN_FRAME_MAX_DATA_LEN;
+ if (frame->len > CAN_FRAME_MAX_DATA_LEN)
+ frame->len = CAN_FRAME_MAX_DATA_LEN;
mcp251x_hw_tx(spi, frame, 0);
- priv->tx_len = 1 + frame->can_dlc;
+ priv->tx_len = 1 + frame->len;
can_put_echo_skb(priv->tx_skb, net, 0);
priv->tx_skb = NULL;
}
static void mcp251xfd_ring_init(struct mcp251xfd_priv *priv)
{
+ struct mcp251xfd_tef_ring *tef_ring;
struct mcp251xfd_tx_ring *tx_ring;
struct mcp251xfd_rx_ring *rx_ring, *prev_rx_ring = NULL;
struct mcp251xfd_tx_obj *tx_obj;
u32 val;
u16 addr;
u8 len;
- int i;
+ int i, j;
/* TEF */
- priv->tef.head = 0;
- priv->tef.tail = 0;
+ tef_ring = priv->tef;
+ tef_ring->head = 0;
+ tef_ring->tail = 0;
+
+ /* FIFO increment TEF tail pointer */
+ addr = MCP251XFD_REG_TEFCON;
+ val = MCP251XFD_REG_TEFCON_UINC;
+ len = mcp251xfd_cmd_prepare_write_reg(priv, &tef_ring->uinc_buf,
+ addr, val, val);
+
+ for (j = 0; j < ARRAY_SIZE(tef_ring->uinc_xfer); j++) {
+ struct spi_transfer *xfer;
+
+ xfer = &tef_ring->uinc_xfer[j];
+ xfer->tx_buf = &tef_ring->uinc_buf;
+ xfer->len = len;
+ xfer->cs_change = 1;
+ xfer->cs_change_delay.value = 0;
+ xfer->cs_change_delay.unit = SPI_DELAY_UNIT_NSECS;
+ }
/* TX */
tx_ring = priv->tx;
prev_rx_ring->obj_num;
prev_rx_ring = rx_ring;
+
+ /* FIFO increment RX tail pointer */
+ addr = MCP251XFD_REG_FIFOCON(rx_ring->fifo_nr);
+ val = MCP251XFD_REG_FIFOCON_UINC;
+ len = mcp251xfd_cmd_prepare_write_reg(priv, &rx_ring->uinc_buf,
+ addr, val, val);
+
+ for (j = 0; j < ARRAY_SIZE(rx_ring->uinc_xfer); j++) {
+ struct spi_transfer *xfer;
+
+ xfer = &rx_ring->uinc_xfer[j];
+ xfer->tx_buf = &rx_ring->uinc_buf;
+ xfer->len = len;
+ xfer->cs_change = 1;
+ xfer->cs_change_delay.value = 0;
+ xfer->cs_change_delay.unit = SPI_DELAY_UNIT_NSECS;
+ }
}
}
int rx_obj_num;
rx_obj_num = ram_free / rx_obj_size;
- rx_obj_num = min(1 << (fls(rx_obj_num) - 1), 32);
+ rx_obj_num = min(1 << (fls(rx_obj_num) - 1),
+ MCP251XFD_RX_OBJ_NUM_MAX);
rx_ring = kzalloc(sizeof(*rx_ring) + rx_obj_size * rx_obj_num,
GFP_KERNEL);
return 0;
}
- if (err)
- return err;
-
- return -ETIMEDOUT;
+ return err;
}
static int mcp251xfd_chip_clock_init(const struct mcp251xfd_priv *priv)
tef_sta & MCP251XFD_REG_TEFSTA_TEFFIF ?
"full" : tef_sta & MCP251XFD_REG_TEFSTA_TEFNEIF ?
"not empty" : "empty",
- seq, priv->tef.tail, priv->tef.head, tx_ring->head);
+ seq, priv->tef->tail, priv->tef->head, tx_ring->head);
/* The Sequence Number in the TEF doesn't match our tef_tail. */
return -EAGAIN;
mcp251xfd_handle_tefif_one(struct mcp251xfd_priv *priv,
const struct mcp251xfd_hw_tef_obj *hw_tef_obj)
{
- struct mcp251xfd_tx_ring *tx_ring = priv->tx;
struct net_device_stats *stats = &priv->ndev->stats;
u32 seq, seq_masked, tef_tail_masked;
- int err;
seq = FIELD_GET(MCP251XFD_OBJ_FLAGS_SEQ_MCP2518FD_MASK,
hw_tef_obj->flags);
*/
seq_masked = seq &
field_mask(MCP251XFD_OBJ_FLAGS_SEQ_MCP2517FD_MASK);
- tef_tail_masked = priv->tef.tail &
+ tef_tail_masked = priv->tef->tail &
field_mask(MCP251XFD_OBJ_FLAGS_SEQ_MCP2517FD_MASK);
if (seq_masked != tef_tail_masked)
return mcp251xfd_handle_tefif_recover(priv, seq);
mcp251xfd_get_tef_tail(priv),
hw_tef_obj->ts);
stats->tx_packets++;
+ priv->tef->tail++;
- /* finally increment the TEF pointer */
- err = regmap_update_bits(priv->map_reg, MCP251XFD_REG_TEFCON,
- GENMASK(15, 8),
- MCP251XFD_REG_TEFCON_UINC);
- if (err)
- return err;
-
- priv->tef.tail++;
- tx_ring->tail++;
-
- return mcp251xfd_check_tef_tail(priv);
+ return 0;
}
static int mcp251xfd_tef_ring_update(struct mcp251xfd_priv *priv)
/* chip_tx_tail, is the next TX-Object send by the HW.
* The new TEF head must be >= the old head, ...
*/
- new_head = round_down(priv->tef.head, tx_ring->obj_num) + chip_tx_tail;
- if (new_head <= priv->tef.head)
+ new_head = round_down(priv->tef->head, tx_ring->obj_num) + chip_tx_tail;
+ if (new_head <= priv->tef->head)
new_head += tx_ring->obj_num;
/* ... but it cannot exceed the TX head. */
- priv->tef.head = min(new_head, tx_ring->head);
+ priv->tef->head = min(new_head, tx_ring->head);
return mcp251xfd_check_tef_tail(priv);
}
}
out_netif_wake_queue:
+ len = i; /* number of handled goods TEFs */
+ if (len) {
+ struct mcp251xfd_tef_ring *ring = priv->tef;
+ struct mcp251xfd_tx_ring *tx_ring = priv->tx;
+ struct spi_transfer *last_xfer;
+
+ tx_ring->tail += len;
+
+ /* Increment the TEF FIFO tail pointer 'len' times in
+ * a single SPI message.
+ */
+
+ /* Note:
+ *
+ * "cs_change == 1" on the last transfer results in an
+ * active chip select after the complete SPI
+ * message. This causes the controller to interpret
+ * the next register access as data. Temporary set
+ * "cs_change" of the last transfer to "0" to properly
+ * deactivate the chip select at the end of the
+ * message.
+ */
+ last_xfer = &ring->uinc_xfer[len - 1];
+ last_xfer->cs_change = 0;
+ err = spi_sync_transfer(priv->spi, ring->uinc_xfer, len);
+ last_xfer->cs_change = 1;
+ if (err)
+ return err;
+
+ err = mcp251xfd_check_tef_tail(priv);
+ if (err)
+ return err;
+ }
+
mcp251xfd_ecc_tefif_successful(priv);
if (mcp251xfd_get_tx_free(priv->tx)) {
cfd->flags |= CANFD_BRS;
dlc = FIELD_GET(MCP251XFD_OBJ_FLAGS_DLC, hw_rx_obj->flags);
- cfd->len = can_dlc2len(get_canfd_dlc(dlc));
+ cfd->len = can_fd_dlc2len(dlc);
} else {
if (hw_rx_obj->flags & MCP251XFD_OBJ_FLAGS_RTR)
cfd->can_id |= CAN_RTR_FLAG;
- cfd->len = get_can_dlc(FIELD_GET(MCP251XFD_OBJ_FLAGS_DLC,
+ cfd->len = can_cc_dlc2len(FIELD_GET(MCP251XFD_OBJ_FLAGS_DLC,
hw_rx_obj->flags));
}
if (err)
stats->rx_fifo_errors++;
- ring->tail++;
-
- /* finally increment the RX pointer */
- return regmap_update_bits(priv->map_reg,
- MCP251XFD_REG_FIFOCON(ring->fifo_nr),
- GENMASK(15, 8),
- MCP251XFD_REG_FIFOCON_UINC);
+ return 0;
}
static inline int
return err;
while ((len = mcp251xfd_get_rx_linear_len(ring))) {
+ struct spi_transfer *last_xfer;
+
rx_tail = mcp251xfd_get_rx_tail(ring);
err = mcp251xfd_rx_obj_read(priv, ring, hw_rx_obj,
if (err)
return err;
}
+
+ /* Increment the RX FIFO tail pointer 'len' times in a
+ * single SPI message.
+ */
+ ring->tail += len;
+
+ /* Note:
+ *
+ * "cs_change == 1" on the last transfer results in an
+ * active chip select after the complete SPI
+ * message. This causes the controller to interpret
+ * the next register access as data. Temporary set
+ * "cs_change" of the last transfer to "0" to properly
+ * deactivate the chip select at the end of the
+ * message.
+ */
+ last_xfer = &ring->uinc_xfer[len - 1];
+ last_xfer->cs_change = 0;
+ err = spi_sync_transfer(priv->spi, ring->uinc_xfer, len);
+ last_xfer->cs_change = 1;
+ if (err)
+ return err;
}
return 0;
* harm, only the lower 7 bits will be transferred into the
* TEF object.
*/
- dlc = can_len2dlc(cfd->len);
+ dlc = can_fd_len2dlc(cfd->len);
flags |= FIELD_PREP(MCP251XFD_OBJ_FLAGS_SEQ_MCP2518FD_MASK, seq) |
FIELD_PREP(MCP251XFD_OBJ_FLAGS_DLC, dlc);
/* Clear data at end of CAN frame */
offset = round_down(cfd->len, sizeof(u32));
- len = round_up(can_dlc2len(dlc), sizeof(u32)) - offset;
+ len = round_up(can_fd_dlc2len(dlc), sizeof(u32)) - offset;
if (MCP251XFD_SANITIZE_CAN && len)
memset(hw_tx_obj->data + offset, 0x0, len);
memcpy(hw_tx_obj->data, cfd->data, cfd->len);
/* Number of bytes to be written into the RAM of the controller */
len = sizeof(hw_tx_obj->id) + sizeof(hw_tx_obj->flags);
if (MCP251XFD_SANITIZE_CAN)
- len += round_up(can_dlc2len(dlc), sizeof(u32));
+ len += round_up(can_fd_dlc2len(dlc), sizeof(u32));
else
len += round_up(cfd->len, sizeof(u32));
u32 freq;
int err;
+ if (!spi->irq)
+ return dev_err_probe(&spi->dev, -ENXIO,
+ "No IRQ specified (maybe node \"interrupts-extended\" in DT missing)!\n");
+
rx_int = devm_gpiod_get_optional(&spi->dev, "microchip,rx-int",
GPIOD_IN);
if (PTR_ERR(rx_int) == -EPROBE_DEFER)
* FIFO setup: tef: 8*12 bytes = 96 bytes, tx: 8*16 bytes = 128 bytes
* FIFO setup: tef: 4*12 bytes = 48 bytes, tx: 4*72 bytes = 288 bytes
*/
+#define MCP251XFD_RX_OBJ_NUM_MAX 32
#define MCP251XFD_TX_OBJ_NUM_CAN 8
#define MCP251XFD_TX_OBJ_NUM_CANFD 4
u8 data[sizeof_field(struct canfd_frame, data)];
};
-struct mcp251xfd_tef_ring {
- unsigned int head;
- unsigned int tail;
-
- /* u8 obj_num equals tx_ring->obj_num */
- /* u8 obj_size equals sizeof(struct mcp251xfd_hw_tef_obj) */
-};
-
struct __packed mcp251xfd_buf_cmd {
__be16 cmd;
};
union mcp251xfd_tx_obj_load_buf buf;
};
+struct mcp251xfd_tef_ring {
+ unsigned int head;
+ unsigned int tail;
+
+ /* u8 obj_num equals tx_ring->obj_num */
+ /* u8 obj_size equals sizeof(struct mcp251xfd_hw_tef_obj) */
+
+ union mcp251xfd_write_reg_buf uinc_buf;
+ struct spi_transfer uinc_xfer[MCP251XFD_TX_OBJ_NUM_MAX];
+};
+
struct mcp251xfd_tx_ring {
unsigned int head;
unsigned int tail;
u8 obj_num;
u8 obj_size;
+ union mcp251xfd_write_reg_buf uinc_buf;
+ struct spi_transfer uinc_xfer[MCP251XFD_RX_OBJ_NUM_MAX];
struct mcp251xfd_hw_rx_obj_canfd obj[];
};
struct spi_device *spi;
u32 spi_max_speed_hz_orig;
- struct mcp251xfd_tef_ring tef;
+ struct mcp251xfd_tef_ring tef[1];
struct mcp251xfd_tx_ring tx[1];
struct mcp251xfd_rx_ring *rx[1];
static inline u8 mcp251xfd_get_tef_head(const struct mcp251xfd_priv *priv)
{
- return priv->tef.head & (priv->tx->obj_num - 1);
+ return priv->tef->head & (priv->tx->obj_num - 1);
}
static inline u8 mcp251xfd_get_tef_tail(const struct mcp251xfd_priv *priv)
{
- return priv->tef.tail & (priv->tx->obj_num - 1);
+ return priv->tef->tail & (priv->tx->obj_num - 1);
}
static inline u8 mcp251xfd_get_tef_len(const struct mcp251xfd_priv *priv)
{
- return priv->tef.head - priv->tef.tail;
+ return priv->tef->head - priv->tef->tail;
}
static inline u8 mcp251xfd_get_tef_linear_len(const struct mcp251xfd_priv *priv)
netif_stop_queue(dev);
id = cf->can_id;
- dlc = cf->can_dlc;
+ dlc = cf->len;
msg_flag_n = dlc;
if (id & CAN_RTR_FLAG)
return;
fi = readl(priv->base + SUN4I_REG_BUF0_ADDR);
- cf->can_dlc = get_can_dlc(fi & 0x0F);
+ cf->len = can_cc_dlc2len(fi & 0x0F);
if (fi & SUN4I_MSG_EFF_FLAG) {
dreg = SUN4I_REG_BUF5_ADDR;
id = (readl(priv->base + SUN4I_REG_BUF1_ADDR) << 21) |
if (fi & SUN4I_MSG_RTR_FLAG)
id |= CAN_RTR_FLAG;
else
- for (i = 0; i < cf->can_dlc; i++)
+ for (i = 0; i < cf->len; i++)
cf->data[i] = readl(priv->base + dreg + i * 4);
cf->can_id = id;
sun4i_can_write_cmdreg(priv, SUN4I_CMD_RELEASE_RBUF);
stats->rx_packets++;
- stats->rx_bytes += cf->can_dlc;
+ stats->rx_bytes += cf->len;
netif_rx(skb);
can_led_event(dev, CAN_LED_EVENT_RX);
netdev_dbg(dev, "arbitration lost interrupt\n");
alc = readl(priv->base + SUN4I_REG_STA_ADDR);
priv->can.can_stats.arbitration_lost++;
- stats->tx_errors++;
if (likely(skb)) {
cf->can_id |= CAN_ERR_LOSTARB;
cf->data[0] = (alc >> 8) & 0x1f;
if (likely(skb)) {
stats->rx_packets++;
- stats->rx_bytes += cf->can_dlc;
+ stats->rx_bytes += cf->len;
netif_rx(skb);
} else {
return -ENOMEM;
spin_unlock_irqrestore(&priv->mbx_lock, flags);
/* Prepare mailbox for transmission */
- data = cf->can_dlc | (get_tx_head_prio(priv) << 8);
+ data = cf->len | (get_tx_head_prio(priv) << 8);
if (cf->can_id & CAN_RTR_FLAG) /* Remote transmission request */
data |= HECC_CANMCF_RTR;
hecc_write_mbx(priv, mbxno, HECC_CANMCF, data);
hecc_write_mbx(priv, mbxno, HECC_CANMID, data);
hecc_write_mbx(priv, mbxno, HECC_CANMDL,
be32_to_cpu(*(__be32 *)(cf->data)));
- if (cf->can_dlc > 4)
+ if (cf->len > 4)
hecc_write_mbx(priv, mbxno, HECC_CANMDH,
be32_to_cpu(*(__be32 *)(cf->data + 4)));
else
data = hecc_read_mbx(priv, mbxno, HECC_CANMCF);
if (data & HECC_CANMCF_RTR)
cf->can_id |= CAN_RTR_FLAG;
- cf->can_dlc = get_can_dlc(data & 0xF);
+ cf->len = can_cc_dlc2len(data & 0xF);
data = hecc_read_mbx(priv, mbxno, HECC_CANMDL);
*(__be32 *)(cf->data) = cpu_to_be32(data);
- if (cf->can_dlc > 4) {
+ if (cf->len > 4) {
data = hecc_read_mbx(priv, mbxno, HECC_CANMDH);
*(__be32 *)(cf->data + 4) = cpu_to_be32(data);
}
priv->base = devm_platform_ioremap_resource_byname(pdev, "hecc");
if (IS_ERR(priv->base)) {
dev_err(&pdev->dev, "hecc ioremap failed\n");
- return PTR_ERR(priv->base);
+ err = PTR_ERR(priv->base);
+ goto probe_exit_candev;
}
/* handle hecc-ram memory */
"hecc-ram");
if (IS_ERR(priv->hecc_ram)) {
dev_err(&pdev->dev, "hecc-ram ioremap failed\n");
- return PTR_ERR(priv->hecc_ram);
+ err = PTR_ERR(priv->hecc_ram);
+ goto probe_exit_candev;
}
/* handle mbx memory */
priv->mbx = devm_platform_ioremap_resource_byname(pdev, "mbx");
if (IS_ERR(priv->mbx)) {
dev_err(&pdev->dev, "mbx ioremap failed\n");
- return PTR_ERR(priv->mbx);
+ err = PTR_ERR(priv->mbx);
+ goto probe_exit_candev;
}
irq = platform_get_resource(pdev, IORESOURCE_IRQ, 0);
if (!irq) {
dev_err(&pdev->dev, "No irq resource\n");
- goto probe_exit;
+ goto probe_exit_candev;
}
priv->ndev = ndev;
clk_put(priv->clk);
probe_exit_candev:
free_candev(ndev);
-probe_exit:
+
return err;
}
- Kvaser Leaf Light "China"
- Kvaser BlackBird SemiPro
- Kvaser USBcan R
+ - Kvaser USBcan R v2
- Kvaser Leaf Light v2
+ - Kvaser Leaf Light R v2
- Kvaser Mini PCI Express HS
- Kvaser Mini PCI Express 2xHS
- Kvaser USBcan Light 2xHS
- Kvaser USBcan Light 4xHS
- Kvaser USBcan Pro 2xHS v2
- Kvaser USBcan Pro 5xHS
+ - Kvaser U100
+ - Kvaser U100P
+ - Kvaser U100S
- ATI Memorator Pro 2xHS v2
- ATI USBcan Pro 2xHS v2
return;
cf->can_id = le32_to_cpu(msg->msg.can_msg.id);
- cf->can_dlc = get_can_dlc(msg->msg.can_msg.length & 0xF);
+ cf->len = can_cc_dlc2len(msg->msg.can_msg.length & 0xF);
if (msg->type == CPC_MSG_TYPE_EXT_CAN_FRAME ||
msg->type == CPC_MSG_TYPE_EXT_RTR_FRAME)
msg->type == CPC_MSG_TYPE_EXT_RTR_FRAME) {
cf->can_id |= CAN_RTR_FLAG;
} else {
- for (i = 0; i < cf->can_dlc; i++)
+ for (i = 0; i < cf->len; i++)
cf->data[i] = msg->msg.can_msg.msg[i];
}
stats->rx_packets++;
- stats->rx_bytes += cf->can_dlc;
+ stats->rx_bytes += cf->len;
netif_rx(skb);
}
}
stats->rx_packets++;
- stats->rx_bytes += cf->can_dlc;
+ stats->rx_bytes += cf->len;
netif_rx(skb);
}
msg = (struct ems_cpc_msg *)&buf[CPC_HEADER_SIZE];
msg->msg.can_msg.id = cpu_to_le32(cf->can_id & CAN_ERR_MASK);
- msg->msg.can_msg.length = cf->can_dlc;
+ msg->msg.can_msg.length = cf->len;
if (cf->can_id & CAN_RTR_FLAG) {
msg->type = cf->can_id & CAN_EFF_FLAG ?
msg->type = cf->can_id & CAN_EFF_FLAG ?
CPC_CMD_TYPE_EXT_CAN_FRAME : CPC_CMD_TYPE_CAN_FRAME;
- for (i = 0; i < cf->can_dlc; i++)
+ for (i = 0; i < cf->len; i++)
msg->msg.can_msg.msg[i] = cf->data[i];
- msg->length = CPC_CAN_MSG_MIN_SIZE + cf->can_dlc;
+ msg->length = CPC_CAN_MSG_MIN_SIZE + cf->len;
}
for (i = 0; i < MAX_TX_URBS; i++) {
context->dev = dev;
context->echo_index = i;
- context->dlc = cf->can_dlc;
+ context->dlc = cf->len;
usb_fill_bulk_urb(urb, dev->udev, usb_sndbulkpipe(dev->udev, 2), buf,
size, ems_usb_write_bulk_callback, context);
struct esd_tx_urb_context {
struct esd_usb2_net_priv *priv;
u32 echo_index;
- int dlc;
+ int len; /* CAN payload length */
};
struct esd_usb2 {
priv->bec.rxerr = rxerr;
stats->rx_packets++;
- stats->rx_bytes += cf->can_dlc;
+ stats->rx_bytes += cf->len;
netif_rx(skb);
}
}
}
cf->can_id = id & ESD_IDMASK;
- cf->can_dlc = get_can_dlc(msg->msg.rx.dlc & ~ESD_RTR);
+ can_frame_set_cc_len(cf, msg->msg.rx.dlc & ~ESD_RTR,
+ priv->can.ctrlmode);
if (id & ESD_EXTID)
cf->can_id |= CAN_EFF_FLAG;
if (msg->msg.rx.dlc & ESD_RTR) {
cf->can_id |= CAN_RTR_FLAG;
} else {
- for (i = 0; i < cf->can_dlc; i++)
+ for (i = 0; i < cf->len; i++)
cf->data[i] = msg->msg.rx.data[i];
}
stats->rx_packets++;
- stats->rx_bytes += cf->can_dlc;
+ stats->rx_bytes += cf->len;
netif_rx(skb);
}
if (!msg->msg.txdone.status) {
stats->tx_packets++;
- stats->tx_bytes += context->dlc;
+ stats->tx_bytes += context->len;
can_get_echo_skb(netdev, context->echo_index);
} else {
stats->tx_errors++;
msg->msg.hdr.len = 3; /* minimal length */
msg->msg.hdr.cmd = CMD_CAN_TX;
msg->msg.tx.net = priv->index;
- msg->msg.tx.dlc = cf->can_dlc;
+ msg->msg.tx.dlc = can_get_cc_dlc(cf, priv->can.ctrlmode);
msg->msg.tx.id = cpu_to_le32(cf->can_id & CAN_ERR_MASK);
if (cf->can_id & CAN_RTR_FLAG)
if (cf->can_id & CAN_EFF_FLAG)
msg->msg.tx.id |= cpu_to_le32(ESD_EXTID);
- for (i = 0; i < cf->can_dlc; i++)
+ for (i = 0; i < cf->len; i++)
msg->msg.tx.data[i] = cf->data[i];
- msg->msg.hdr.len += (cf->can_dlc + 3) >> 2;
+ msg->msg.hdr.len += (cf->len + 3) >> 2;
for (i = 0; i < MAX_TX_URBS; i++) {
if (priv->tx_contexts[i].echo_index == MAX_TX_URBS) {
context->priv = priv;
context->echo_index = i;
- context->dlc = cf->can_dlc;
+ context->len = cf->len;
/* hnd must not be 0 - MSB is stripped in txdone handling */
msg->msg.tx.hnd = 0x80000000 | i; /* returned in TX done message */
priv->index = index;
priv->can.state = CAN_STATE_STOPPED;
- priv->can.ctrlmode_supported = CAN_CTRLMODE_LISTENONLY;
+ priv->can.ctrlmode_supported = CAN_CTRLMODE_LISTENONLY |
+ CAN_CTRLMODE_CC_LEN8_DLC;
if (le16_to_cpu(dev->udev->descriptor.idProduct) ==
USB_CANUSBM_PRODUCT_ID)
* Many thanks to all socketcan devs!
*/
+#include <linux/ethtool.h>
#include <linux/init.h>
#include <linux/signal.h>
#include <linux/module.h>
};
/* data types passed between host and device */
+
+/* The firmware on the original USB2CAN by Geschwister Schneider
+ * Technologie Entwicklungs- und Vertriebs UG exchanges all data
+ * between the host and the device in host byte order. This is done
+ * with the struct gs_host_config::byte_order member, which is sent
+ * first to indicate the desired byte order.
+ *
+ * The widely used open source firmware candleLight doesn't support
+ * this feature and exchanges the data in little endian byte order.
+ */
struct gs_host_config {
- u32 byte_order;
+ __le32 byte_order;
} __packed;
-/* All data exchanged between host and device is exchanged in host byte order,
- * thanks to the struct gs_host_config byte_order member, which is sent first
- * to indicate the desired byte order.
- */
struct gs_device_config {
u8 reserved1;
u8 reserved2;
u8 reserved3;
u8 icount;
- u32 sw_version;
- u32 hw_version;
+ __le32 sw_version;
+ __le32 hw_version;
} __packed;
#define GS_CAN_MODE_NORMAL 0
#define GS_CAN_MODE_ONE_SHOT BIT(3)
struct gs_device_mode {
- u32 mode;
- u32 flags;
+ __le32 mode;
+ __le32 flags;
} __packed;
struct gs_device_state {
- u32 state;
- u32 rxerr;
- u32 txerr;
+ __le32 state;
+ __le32 rxerr;
+ __le32 txerr;
} __packed;
struct gs_device_bittiming {
- u32 prop_seg;
- u32 phase_seg1;
- u32 phase_seg2;
- u32 sjw;
- u32 brp;
+ __le32 prop_seg;
+ __le32 phase_seg1;
+ __le32 phase_seg2;
+ __le32 sjw;
+ __le32 brp;
} __packed;
struct gs_identify_mode {
- u32 mode;
+ __le32 mode;
} __packed;
#define GS_CAN_FEATURE_LISTEN_ONLY BIT(0)
#define GS_CAN_FEATURE_IDENTIFY BIT(5)
struct gs_device_bt_const {
- u32 feature;
- u32 fclk_can;
- u32 tseg1_min;
- u32 tseg1_max;
- u32 tseg2_min;
- u32 tseg2_max;
- u32 sjw_max;
- u32 brp_min;
- u32 brp_max;
- u32 brp_inc;
+ __le32 feature;
+ __le32 fclk_can;
+ __le32 tseg1_min;
+ __le32 tseg1_max;
+ __le32 tseg2_min;
+ __le32 tseg2_max;
+ __le32 sjw_max;
+ __le32 brp_min;
+ __le32 brp_max;
+ __le32 brp_inc;
} __packed;
#define GS_CAN_FLAG_OVERFLOW 1
struct gs_host_frame {
u32 echo_id;
- u32 can_id;
+ __le32 can_id;
u8 can_dlc;
u8 channel;
if (!skb)
return;
- cf->can_id = hf->can_id;
+ cf->can_id = le32_to_cpu(hf->can_id);
- cf->can_dlc = get_can_dlc(hf->can_dlc);
+ can_frame_set_cc_len(cf, hf->can_dlc, dev->can.ctrlmode);
memcpy(cf->data, hf->data, 8);
/* ERROR frames tell us information about the controller */
- if (hf->can_id & CAN_ERR_FLAG)
+ if (le32_to_cpu(hf->can_id) & CAN_ERR_FLAG)
gs_update_state(dev, cf);
netdev->stats.rx_packets++;
goto resubmit_urb;
cf->can_id |= CAN_ERR_CRTL;
- cf->can_dlc = CAN_ERR_DLC;
+ cf->len = CAN_ERR_DLC;
cf->data[1] = CAN_ERR_CRTL_RX_OVERFLOW;
stats->rx_over_errors++;
stats->rx_errors++;
if (!dbt)
return -ENOMEM;
- dbt->prop_seg = bt->prop_seg;
- dbt->phase_seg1 = bt->phase_seg1;
- dbt->phase_seg2 = bt->phase_seg2;
- dbt->sjw = bt->sjw;
- dbt->brp = bt->brp;
+ dbt->prop_seg = cpu_to_le32(bt->prop_seg);
+ dbt->phase_seg1 = cpu_to_le32(bt->phase_seg1);
+ dbt->phase_seg2 = cpu_to_le32(bt->phase_seg2);
+ dbt->sjw = cpu_to_le32(bt->sjw);
+ dbt->brp = cpu_to_le32(bt->brp);
/* request bit timings */
rc = usb_control_msg(interface_to_usbdev(intf),
cf = (struct can_frame *)skb->data;
- hf->can_id = cf->can_id;
- hf->can_dlc = cf->can_dlc;
- memcpy(hf->data, cf->data, cf->can_dlc);
+ hf->can_id = cpu_to_le32(cf->can_id);
+ hf->can_dlc = can_get_cc_dlc(cf, dev->can.ctrlmode);
+
+ memcpy(hf->data, cf->data, cf->len);
usb_fill_bulk_urb(urb, dev->udev,
usb_sndbulkpipe(dev->udev, GSUSB_ENDPOINT_OUT),
int rc, i;
struct gs_device_mode *dm;
u32 ctrlmode;
+ u32 flags = 0;
rc = open_candev(netdev);
if (rc)
/* flags */
ctrlmode = dev->can.ctrlmode;
- dm->flags = 0;
if (ctrlmode & CAN_CTRLMODE_LOOPBACK)
- dm->flags |= GS_CAN_MODE_LOOP_BACK;
+ flags |= GS_CAN_MODE_LOOP_BACK;
else if (ctrlmode & CAN_CTRLMODE_LISTENONLY)
- dm->flags |= GS_CAN_MODE_LISTEN_ONLY;
+ flags |= GS_CAN_MODE_LISTEN_ONLY;
/* Controller is not allowed to retry TX
* this mode is unavailable on atmels uc3c hardware
*/
if (ctrlmode & CAN_CTRLMODE_ONE_SHOT)
- dm->flags |= GS_CAN_MODE_ONE_SHOT;
+ flags |= GS_CAN_MODE_ONE_SHOT;
if (ctrlmode & CAN_CTRLMODE_3_SAMPLES)
- dm->flags |= GS_CAN_MODE_TRIPLE_SAMPLE;
+ flags |= GS_CAN_MODE_TRIPLE_SAMPLE;
/* finally start device */
- dm->mode = GS_CAN_MODE_START;
+ dm->mode = cpu_to_le32(GS_CAN_MODE_START);
+ dm->flags = cpu_to_le32(flags);
rc = usb_control_msg(interface_to_usbdev(dev->iface),
usb_sndctrlpipe(interface_to_usbdev(dev->iface), 0),
GS_USB_BREQ_MODE,
return -ENOMEM;
if (do_identify)
- imode->mode = GS_CAN_IDENTIFY_ON;
+ imode->mode = cpu_to_le32(GS_CAN_IDENTIFY_ON);
else
- imode->mode = GS_CAN_IDENTIFY_OFF;
+ imode->mode = cpu_to_le32(GS_CAN_IDENTIFY_OFF);
rc = usb_control_msg(interface_to_usbdev(dev->iface),
usb_sndctrlpipe(interface_to_usbdev(dev->iface),
struct net_device *netdev;
int rc;
struct gs_device_bt_const *bt_const;
+ u32 feature;
bt_const = kmalloc(sizeof(*bt_const), GFP_KERNEL);
if (!bt_const)
/* dev setup */
strcpy(dev->bt_const.name, "gs_usb");
- dev->bt_const.tseg1_min = bt_const->tseg1_min;
- dev->bt_const.tseg1_max = bt_const->tseg1_max;
- dev->bt_const.tseg2_min = bt_const->tseg2_min;
- dev->bt_const.tseg2_max = bt_const->tseg2_max;
- dev->bt_const.sjw_max = bt_const->sjw_max;
- dev->bt_const.brp_min = bt_const->brp_min;
- dev->bt_const.brp_max = bt_const->brp_max;
- dev->bt_const.brp_inc = bt_const->brp_inc;
+ dev->bt_const.tseg1_min = le32_to_cpu(bt_const->tseg1_min);
+ dev->bt_const.tseg1_max = le32_to_cpu(bt_const->tseg1_max);
+ dev->bt_const.tseg2_min = le32_to_cpu(bt_const->tseg2_min);
+ dev->bt_const.tseg2_max = le32_to_cpu(bt_const->tseg2_max);
+ dev->bt_const.sjw_max = le32_to_cpu(bt_const->sjw_max);
+ dev->bt_const.brp_min = le32_to_cpu(bt_const->brp_min);
+ dev->bt_const.brp_max = le32_to_cpu(bt_const->brp_max);
+ dev->bt_const.brp_inc = le32_to_cpu(bt_const->brp_inc);
dev->udev = interface_to_usbdev(intf);
dev->iface = intf;
/* can setup */
dev->can.state = CAN_STATE_STOPPED;
- dev->can.clock.freq = bt_const->fclk_can;
+ dev->can.clock.freq = le32_to_cpu(bt_const->fclk_can);
dev->can.bittiming_const = &dev->bt_const;
dev->can.do_set_bittiming = gs_usb_set_bittiming;
- dev->can.ctrlmode_supported = 0;
+ dev->can.ctrlmode_supported = CAN_CTRLMODE_CC_LEN8_DLC;
- if (bt_const->feature & GS_CAN_FEATURE_LISTEN_ONLY)
+ feature = le32_to_cpu(bt_const->feature);
+ if (feature & GS_CAN_FEATURE_LISTEN_ONLY)
dev->can.ctrlmode_supported |= CAN_CTRLMODE_LISTENONLY;
- if (bt_const->feature & GS_CAN_FEATURE_LOOP_BACK)
+ if (feature & GS_CAN_FEATURE_LOOP_BACK)
dev->can.ctrlmode_supported |= CAN_CTRLMODE_LOOPBACK;
- if (bt_const->feature & GS_CAN_FEATURE_TRIPLE_SAMPLE)
+ if (feature & GS_CAN_FEATURE_TRIPLE_SAMPLE)
dev->can.ctrlmode_supported |= CAN_CTRLMODE_3_SAMPLES;
- if (bt_const->feature & GS_CAN_FEATURE_ONE_SHOT)
+ if (feature & GS_CAN_FEATURE_ONE_SHOT)
dev->can.ctrlmode_supported |= CAN_CTRLMODE_ONE_SHOT;
SET_NETDEV_DEV(netdev, &intf->dev);
- if (dconf->sw_version > 1)
- if (bt_const->feature & GS_CAN_FEATURE_IDENTIFY)
+ if (le32_to_cpu(dconf->sw_version) > 1)
+ if (feature & GS_CAN_FEATURE_IDENTIFY)
netdev->ethtool_ops = &gs_usb_ethtool_ops;
kfree(bt_const);
if (!hconf)
return -ENOMEM;
- hconf->byte_order = 0x0000beef;
+ hconf->byte_order = cpu_to_le32(0x0000beef);
/* send host config */
rc = usb_control_msg(interface_to_usbdev(intf),
#define USB_LEAF_LIGHT_HS_V2_OEM_PRODUCT_ID 290
#define USB_USBCAN_LIGHT_2HS_PRODUCT_ID 291
#define USB_MINI_PCIE_2HS_PRODUCT_ID 292
+#define USB_USBCAN_R_V2_PRODUCT_ID 294
+#define USB_LEAF_LIGHT_R_V2_PRODUCT_ID 295
+#define USB_LEAF_LIGHT_HS_V2_OEM2_PRODUCT_ID 296
+#define USB_LEAF_PRODUCT_ID_END \
+ USB_LEAF_LIGHT_HS_V2_OEM2_PRODUCT_ID
/* Kvaser USBCan-II devices product ids */
#define USB_USBCAN_REVB_PRODUCT_ID 2
#define USB_ATI_USBCAN_PRO_2HS_V2_PRODUCT_ID 268
#define USB_ATI_MEMO_PRO_2HS_V2_PRODUCT_ID 269
#define USB_HYBRID_PRO_CANLIN_PRODUCT_ID 270
+#define USB_U100_PRODUCT_ID 273
+#define USB_U100P_PRODUCT_ID 274
+#define USB_U100S_PRODUCT_ID 275
+#define USB_HYDRA_PRODUCT_ID_END \
+ USB_U100S_PRODUCT_ID
static inline bool kvaser_is_leaf(const struct usb_device_id *id)
{
return (id->idProduct >= USB_LEAF_DEVEL_PRODUCT_ID &&
id->idProduct <= USB_CAN_R_PRODUCT_ID) ||
(id->idProduct >= USB_LEAF_LITE_V2_PRODUCT_ID &&
- id->idProduct <= USB_MINI_PCIE_2HS_PRODUCT_ID);
+ id->idProduct <= USB_LEAF_PRODUCT_ID_END);
}
static inline bool kvaser_is_usbcan(const struct usb_device_id *id)
static inline bool kvaser_is_hydra(const struct usb_device_id *id)
{
return id->idProduct >= USB_BLACKBIRD_V2_PRODUCT_ID &&
- id->idProduct <= USB_HYBRID_PRO_CANLIN_PRODUCT_ID;
+ id->idProduct <= USB_HYDRA_PRODUCT_ID_END;
}
static const struct usb_device_id kvaser_usb_table[] = {
{ USB_DEVICE(KVASER_VENDOR_ID, USB_LEAF_LIGHT_HS_V2_OEM_PRODUCT_ID) },
{ USB_DEVICE(KVASER_VENDOR_ID, USB_USBCAN_LIGHT_2HS_PRODUCT_ID) },
{ USB_DEVICE(KVASER_VENDOR_ID, USB_MINI_PCIE_2HS_PRODUCT_ID) },
+ { USB_DEVICE(KVASER_VENDOR_ID, USB_USBCAN_R_V2_PRODUCT_ID) },
+ { USB_DEVICE(KVASER_VENDOR_ID, USB_LEAF_LIGHT_R_V2_PRODUCT_ID) },
+ { USB_DEVICE(KVASER_VENDOR_ID, USB_LEAF_LIGHT_HS_V2_OEM2_PRODUCT_ID) },
/* USBCANII USB product IDs */
{ USB_DEVICE(KVASER_VENDOR_ID, USB_USBCAN2_PRODUCT_ID),
{ USB_DEVICE(KVASER_VENDOR_ID, USB_ATI_USBCAN_PRO_2HS_V2_PRODUCT_ID) },
{ USB_DEVICE(KVASER_VENDOR_ID, USB_ATI_MEMO_PRO_2HS_V2_PRODUCT_ID) },
{ USB_DEVICE(KVASER_VENDOR_ID, USB_HYBRID_PRO_CANLIN_PRODUCT_ID) },
+ { USB_DEVICE(KVASER_VENDOR_ID, USB_U100_PRODUCT_ID) },
+ { USB_DEVICE(KVASER_VENDOR_ID, USB_U100P_PRODUCT_ID) },
+ { USB_DEVICE(KVASER_VENDOR_ID, USB_U100S_PRODUCT_ID) },
{ }
};
MODULE_DEVICE_TABLE(usb, kvaser_usb_table);
cf->data[1] = CAN_ERR_CRTL_RX_OVERFLOW;
stats->rx_packets++;
- stats->rx_bytes += cf->can_dlc;
+ stats->rx_bytes += cf->len;
netif_rx(skb);
return 0;
/* Forward declarations */
static const struct kvaser_usb_dev_cfg kvaser_usb_hydra_dev_cfg_kcan;
static const struct kvaser_usb_dev_cfg kvaser_usb_hydra_dev_cfg_flexc;
+static const struct kvaser_usb_dev_cfg kvaser_usb_hydra_dev_cfg_rt;
#define KVASER_USB_HYDRA_BULK_EP_IN_ADDR 0x82
#define KVASER_USB_HYDRA_BULK_EP_OUT_ADDR 0x02
#define KVASER_USB_HYDRA_SW_FLAG_CANFD BIT(10)
#define KVASER_USB_HYDRA_SW_FLAG_NONISO BIT(11)
#define KVASER_USB_HYDRA_SW_FLAG_EXT_CAP BIT(12)
+#define KVASER_USB_HYDRA_SW_FLAG_CAN_FREQ_80M BIT(13)
struct kvaser_cmd_sw_detail_res {
__le32 sw_flags;
__le32 sw_version;
.tseg2_max = 32,
.sjw_max = 16,
.brp_min = 1,
- .brp_max = 4096,
+ .brp_max = 8192,
.brp_inc = 1,
};
.brp_inc = 1,
};
+static const struct can_bittiming_const kvaser_usb_hydra_rt_bittiming_c = {
+ .name = "kvaser_usb_rt",
+ .tseg1_min = 2,
+ .tseg1_max = 96,
+ .tseg2_min = 2,
+ .tseg2_max = 32,
+ .sjw_max = 32,
+ .brp_min = 1,
+ .brp_max = 1024,
+ .brp_inc = 1,
+};
+
+static const struct can_bittiming_const kvaser_usb_hydra_rtd_bittiming_c = {
+ .name = "kvaser_usb_rt",
+ .tseg1_min = 2,
+ .tseg1_max = 39,
+ .tseg2_min = 2,
+ .tseg2_max = 8,
+ .sjw_max = 8,
+ .brp_min = 1,
+ .brp_max = 1024,
+ .brp_inc = 1,
+};
+
#define KVASER_USB_HYDRA_TRANSID_BITS 12
#define KVASER_USB_HYDRA_TRANSID_MASK \
GENMASK(KVASER_USB_HYDRA_TRANSID_BITS - 1, 0)
stats = &netdev->stats;
stats->rx_packets++;
- stats->rx_bytes += cf->can_dlc;
+ stats->rx_bytes += cf->len;
netif_rx(skb);
}
cf->data[7] = bec.rxerr;
stats->rx_packets++;
- stats->rx_bytes += cf->can_dlc;
+ stats->rx_bytes += cf->len;
netif_rx(skb);
priv->bec.txerr = bec.txerr;
stats->tx_errors++;
stats->rx_packets++;
- stats->rx_bytes += cf->can_dlc;
+ stats->rx_bytes += cf->len;
netif_rx(skb);
}
struct net_device_stats *stats = &priv->netdev->stats;
stats->tx_packets++;
- stats->tx_bytes += can_dlc2len(context->dlc);
+ stats->tx_bytes += can_fd_dlc2len(context->dlc);
}
spin_lock_irqsave(&priv->tx_contexts_lock, irq_flags);
if (flags & KVASER_USB_HYDRA_CF_FLAG_OVERRUN)
kvaser_usb_can_rx_over_error(priv->netdev);
- cf->can_dlc = get_can_dlc(cmd->rx_can.dlc);
+ cf->len = can_cc_dlc2len(cmd->rx_can.dlc);
if (flags & KVASER_USB_HYDRA_CF_FLAG_REMOTE_FRAME)
cf->can_id |= CAN_RTR_FLAG;
else
- memcpy(cf->data, cmd->rx_can.data, cf->can_dlc);
+ memcpy(cf->data, cmd->rx_can.data, cf->len);
stats->rx_packets++;
- stats->rx_bytes += cf->can_dlc;
+ stats->rx_bytes += cf->len;
netif_rx(skb);
}
kvaser_usb_can_rx_over_error(priv->netdev);
if (flags & KVASER_USB_HYDRA_CF_FLAG_FDF) {
- cf->len = can_dlc2len(get_canfd_dlc(dlc));
+ cf->len = can_fd_dlc2len(dlc);
if (flags & KVASER_USB_HYDRA_CF_FLAG_BRS)
cf->flags |= CANFD_BRS;
if (flags & KVASER_USB_HYDRA_CF_FLAG_ESI)
cf->flags |= CANFD_ESI;
} else {
- cf->len = get_can_dlc(dlc);
+ cf->len = can_cc_dlc2len(dlc);
}
if (flags & KVASER_USB_HYDRA_CF_FLAG_REMOTE_FRAME)
struct kvaser_usb *dev = priv->dev;
struct kvaser_cmd_ext *cmd;
struct canfd_frame *cf = (struct canfd_frame *)skb->data;
- u8 dlc = can_len2dlc(cf->len);
+ u8 dlc = can_fd_len2dlc(cf->len);
u8 nbr_of_bytes = cf->len;
u32 flags;
u32 id;
u32 flags;
u32 id;
- *frame_len = cf->can_dlc;
+ *frame_len = cf->len;
cmd = kcalloc(1, sizeof(struct kvaser_cmd), GFP_ATOMIC);
if (!cmd)
id = cf->can_id & CAN_SFF_MASK;
}
- cmd->tx_can.dlc = cf->can_dlc;
+ cmd->tx_can.dlc = cf->len;
flags = (cf->can_id & CAN_EFF_FLAG ?
KVASER_USB_HYDRA_CF_FLAG_EXTENDED_ID : 0);
if (flags & KVASER_USB_HYDRA_SW_FLAG_FREQ_80M)
dev->cfg = &kvaser_usb_hydra_dev_cfg_kcan;
+ else if (flags & KVASER_USB_HYDRA_SW_FLAG_CAN_FREQ_80M)
+ dev->cfg = &kvaser_usb_hydra_dev_cfg_rt;
else
dev->cfg = &kvaser_usb_hydra_dev_cfg_flexc;
.timestamp_freq = 1,
.bittiming_const = &kvaser_usb_hydra_flexc_bittiming_c,
};
+
+static const struct kvaser_usb_dev_cfg kvaser_usb_hydra_dev_cfg_rt = {
+ .clock = {
+ .freq = 80000000,
+ },
+ .timestamp_freq = 24,
+ .bittiming_const = &kvaser_usb_hydra_rt_bittiming_c,
+ .data_bittiming_const = &kvaser_usb_hydra_rtd_bittiming_c,
+};
u8 *cmd_tx_can_flags = NULL; /* GCC */
struct can_frame *cf = (struct can_frame *)skb->data;
- *frame_len = cf->can_dlc;
+ *frame_len = cf->len;
cmd = kmalloc(sizeof(*cmd), GFP_ATOMIC);
if (cmd) {
cmd->u.tx_can.data[1] = cf->can_id & 0x3f;
}
- cmd->u.tx_can.data[5] = cf->can_dlc;
- memcpy(&cmd->u.tx_can.data[6], cf->data, cf->can_dlc);
+ cmd->u.tx_can.data[5] = cf->len;
+ memcpy(&cmd->u.tx_can.data[6], cf->data, cf->len);
if (cf->can_id & CAN_RTR_FLAG)
*cmd_tx_can_flags |= MSG_FLAG_REMOTE_FRAME;
cf->can_id |= CAN_ERR_RESTARTED;
stats->rx_packets++;
- stats->rx_bytes += cf->can_dlc;
+ stats->rx_bytes += cf->len;
netif_rx(skb);
} else {
netdev_err(priv->netdev,
{
struct can_frame *cf;
struct can_frame tmp_cf = { .can_id = CAN_ERR_FLAG,
- .can_dlc = CAN_ERR_DLC };
+ .len = CAN_ERR_DLC };
struct sk_buff *skb;
struct net_device_stats *stats;
struct kvaser_usb_net_priv *priv;
cf->data[7] = es->rxerr;
stats->rx_packets++;
- stats->rx_bytes += cf->can_dlc;
+ stats->rx_bytes += cf->len;
netif_rx(skb);
}
else
cf->can_id &= CAN_SFF_MASK;
- cf->can_dlc = get_can_dlc(cmd->u.leaf.log_message.dlc);
+ cf->len = can_cc_dlc2len(cmd->u.leaf.log_message.dlc);
if (cmd->u.leaf.log_message.flags & MSG_FLAG_REMOTE_FRAME)
cf->can_id |= CAN_RTR_FLAG;
else
memcpy(cf->data, &cmd->u.leaf.log_message.data,
- cf->can_dlc);
+ cf->len);
} else {
cf->can_id = ((rx_data[0] & 0x1f) << 6) | (rx_data[1] & 0x3f);
cf->can_id |= CAN_EFF_FLAG;
}
- cf->can_dlc = get_can_dlc(rx_data[5]);
+ cf->len = can_cc_dlc2len(rx_data[5]);
if (cmd->u.rx_can_header.flag & MSG_FLAG_REMOTE_FRAME)
cf->can_id |= CAN_RTR_FLAG;
else
- memcpy(cf->data, &rx_data[6], cf->can_dlc);
+ memcpy(cf->data, &rx_data[6], cf->len);
}
stats->rx_packets++;
- stats->rx_bytes += cf->can_dlc;
+ stats->rx_bytes += cf->len;
netif_rx(skb);
}
if (cf) {
ctx->can = true;
- ctx->dlc = cf->can_dlc;
+ ctx->dlc = cf->len;
} else {
ctx->can = false;
ctx->dlc = 0;
if (!ctx)
return NETDEV_TX_BUSY;
- can_put_echo_skb(skb, priv->netdev, ctx->ndx);
-
if (cf->can_id & CAN_EFF_FLAG) {
/* SIDH | SIDL | EIDH | EIDL
* 28 - 21 | 20 19 18 x x x 17 16 | 15 - 8 | 7 - 0
usb_msg.eid = 0;
}
- usb_msg.dlc = cf->can_dlc;
+ usb_msg.dlc = cf->len;
memcpy(usb_msg.data, cf->data, usb_msg.dlc);
if (cf->can_id & CAN_RTR_FLAG)
usb_msg.dlc |= MCBA_DLC_RTR_MASK;
+ can_put_echo_skb(skb, priv->netdev, ctx->ndx);
+
err = mcba_usb_xmit(priv, (struct mcba_usb_msg *)&usb_msg, ctx);
if (err)
goto xmit_failed;
if (msg->dlc & MCBA_DLC_RTR_MASK)
cf->can_id |= CAN_RTR_FLAG;
- cf->can_dlc = get_can_dlc(msg->dlc & MCBA_DLC_MASK);
+ cf->len = can_cc_dlc2len(msg->dlc & MCBA_DLC_MASK);
- memcpy(cf->data, msg->data, cf->can_dlc);
+ memcpy(cf->data, msg->data, cf->len);
stats->rx_packets++;
- stats->rx_bytes += cf->can_dlc;
+ stats->rx_bytes += cf->len;
can_led_event(priv->netdev, CAN_LED_EVENT_RX);
netif_rx(skb);
}
mc->netdev->stats.rx_packets++;
- mc->netdev->stats.rx_bytes += cf->can_dlc;
+ mc->netdev->stats.rx_bytes += cf->len;
netif_rx(skb);
return 0;
cf->can_id = le16_to_cpu(tmp16) >> 5;
}
- cf->can_dlc = get_can_dlc(rec_len);
+ can_frame_set_cc_len(cf, rec_len, mc->pdev->dev.can.ctrlmode);
/* Only first packet timestamp is a word */
if (pcan_usb_decode_ts(mc, !mc->rec_ts_idx))
if ((mc->ptr + rec_len) > mc->end)
goto decode_failed;
- memcpy(cf->data, mc->ptr, cf->can_dlc);
+ memcpy(cf->data, mc->ptr, cf->len);
mc->ptr += rec_len;
}
/* update statistics */
mc->netdev->stats.rx_packets++;
- mc->netdev->stats.rx_bytes += cf->can_dlc;
+ mc->netdev->stats.rx_bytes += cf->len;
/* push the skb */
netif_rx(skb);
pc = obuf + PCAN_USB_MSG_HEADER_LEN;
/* status/len byte */
- *pc = cf->can_dlc;
+ *pc = can_get_cc_dlc(cf, dev->can.ctrlmode);
+
if (cf->can_id & CAN_RTR_FLAG)
*pc |= PCAN_USB_STATUSLEN_RTR;
/* can data */
if (!(cf->can_id & CAN_RTR_FLAG)) {
- memcpy(pc, cf->data, cf->can_dlc);
- pc += cf->can_dlc;
+ memcpy(pc, cf->data, cf->len);
+ pc += cf->len;
}
obuf[(*size)-1] = (u8)(stats->tx_packets & 0xff);
.device_id = PCAN_USB_PRODUCT_ID,
.ctrl_count = 1,
.ctrlmode_supported = CAN_CTRLMODE_3_SAMPLES | CAN_CTRLMODE_LISTENONLY |
- CAN_CTRLMODE_BERR_REPORTING,
+ CAN_CTRLMODE_BERR_REPORTING |
+ CAN_CTRLMODE_CC_LEN8_DLC,
.clock = {
.freq = PCAN_USB_CRYSTAL_HZ / 2 ,
},
if (time_ref->ts_dev_1 < time_ref->ts_dev_2) {
/* case when event time (tsw) wraps */
if (ts < time_ref->ts_dev_1)
- delta_ts = 1 << time_ref->adapter->ts_used_bits;
+ delta_ts = BIT_ULL(time_ref->adapter->ts_used_bits);
/* Otherwise, sync time counter (ts_dev_2) has wrapped:
* handle case when event time (tsn) hasn't.
* tsn ts
*/
} else if (time_ref->ts_dev_1 < ts) {
- delta_ts = -(1 << time_ref->adapter->ts_used_bits);
+ delta_ts = -BIT_ULL(time_ref->adapter->ts_used_bits);
}
/* add delay between last sync and event timestamps */
netif_trans_update(netdev);
break;
- default:
- if (net_ratelimit())
- netdev_err(netdev, "Tx urb aborted (%d)\n",
- urb->status);
case -EPROTO:
case -ENOENT:
case -ECONNRESET:
case -ESHUTDOWN:
+ break;
+ default:
+ if (net_ratelimit())
+ netdev_err(netdev, "Tx urb aborted (%d)\n",
+ urb->status);
break;
}
if (rx_msg_flags & PUCAN_MSG_ERROR_STATE_IND)
cfd->flags |= CANFD_ESI;
- cfd->len = can_dlc2len(get_canfd_dlc(pucan_msg_get_dlc(rm)));
+ cfd->len = can_fd_dlc2len(pucan_msg_get_dlc(rm));
} else {
/* CAN 2.0 frame case */
skb = alloc_can_skb(netdev, (struct can_frame **)&cfd);
if (!skb)
return -ENOMEM;
- cfd->len = get_can_dlc(pucan_msg_get_dlc(rm));
+ can_frame_set_cc_len((struct can_frame *)cfd,
+ pucan_msg_get_dlc(rm),
+ dev->can.ctrlmode);
}
cfd->can_id = le32_to_cpu(rm->can_id);
peak_usb_netif_rx(skb, &usb_if->time_ref, le32_to_cpu(sm->ts_low));
netdev->stats.rx_packets++;
- netdev->stats.rx_bytes += cf->can_dlc;
+ netdev->stats.rx_bytes += cf->len;
return 0;
}
struct pucan_tx_msg *tx_msg = (struct pucan_tx_msg *)obuf;
struct canfd_frame *cfd = (struct canfd_frame *)skb->data;
u16 tx_msg_size, tx_msg_flags;
- u8 can_dlc;
+ u8 dlc;
if (cfd->len > CANFD_MAX_DLEN)
return -EINVAL;
if (can_is_canfd_skb(skb)) {
/* considering a CANFD frame */
- can_dlc = can_len2dlc(cfd->len);
+ dlc = can_fd_len2dlc(cfd->len);
tx_msg_flags |= PUCAN_MSG_EXT_DATA_LEN;
tx_msg_flags |= PUCAN_MSG_ERROR_STATE_IND;
} else {
/* CAND 2.0 frames */
- can_dlc = cfd->len;
+ dlc = can_get_cc_dlc((struct can_frame *)cfd,
+ dev->can.ctrlmode);
if (cfd->can_id & CAN_RTR_FLAG)
tx_msg_flags |= PUCAN_MSG_RTR;
}
tx_msg->flags = cpu_to_le16(tx_msg_flags);
- tx_msg->channel_dlc = PUCAN_MSG_CHANNEL_DLC(dev->ctrl_idx, can_dlc);
+ tx_msg->channel_dlc = PUCAN_MSG_CHANNEL_DLC(dev->ctrl_idx, dlc);
memcpy(tx_msg->d, cfd->data, cfd->len);
/* add null size message to tag the end (messages are 32-bits aligned)
.device_id = PCAN_USBFD_PRODUCT_ID,
.ctrl_count = PCAN_USBFD_CHANNEL_COUNT,
.ctrlmode_supported = CAN_CTRLMODE_FD |
- CAN_CTRLMODE_3_SAMPLES | CAN_CTRLMODE_LISTENONLY,
+ CAN_CTRLMODE_3_SAMPLES | CAN_CTRLMODE_LISTENONLY |
+ CAN_CTRLMODE_CC_LEN8_DLC,
.clock = {
.freq = PCAN_UFD_CRYSTAL_HZ,
},
.device_id = PCAN_USBCHIP_PRODUCT_ID,
.ctrl_count = PCAN_USBFD_CHANNEL_COUNT,
.ctrlmode_supported = CAN_CTRLMODE_FD |
- CAN_CTRLMODE_3_SAMPLES | CAN_CTRLMODE_LISTENONLY,
+ CAN_CTRLMODE_3_SAMPLES | CAN_CTRLMODE_LISTENONLY |
+ CAN_CTRLMODE_CC_LEN8_DLC,
.clock = {
.freq = PCAN_UFD_CRYSTAL_HZ,
},
.device_id = PCAN_USBPROFD_PRODUCT_ID,
.ctrl_count = PCAN_USBPROFD_CHANNEL_COUNT,
.ctrlmode_supported = CAN_CTRLMODE_FD |
- CAN_CTRLMODE_3_SAMPLES | CAN_CTRLMODE_LISTENONLY,
+ CAN_CTRLMODE_3_SAMPLES | CAN_CTRLMODE_LISTENONLY |
+ CAN_CTRLMODE_CC_LEN8_DLC,
.clock = {
.freq = PCAN_UFD_CRYSTAL_HZ,
},
.device_id = PCAN_USBX6_PRODUCT_ID,
.ctrl_count = PCAN_USBPROFD_CHANNEL_COUNT,
.ctrlmode_supported = CAN_CTRLMODE_FD |
- CAN_CTRLMODE_3_SAMPLES | CAN_CTRLMODE_LISTENONLY,
+ CAN_CTRLMODE_3_SAMPLES | CAN_CTRLMODE_LISTENONLY |
+ CAN_CTRLMODE_CC_LEN8_DLC,
.clock = {
.freq = PCAN_UFD_CRYSTAL_HZ,
},
return -ENOMEM;
can_frame->can_id = le32_to_cpu(rx->id);
- can_frame->can_dlc = rx->len & 0x0f;
+ can_frame->len = rx->len & 0x0f;
if (rx->flags & PCAN_USBPRO_EXT)
can_frame->can_id |= CAN_EFF_FLAG;
if (rx->flags & PCAN_USBPRO_RTR)
can_frame->can_id |= CAN_RTR_FLAG;
else
- memcpy(can_frame->data, rx->data, can_frame->can_dlc);
+ memcpy(can_frame->data, rx->data, can_frame->len);
hwts = skb_hwtstamps(skb);
peak_usb_get_ts_time(&usb_if->time_ref, le32_to_cpu(rx->ts32),
&hwts->hwtstamp);
netdev->stats.rx_packets++;
- netdev->stats.rx_bytes += can_frame->can_dlc;
+ netdev->stats.rx_bytes += can_frame->len;
netif_rx(skb);
return 0;
hwts = skb_hwtstamps(skb);
peak_usb_get_ts_time(&usb_if->time_ref, le32_to_cpu(er->ts32), &hwts->hwtstamp);
netdev->stats.rx_packets++;
- netdev->stats.rx_bytes += can_frame->can_dlc;
+ netdev->stats.rx_bytes += can_frame->len;
netif_rx(skb);
return 0;
pcan_msg_init_empty(&usb_msg, obuf, *size);
- if ((cf->can_id & CAN_RTR_FLAG) || (cf->can_dlc == 0))
+ if ((cf->can_id & CAN_RTR_FLAG) || (cf->len == 0))
data_type = PCAN_USBPRO_TXMSG0;
- else if (cf->can_dlc <= 4)
+ else if (cf->len <= 4)
data_type = PCAN_USBPRO_TXMSG4;
else
data_type = PCAN_USBPRO_TXMSG8;
- len = (dev->ctrl_idx << 4) | (cf->can_dlc & 0x0f);
+ len = (dev->ctrl_idx << 4) | (cf->len & 0x0f);
flags = 0;
if (cf->can_id & CAN_EFF_FLAG)
struct ucan_urb_context *context_array;
};
-static u8 ucan_get_can_dlc(struct ucan_can_msg *msg, u16 len)
+static u8 ucan_can_cc_dlc2len(struct ucan_can_msg *msg, u16 len)
{
if (le32_to_cpu(msg->id) & CAN_RTR_FLAG)
- return get_can_dlc(msg->dlc);
+ return can_cc_dlc2len(msg->dlc);
else
- return get_can_dlc(len - (UCAN_IN_HDR_SIZE + sizeof(msg->id)));
+ return can_cc_dlc2len(len - (UCAN_IN_HDR_SIZE + sizeof(msg->id)));
}
static void ucan_release_context_array(struct ucan_priv *up)
cf->can_id = canid;
/* compute DLC taking RTR_FLAG into account */
- cf->can_dlc = ucan_get_can_dlc(&m->msg.can_msg, len);
+ cf->len = ucan_can_cc_dlc2len(&m->msg.can_msg, len);
/* copy the payload of non RTR frames */
if (!(cf->can_id & CAN_RTR_FLAG) || (cf->can_id & CAN_ERR_FLAG))
- memcpy(cf->data, m->msg.can_msg.data, cf->can_dlc);
+ memcpy(cf->data, m->msg.can_msg.data, cf->len);
/* don't count error frames as real packets */
stats->rx_packets++;
- stats->rx_bytes += cf->can_dlc;
+ stats->rx_bytes += cf->len;
/* pass it to Linux */
netif_rx(skb);
mlen = UCAN_OUT_HDR_SIZE +
offsetof(struct ucan_can_msg, dlc) +
sizeof(m->msg.can_msg.dlc);
- m->msg.can_msg.dlc = cf->can_dlc;
+ m->msg.can_msg.dlc = cf->len;
} else {
mlen = UCAN_OUT_HDR_SIZE +
- sizeof(m->msg.can_msg.id) + cf->can_dlc;
- memcpy(m->msg.can_msg.data, cf->data, cf->can_dlc);
+ sizeof(m->msg.can_msg.id) + cf->len;
+ memcpy(m->msg.can_msg.data, cf->data, cf->len);
}
m->len = cpu_to_le16(mlen);
- context->dlc = cf->can_dlc;
+ context->dlc = cf->len;
m->subtype = echo_index;
priv->bec.rxerr = rxerr;
stats->rx_packets++;
- stats->rx_bytes += cf->can_dlc;
+ stats->rx_bytes += cf->len;
netif_rx(skb);
}
return;
cf->can_id = be32_to_cpu(msg->id);
- cf->can_dlc = get_can_dlc(msg->dlc & 0xF);
+ can_frame_set_cc_len(cf, msg->dlc & 0xF, priv->can.ctrlmode);
if (msg->flags & USB_8DEV_EXTID)
cf->can_id |= CAN_EFF_FLAG;
if (msg->flags & USB_8DEV_RTR)
cf->can_id |= CAN_RTR_FLAG;
else
- memcpy(cf->data, msg->data, cf->can_dlc);
+ memcpy(cf->data, msg->data, cf->len);
stats->rx_packets++;
- stats->rx_bytes += cf->can_dlc;
+ stats->rx_bytes += cf->len;
netif_rx(skb);
can_led_event(priv->netdev, CAN_LED_EVENT_RX);
msg->flags |= USB_8DEV_EXTID;
msg->id = cpu_to_be32(cf->can_id & CAN_ERR_MASK);
- msg->dlc = cf->can_dlc;
- memcpy(msg->data, cf->data, cf->can_dlc);
+ msg->dlc = can_get_cc_dlc(cf, priv->can.ctrlmode);
+ memcpy(msg->data, cf->data, cf->len);
msg->end = USB_8DEV_DATA_END;
for (i = 0; i < MAX_TX_URBS; i++) {
context->priv = priv;
context->echo_index = i;
- context->dlc = cf->can_dlc;
+ context->dlc = cf->len;
usb_fill_bulk_urb(urb, priv->udev,
usb_sndbulkpipe(priv->udev, USB_8DEV_ENDP_DATA_TX),
priv->can.do_get_berr_counter = usb_8dev_get_berr_counter;
priv->can.ctrlmode_supported = CAN_CTRLMODE_LOOPBACK |
CAN_CTRLMODE_LISTENONLY |
- CAN_CTRLMODE_ONE_SHOT;
+ CAN_CTRLMODE_ONE_SHOT |
+ CAN_CTRLMODE_CC_LEN8_DLC;
netdev->netdev_ops = &usb_8dev_netdev_ops;
}
if (ifmp && tbp[IFLA_IFNAME]) {
- nla_strlcpy(ifname, tbp[IFLA_IFNAME], IFNAMSIZ);
+ nla_strscpy(ifname, tbp[IFLA_IFNAME], IFNAMSIZ);
name_assign_type = NET_NAME_USER;
} else {
snprintf(ifname, IFNAMSIZ, DRV_NAME "%%d");
/* register first device */
if (tb[IFLA_IFNAME])
- nla_strlcpy(dev->name, tb[IFLA_IFNAME], IFNAMSIZ);
+ nla_strscpy(dev->name, tb[IFLA_IFNAME], IFNAMSIZ);
else
snprintf(dev->name, IFNAMSIZ, DRV_NAME "%%d");
id |= XCAN_IDR_SRR_MASK;
}
- dlc = can_len2dlc(cf->len) << XCAN_DLCR_DLC_SHIFT;
+ dlc = can_fd_len2dlc(cf->len) << XCAN_DLCR_DLC_SHIFT;
if (can_is_canfd_skb(skb)) {
if (cf->flags & CANFD_BRS)
dlc |= XCAN_DLCR_BRS_MASK;
XCAN_DLCR_DLC_SHIFT;
/* Change Xilinx CAN data length format to socketCAN data format */
- cf->can_dlc = get_can_dlc(dlc);
+ cf->len = can_cc_dlc2len(dlc);
/* Change Xilinx CAN ID format to socketCAN ID format */
if (id_xcan & XCAN_IDR_IDE_MASK) {
if (!(cf->can_id & CAN_RTR_FLAG)) {
/* Change Xilinx CAN data format to socketCAN data format */
- if (cf->can_dlc > 0)
+ if (cf->len > 0)
*(__be32 *)(cf->data) = cpu_to_be32(data[0]);
- if (cf->can_dlc > 4)
+ if (cf->len > 4)
*(__be32 *)(cf->data + 4) = cpu_to_be32(data[1]);
}
- stats->rx_bytes += cf->can_dlc;
+ stats->rx_bytes += cf->len;
stats->rx_packets++;
netif_receive_skb(skb);
* format
*/
if (dlc & XCAN_DLCR_EDL_MASK)
- cf->len = can_dlc2len((dlc & XCAN_DLCR_DLC_MASK) >>
+ cf->len = can_fd_dlc2len((dlc & XCAN_DLCR_DLC_MASK) >>
XCAN_DLCR_DLC_SHIFT);
else
- cf->len = get_can_dlc((dlc & XCAN_DLCR_DLC_MASK) >>
+ cf->len = can_cc_dlc2len((dlc & XCAN_DLCR_DLC_MASK) >>
XCAN_DLCR_DLC_SHIFT);
/* Change Xilinx CAN ID format to socketCAN ID format */
struct net_device_stats *stats = &ndev->stats;
stats->rx_packets++;
- stats->rx_bytes += cf->can_dlc;
+ stats->rx_bytes += cf->len;
netif_rx(skb);
}
}
ret = dsa_register_switch(hellcreek->ds);
if (ret) {
- dev_err(dev, "Unable to register switch\n");
+ dev_err_probe(dev, ret, "Unable to register switch\n");
return ret;
}
*/
#include <linux/clk.h>
+#include <linux/delay.h>
#include <linux/etherdevice.h>
#include <linux/firmware.h>
#include <linux/if_bridge.h>
i++;
}
+ /* The standalone PHY11G requires 300ms to be fully
+ * initialized and ready for any MDIO communication after being
+ * taken out of reset. For the SoC-internal GPHY variant there
+ * is no (known) documentation for the minimum time after a
+ * reset. Use the same value as for the standalone variant as
+ * some users have reported internal PHYs not being detected
+ * without any delay.
+ */
+ msleep(300);
+
return 0;
remove_gphy:
static const struct {
char string[ETH_GSTRING_LEN];
-} mib_names[TOTAL_SWITCH_COUNTER_NUM] = {
+} mib_names[] = {
{ "rx_hi" },
{ "rx_undersize" },
{ "rx_fragments" },
u8 check;
int loop;
- ctrl_addr = addr + SWITCH_COUNTER_NUM * port;
+ ctrl_addr = addr + dev->reg_mib_cnt * port;
ctrl_addr |= IND_ACC_TABLE(TABLE_MIB | TABLE_READ);
mutex_lock(&dev->alu_mutex);
u8 check;
int loop;
- addr -= SWITCH_COUNTER_NUM;
+ addr -= dev->reg_mib_cnt;
ctrl_addr = (KS_MIB_TOTAL_RX_1 - KS_MIB_TOTAL_RX_0) * port;
ctrl_addr += addr + KS_MIB_TOTAL_RX_0;
ctrl_addr |= IND_ACC_TABLE(TABLE_MIB | TABLE_READ);
int i;
ksz8795_r_table(dev, TABLE_VLAN, addr, &data);
- addr *= 4;
- for (i = 0; i < 4; i++) {
+ addr *= dev->phy_port_cnt;
+ for (i = 0; i < dev->phy_port_cnt; i++) {
dev->vlan_cache[addr + i].table[0] = (u16)data;
data >>= VLAN_TABLE_S;
}
u64 buf;
data = (u16 *)&buf;
- addr = vid / 4;
+ addr = vid / dev->phy_port_cnt;
index = vid & 3;
ksz8795_r_table(dev, TABLE_VLAN, addr, &buf);
*vlan = data[index];
u64 buf;
data = (u16 *)&buf;
- addr = vid / 4;
+ addr = vid / dev->phy_port_cnt;
index = vid & 3;
ksz8795_r_table(dev, TABLE_VLAN, addr, &buf);
data[index] = vlan;
static void ksz8795_get_strings(struct dsa_switch *ds, int port,
u32 stringset, uint8_t *buf)
{
+ struct ksz_device *dev = ds->priv;
int i;
- for (i = 0; i < TOTAL_SWITCH_COUNTER_NUM; i++) {
+ for (i = 0; i < dev->mib_cnt; i++) {
memcpy(buf + i * ETH_GSTRING_LEN, mib_names[i].string,
ETH_GSTRING_LEN);
}
switch (state) {
case BR_STATE_DISABLED:
data |= PORT_LEARN_DISABLE;
- if (port < SWITCH_PORT_NUM)
+ if (port < dev->phy_port_cnt)
member = 0;
break;
case BR_STATE_LISTENING:
data |= (PORT_RX_ENABLE | PORT_LEARN_DISABLE);
- if (port < SWITCH_PORT_NUM &&
+ if (port < dev->phy_port_cnt &&
p->stp_state == BR_STATE_DISABLED)
member = dev->host_mask | p->vid_member;
break;
break;
case BR_STATE_BLOCKING:
data |= PORT_LEARN_DISABLE;
- if (port < SWITCH_PORT_NUM &&
+ if (port < dev->phy_port_cnt &&
p->stp_state == BR_STATE_DISABLED)
member = dev->host_mask | p->vid_member;
break;
static void ksz8795_flush_dyn_mac_table(struct ksz_device *dev, int port)
{
- u8 learn[TOTAL_PORT_NUM];
+ u8 learn[DSA_MAX_PORTS];
int first, index, cnt;
struct ksz_port *p;
- if ((uint)port < TOTAL_PORT_NUM) {
+ if ((uint)port < dev->port_cnt) {
first = port;
cnt = port + 1;
} else {
/* Flush all ports. */
first = 0;
- cnt = dev->mib_port_cnt;
+ cnt = dev->port_cnt;
}
for (index = first; index < cnt; index++) {
p = &dev->ports[index];
u8 remote;
int i;
- ds->num_ports = dev->port_cnt + 1;
-
/* Switch marks the maximum frame with extra byte as oversize. */
ksz_cfg(dev, REG_SW_CTRL_2, SW_LEGAL_PACKET_DISABLE, true);
ksz_cfg(dev, S_TAIL_TAG_CTRL, SW_TAIL_TAG_ENABLE, true);
ksz8795_port_setup(dev, dev->cpu_port, true);
dev->member = dev->host_mask;
- for (i = 0; i < SWITCH_PORT_NUM; i++) {
+ for (i = 0; i < dev->phy_port_cnt; i++) {
p = &dev->ports[i];
/* Initialize to non-zero so that ksz_cfg_port_member() will
ksz8795_port_stp_state_set(ds, i, BR_STATE_DISABLED);
/* Last port may be disabled. */
- if (i == dev->port_cnt)
+ if (i == dev->phy_port_cnt)
break;
p->on = 1;
p->phy = 1;
(BROADCAST_STORM_VALUE *
BROADCAST_STORM_PROT_RATE) / 100);
- for (i = 0; i < VLAN_TABLE_ENTRIES; i++)
+ for (i = 0; i < (dev->num_vlans / 4); i++)
ksz8795_r_vlan_entries(dev, i);
/* Setup STP address for STP operation. */
(id2 != CHIP_ID_94 && id2 != CHIP_ID_95))
return -ENODEV;
- dev->mib_port_cnt = TOTAL_PORT_NUM;
- dev->phy_port_cnt = SWITCH_PORT_NUM;
- dev->port_cnt = SWITCH_PORT_NUM;
-
if (id2 == CHIP_ID_95) {
u8 val;
if (val & PORT_FIBER_MODE)
id2 = 0x65;
} else if (id2 == CHIP_ID_94) {
- dev->port_cnt--;
- dev->last_port = dev->port_cnt;
id2 = 0x94;
}
id16 &= ~0xff;
id16 |= id2;
dev->chip_id = id16;
- dev->cpu_port = dev->mib_port_cnt - 1;
- dev->host_mask = BIT(dev->cpu_port);
-
return 0;
}
.num_alus = 0,
.num_statics = 8,
.cpu_ports = 0x10, /* can be configured as cpu port */
- .port_cnt = 4, /* total physical port count */
+ .port_cnt = 5, /* total cpu and user ports */
},
{
.chip_id = 0x8794,
.num_alus = 0,
.num_statics = 8,
.cpu_ports = 0x10, /* can be configured as cpu port */
- .port_cnt = 3, /* total physical port count */
+ .port_cnt = 4, /* total cpu and user ports */
},
{
.chip_id = 0x8765,
.num_alus = 0,
.num_statics = 8,
.cpu_ports = 0x10, /* can be configured as cpu port */
- .port_cnt = 4, /* total physical port count */
+ .port_cnt = 5, /* total cpu and user ports */
},
};
dev->port_mask = BIT(dev->port_cnt) - 1;
dev->port_mask |= dev->host_mask;
- dev->reg_mib_cnt = SWITCH_COUNTER_NUM;
- dev->mib_cnt = TOTAL_SWITCH_COUNTER_NUM;
+ dev->reg_mib_cnt = KSZ8795_COUNTER_NUM;
+ dev->mib_cnt = ARRAY_SIZE(mib_names);
+
+ dev->phy_port_cnt = dev->port_cnt - 1;
+
+ dev->cpu_port = dev->port_cnt - 1;
+ dev->host_mask = BIT(dev->cpu_port);
- i = dev->mib_port_cnt;
- dev->ports = devm_kzalloc(dev->dev, sizeof(struct ksz_port) * i,
+ dev->ports = devm_kzalloc(dev->dev,
+ dev->port_cnt * sizeof(struct ksz_port),
GFP_KERNEL);
if (!dev->ports)
return -ENOMEM;
- for (i = 0; i < dev->mib_port_cnt; i++) {
+ for (i = 0; i < dev->port_cnt; i++) {
mutex_init(&dev->ports[i].mib.cnt_mutex);
dev->ports[i].mib.counters =
devm_kzalloc(dev->dev,
sizeof(u64) *
- (TOTAL_SWITCH_COUNTER_NUM + 1),
+ (dev->mib_cnt + 1),
GFP_KERNEL);
if (!dev->ports[i].mib.counters)
return -ENOMEM;
}
/* set the real number of ports */
- dev->ds->num_ports = dev->port_cnt + 1;
+ dev->ds->num_ports = dev->port_cnt;
return 0;
}
#define KS_PRIO_IN_REG 4
-#define TOTAL_PORT_NUM 5
-
-/* Host port can only be last of them. */
-#define SWITCH_PORT_NUM (TOTAL_PORT_NUM - 1)
-
#define KSZ8795_COUNTER_NUM 0x20
-#define TOTAL_KSZ8795_COUNTER_NUM (KSZ8795_COUNTER_NUM + 4)
-
-#define SWITCH_COUNTER_NUM KSZ8795_COUNTER_NUM
-#define TOTAL_SWITCH_COUNTER_NUM TOTAL_KSZ8795_COUNTER_NUM
/* Common names used by other drivers */
#define TAIL_TAG_OVERRIDE BIT(6)
#define TAIL_TAG_LOOKUP BIT(7)
-#define VLAN_TABLE_ENTRIES (4096 / 4)
#define FID_ENTRIES 128
#endif
if (spi->dev.platform_data)
dev->pdata = spi->dev.platform_data;
+ /* setup spi */
+ spi->mode = SPI_MODE_3;
+ ret = spi_setup(spi);
+ if (ret)
+ return ret;
+
ret = ksz8795_switch_register(dev);
/* Main DSA driver may not be started yet. */
SW_FLUSH_OPTION_M << SW_FLUSH_OPTION_S,
SW_FLUSH_OPTION_DYN_MAC << SW_FLUSH_OPTION_S);
- if (port < dev->mib_port_cnt) {
+ if (port < dev->port_cnt) {
/* flush individual port */
ksz_pread8(dev, port, P_STP_CTRL, &data);
if (!(data & PORT_LEARN_DISABLE))
struct ksz_port *p;
int i;
- ds->num_ports = dev->port_cnt;
-
for (i = 0; i < dev->port_cnt; i++) {
if (dsa_is_cpu_port(ds, i) && (dev->cpu_ports & (1 << i))) {
phy_interface_t interface;
dev->member = dev->host_mask;
- for (i = 0; i < dev->mib_port_cnt; i++) {
+ for (i = 0; i < dev->port_cnt; i++) {
if (i == dev->cpu_port)
continue;
p = &dev->ports[i];
return ret;
/* Number of ports can be reduced depending on chip. */
- dev->mib_port_cnt = TOTAL_PORT_NUM;
dev->phy_port_cnt = 5;
/* Default capability is gigabit capable. */
/* Chip does not support gigabit. */
if (data8 & SW_QW_ABLE)
dev->features &= ~GBIT_SUPPORT;
- dev->mib_port_cnt = 3;
dev->phy_port_cnt = 2;
} else {
dev_info(dev->dev, "Found KSZ9477 or compatible\n");
dev->reg_mib_cnt = SWITCH_COUNTER_NUM;
dev->mib_cnt = TOTAL_SWITCH_COUNTER_NUM;
- i = dev->mib_port_cnt;
- dev->ports = devm_kzalloc(dev->dev, sizeof(struct ksz_port) * i,
+ dev->ports = devm_kzalloc(dev->dev,
+ dev->port_cnt * sizeof(struct ksz_port),
GFP_KERNEL);
if (!dev->ports)
return -ENOMEM;
- for (i = 0; i < dev->mib_port_cnt; i++) {
+ for (i = 0; i < dev->port_cnt; i++) {
mutex_init(&dev->ports[i].mib.cnt_mutex);
dev->ports[i].mib.counters =
devm_kzalloc(dev->dev,
if (spi->dev.platform_data)
dev->pdata = spi->dev.platform_data;
+ /* setup spi */
+ spi->mode = SPI_MODE_3;
+ ret = spi_setup(spi);
+ if (ret)
+ return ret;
+
ret = ksz9477_switch_register(dev);
/* Main DSA driver may not be started yet. */
struct ksz_port *p;
int i;
- for (i = 0; i < dev->mib_port_cnt; i++) {
+ for (i = 0; i < dev->port_cnt; i++) {
if (dsa_is_unused_port(dev->ds, i))
continue;
INIT_DELAYED_WORK(&dev->mib_read, ksz_mib_read_work);
- for (i = 0; i < dev->mib_port_cnt; i++)
+ for (i = 0; i < dev->port_cnt; i++)
dev->dev_ops->port_init_cnt(dev, i);
}
EXPORT_SYMBOL_GPL(ksz_init_mib_timer);
ret = of_get_phy_mode(dev->dev->of_node, &interface);
if (ret == 0)
dev->compat_interface = interface;
- ports = of_get_child_by_name(dev->dev->of_node, "ports");
+ ports = of_get_child_by_name(dev->dev->of_node, "ethernet-ports");
+ if (!ports)
+ ports = of_get_child_by_name(dev->dev->of_node, "ports");
if (ports)
for_each_available_child_of_node(ports, port) {
if (of_property_read_u32(port, "reg",
int port_cnt;
int reg_mib_cnt;
int mib_cnt;
- int mib_port_cnt;
- int last_port; /* ports after that not used */
phy_interface_t compat_interface;
u32 regs_size;
bool phy_errata_9477;
mv88e6xxx_reg_lock(chip);
if ((!mv88e6xxx_port_ppu_updates(chip, port) ||
- mode == MLO_AN_FIXED) && ops->port_set_link)
- err = ops->port_set_link(chip, port, LINK_FORCED_DOWN);
+ mode == MLO_AN_FIXED) && ops->port_sync_link)
+ err = ops->port_sync_link(chip, port, mode, false);
mv88e6xxx_reg_unlock(chip);
if (err)
goto error;
}
- if (ops->port_set_link)
- err = ops->port_set_link(chip, port, LINK_FORCED_UP);
+ if (ops->port_sync_link)
+ err = ops->port_sync_link(chip, port, mode, true);
}
error:
mv88e6xxx_reg_unlock(chip);
usleep_range(10000, 20000);
gpiod_set_value_cansleep(gpiod, 0);
usleep_range(10000, 20000);
+
+ mv88e6xxx_g1_wait_eeprom_done(chip);
}
}
.phy_read = mv88e6185_phy_ppu_read,
.phy_write = mv88e6185_phy_ppu_write,
.port_set_link = mv88e6xxx_port_set_link,
+ .port_sync_link = mv88e6xxx_port_sync_link,
.port_set_speed_duplex = mv88e6185_port_set_speed_duplex,
.port_tag_remap = mv88e6095_port_tag_remap,
.port_set_frame_mode = mv88e6351_port_set_frame_mode,
.phy_read = mv88e6185_phy_ppu_read,
.phy_write = mv88e6185_phy_ppu_write,
.port_set_link = mv88e6xxx_port_set_link,
+ .port_sync_link = mv88e6185_port_sync_link,
.port_set_speed_duplex = mv88e6185_port_set_speed_duplex,
.port_set_frame_mode = mv88e6085_port_set_frame_mode,
.port_set_egress_floods = mv88e6185_port_set_egress_floods,
.stats_get_strings = mv88e6095_stats_get_strings,
.stats_get_stats = mv88e6095_stats_get_stats,
.mgmt_rsvd2cpu = mv88e6185_g2_mgmt_rsvd2cpu,
+ .serdes_power = mv88e6185_serdes_power,
+ .serdes_get_lane = mv88e6185_serdes_get_lane,
+ .serdes_pcs_get_state = mv88e6185_serdes_pcs_get_state,
.ppu_enable = mv88e6185_g1_ppu_enable,
.ppu_disable = mv88e6185_g1_ppu_disable,
.reset = mv88e6185_g1_reset,
.phy_read = mv88e6xxx_g2_smi_phy_read,
.phy_write = mv88e6xxx_g2_smi_phy_write,
.port_set_link = mv88e6xxx_port_set_link,
+ .port_sync_link = mv88e6185_port_sync_link,
.port_set_speed_duplex = mv88e6185_port_set_speed_duplex,
.port_tag_remap = mv88e6095_port_tag_remap,
.port_set_frame_mode = mv88e6351_port_set_frame_mode,
.set_egress_port = mv88e6095_g1_set_egress_port,
.watchdog_ops = &mv88e6097_watchdog_ops,
.mgmt_rsvd2cpu = mv88e6352_g2_mgmt_rsvd2cpu,
+ .serdes_power = mv88e6185_serdes_power,
+ .serdes_get_lane = mv88e6185_serdes_get_lane,
+ .serdes_pcs_get_state = mv88e6185_serdes_pcs_get_state,
+ .serdes_irq_mapping = mv88e6390_serdes_irq_mapping,
+ .serdes_irq_enable = mv88e6097_serdes_irq_enable,
+ .serdes_irq_status = mv88e6097_serdes_irq_status,
.pot_clear = mv88e6xxx_g2_pot_clear,
.reset = mv88e6352_g1_reset,
.rmu_disable = mv88e6085_g1_rmu_disable,
.phy_read = mv88e6xxx_g2_smi_phy_read,
.phy_write = mv88e6xxx_g2_smi_phy_write,
.port_set_link = mv88e6xxx_port_set_link,
+ .port_sync_link = mv88e6xxx_port_sync_link,
.port_set_speed_duplex = mv88e6185_port_set_speed_duplex,
.port_set_frame_mode = mv88e6085_port_set_frame_mode,
.port_set_egress_floods = mv88e6352_port_set_egress_floods,
.phy_read = mv88e6185_phy_ppu_read,
.phy_write = mv88e6185_phy_ppu_write,
.port_set_link = mv88e6xxx_port_set_link,
+ .port_sync_link = mv88e6xxx_port_sync_link,
.port_set_speed_duplex = mv88e6185_port_set_speed_duplex,
.port_tag_remap = mv88e6095_port_tag_remap,
.port_set_frame_mode = mv88e6351_port_set_frame_mode,
.phy_read = mv88e6xxx_g2_smi_phy_read,
.phy_write = mv88e6xxx_g2_smi_phy_write,
.port_set_link = mv88e6xxx_port_set_link,
+ .port_sync_link = mv88e6xxx_port_sync_link,
.port_set_rgmii_delay = mv88e6390_port_set_rgmii_delay,
.port_set_speed_duplex = mv88e6341_port_set_speed_duplex,
.port_max_speed_mode = mv88e6341_port_max_speed_mode,
.phy_read = mv88e6xxx_g2_smi_phy_read,
.phy_write = mv88e6xxx_g2_smi_phy_write,
.port_set_link = mv88e6xxx_port_set_link,
+ .port_sync_link = mv88e6xxx_port_sync_link,
.port_set_speed_duplex = mv88e6185_port_set_speed_duplex,
.port_tag_remap = mv88e6095_port_tag_remap,
.port_set_frame_mode = mv88e6351_port_set_frame_mode,
.phy_read = mv88e6165_phy_read,
.phy_write = mv88e6165_phy_write,
.port_set_link = mv88e6xxx_port_set_link,
+ .port_sync_link = mv88e6xxx_port_sync_link,
.port_set_speed_duplex = mv88e6185_port_set_speed_duplex,
.port_disable_learn_limit = mv88e6xxx_port_disable_learn_limit,
.port_disable_pri_override = mv88e6xxx_port_disable_pri_override,
.phy_read = mv88e6xxx_g2_smi_phy_read,
.phy_write = mv88e6xxx_g2_smi_phy_write,
.port_set_link = mv88e6xxx_port_set_link,
+ .port_sync_link = mv88e6xxx_port_sync_link,
.port_set_rgmii_delay = mv88e6352_port_set_rgmii_delay,
.port_set_speed_duplex = mv88e6185_port_set_speed_duplex,
.port_tag_remap = mv88e6095_port_tag_remap,
.phy_read = mv88e6xxx_g2_smi_phy_read,
.phy_write = mv88e6xxx_g2_smi_phy_write,
.port_set_link = mv88e6xxx_port_set_link,
+ .port_sync_link = mv88e6xxx_port_sync_link,
.port_set_rgmii_delay = mv88e6352_port_set_rgmii_delay,
.port_set_speed_duplex = mv88e6352_port_set_speed_duplex,
.port_tag_remap = mv88e6095_port_tag_remap,
.phy_read = mv88e6xxx_g2_smi_phy_read,
.phy_write = mv88e6xxx_g2_smi_phy_write,
.port_set_link = mv88e6xxx_port_set_link,
+ .port_sync_link = mv88e6xxx_port_sync_link,
.port_set_rgmii_delay = mv88e6352_port_set_rgmii_delay,
.port_set_speed_duplex = mv88e6185_port_set_speed_duplex,
.port_tag_remap = mv88e6095_port_tag_remap,
.phy_read = mv88e6xxx_g2_smi_phy_read,
.phy_write = mv88e6xxx_g2_smi_phy_write,
.port_set_link = mv88e6xxx_port_set_link,
+ .port_sync_link = mv88e6xxx_port_sync_link,
.port_set_rgmii_delay = mv88e6352_port_set_rgmii_delay,
.port_set_speed_duplex = mv88e6352_port_set_speed_duplex,
.port_tag_remap = mv88e6095_port_tag_remap,
.phy_read = mv88e6185_phy_ppu_read,
.phy_write = mv88e6185_phy_ppu_write,
.port_set_link = mv88e6xxx_port_set_link,
+ .port_sync_link = mv88e6185_port_sync_link,
.port_set_speed_duplex = mv88e6185_port_set_speed_duplex,
.port_set_frame_mode = mv88e6085_port_set_frame_mode,
.port_set_egress_floods = mv88e6185_port_set_egress_floods,
.set_egress_port = mv88e6095_g1_set_egress_port,
.watchdog_ops = &mv88e6097_watchdog_ops,
.mgmt_rsvd2cpu = mv88e6185_g2_mgmt_rsvd2cpu,
+ .serdes_power = mv88e6185_serdes_power,
+ .serdes_get_lane = mv88e6185_serdes_get_lane,
+ .serdes_pcs_get_state = mv88e6185_serdes_pcs_get_state,
.set_cascade_port = mv88e6185_g1_set_cascade_port,
.ppu_enable = mv88e6185_g1_ppu_enable,
.ppu_disable = mv88e6185_g1_ppu_disable,
.phy_read = mv88e6xxx_g2_smi_phy_read,
.phy_write = mv88e6xxx_g2_smi_phy_write,
.port_set_link = mv88e6xxx_port_set_link,
+ .port_sync_link = mv88e6xxx_port_sync_link,
.port_set_rgmii_delay = mv88e6390_port_set_rgmii_delay,
.port_set_speed_duplex = mv88e6390_port_set_speed_duplex,
.port_max_speed_mode = mv88e6390_port_max_speed_mode,
.phy_read = mv88e6xxx_g2_smi_phy_read,
.phy_write = mv88e6xxx_g2_smi_phy_write,
.port_set_link = mv88e6xxx_port_set_link,
+ .port_sync_link = mv88e6xxx_port_sync_link,
.port_set_rgmii_delay = mv88e6390_port_set_rgmii_delay,
.port_set_speed_duplex = mv88e6390x_port_set_speed_duplex,
.port_max_speed_mode = mv88e6390x_port_max_speed_mode,
.phy_read = mv88e6xxx_g2_smi_phy_read,
.phy_write = mv88e6xxx_g2_smi_phy_write,
.port_set_link = mv88e6xxx_port_set_link,
+ .port_sync_link = mv88e6xxx_port_sync_link,
.port_set_rgmii_delay = mv88e6390_port_set_rgmii_delay,
.port_set_speed_duplex = mv88e6390_port_set_speed_duplex,
.port_max_speed_mode = mv88e6390_port_max_speed_mode,
.phy_read = mv88e6xxx_g2_smi_phy_read,
.phy_write = mv88e6xxx_g2_smi_phy_write,
.port_set_link = mv88e6xxx_port_set_link,
+ .port_sync_link = mv88e6xxx_port_sync_link,
.port_set_rgmii_delay = mv88e6352_port_set_rgmii_delay,
.port_set_speed_duplex = mv88e6352_port_set_speed_duplex,
.port_tag_remap = mv88e6095_port_tag_remap,
.phy_read = mv88e6xxx_g2_smi_phy_read,
.phy_write = mv88e6xxx_g2_smi_phy_write,
.port_set_link = mv88e6xxx_port_set_link,
+ .port_sync_link = mv88e6xxx_port_sync_link,
.port_set_rgmii_delay = mv88e6352_port_set_rgmii_delay,
.port_set_speed_duplex = mv88e6250_port_set_speed_duplex,
.port_tag_remap = mv88e6095_port_tag_remap,
.phy_read = mv88e6xxx_g2_smi_phy_read,
.phy_write = mv88e6xxx_g2_smi_phy_write,
.port_set_link = mv88e6xxx_port_set_link,
+ .port_sync_link = mv88e6xxx_port_sync_link,
.port_set_rgmii_delay = mv88e6390_port_set_rgmii_delay,
.port_set_speed_duplex = mv88e6390_port_set_speed_duplex,
.port_max_speed_mode = mv88e6390_port_max_speed_mode,
.phy_read = mv88e6xxx_g2_smi_phy_read,
.phy_write = mv88e6xxx_g2_smi_phy_write,
.port_set_link = mv88e6xxx_port_set_link,
+ .port_sync_link = mv88e6xxx_port_sync_link,
.port_set_speed_duplex = mv88e6185_port_set_speed_duplex,
.port_tag_remap = mv88e6095_port_tag_remap,
.port_set_frame_mode = mv88e6351_port_set_frame_mode,
.phy_read = mv88e6xxx_g2_smi_phy_read,
.phy_write = mv88e6xxx_g2_smi_phy_write,
.port_set_link = mv88e6xxx_port_set_link,
+ .port_sync_link = mv88e6xxx_port_sync_link,
.port_set_speed_duplex = mv88e6185_port_set_speed_duplex,
.port_tag_remap = mv88e6095_port_tag_remap,
.port_set_frame_mode = mv88e6351_port_set_frame_mode,
.phy_read = mv88e6xxx_g2_smi_phy_read,
.phy_write = mv88e6xxx_g2_smi_phy_write,
.port_set_link = mv88e6xxx_port_set_link,
+ .port_sync_link = mv88e6xxx_port_sync_link,
.port_set_rgmii_delay = mv88e6390_port_set_rgmii_delay,
.port_set_speed_duplex = mv88e6341_port_set_speed_duplex,
.port_max_speed_mode = mv88e6341_port_max_speed_mode,
.phy_read = mv88e6xxx_g2_smi_phy_read,
.phy_write = mv88e6xxx_g2_smi_phy_write,
.port_set_link = mv88e6xxx_port_set_link,
+ .port_sync_link = mv88e6xxx_port_sync_link,
.port_set_rgmii_delay = mv88e6352_port_set_rgmii_delay,
.port_set_speed_duplex = mv88e6185_port_set_speed_duplex,
.port_tag_remap = mv88e6095_port_tag_remap,
.phy_read = mv88e6xxx_g2_smi_phy_read,
.phy_write = mv88e6xxx_g2_smi_phy_write,
.port_set_link = mv88e6xxx_port_set_link,
+ .port_sync_link = mv88e6xxx_port_sync_link,
.port_set_rgmii_delay = mv88e6352_port_set_rgmii_delay,
.port_set_speed_duplex = mv88e6185_port_set_speed_duplex,
.port_tag_remap = mv88e6095_port_tag_remap,
.phy_read = mv88e6xxx_g2_smi_phy_read,
.phy_write = mv88e6xxx_g2_smi_phy_write,
.port_set_link = mv88e6xxx_port_set_link,
+ .port_sync_link = mv88e6xxx_port_sync_link,
.port_set_rgmii_delay = mv88e6352_port_set_rgmii_delay,
.port_set_speed_duplex = mv88e6352_port_set_speed_duplex,
.port_tag_remap = mv88e6095_port_tag_remap,
.phy_read = mv88e6xxx_g2_smi_phy_read,
.phy_write = mv88e6xxx_g2_smi_phy_write,
.port_set_link = mv88e6xxx_port_set_link,
+ .port_sync_link = mv88e6xxx_port_sync_link,
.port_set_rgmii_delay = mv88e6390_port_set_rgmii_delay,
.port_set_speed_duplex = mv88e6390_port_set_speed_duplex,
.port_max_speed_mode = mv88e6390_port_max_speed_mode,
.phy_read = mv88e6xxx_g2_smi_phy_read,
.phy_write = mv88e6xxx_g2_smi_phy_write,
.port_set_link = mv88e6xxx_port_set_link,
+ .port_sync_link = mv88e6xxx_port_sync_link,
.port_set_rgmii_delay = mv88e6390_port_set_rgmii_delay,
.port_set_speed_duplex = mv88e6390x_port_set_speed_duplex,
.port_max_speed_mode = mv88e6390x_port_max_speed_mode,
*/
int (*port_set_link)(struct mv88e6xxx_chip *chip, int port, int link);
+ /* Synchronise the port link state with that of the SERDES
+ */
+ int (*port_sync_link)(struct mv88e6xxx_chip *chip, int port, unsigned int mode, bool isup);
+
#define PAUSE_ON 1
#define PAUSE_OFF 0
return mv88e6xxx_g1_wait_bit(chip, MV88E6XXX_G1_STS, bit, 1);
}
+void mv88e6xxx_g1_wait_eeprom_done(struct mv88e6xxx_chip *chip)
+{
+ const unsigned long timeout = jiffies + 1 * HZ;
+ u16 val;
+ int err;
+
+ /* Wait up to 1 second for the switch to finish reading the
+ * EEPROM.
+ */
+ while (time_before(jiffies, timeout)) {
+ err = mv88e6xxx_g1_read(chip, MV88E6XXX_G1_STS, &val);
+ if (err) {
+ dev_err(chip->dev, "Error reading status");
+ return;
+ }
+
+ /* If the switch is still resetting, it may not
+ * respond on the bus, and so MDIO read returns
+ * 0xffff. Differentiate between that, and waiting for
+ * the EEPROM to be done by bit 0 being set.
+ */
+ if (val != 0xffff &&
+ val & BIT(MV88E6XXX_G1_STS_IRQ_EEPROM_DONE))
+ return;
+
+ usleep_range(1000, 2000);
+ }
+
+ dev_err(chip->dev, "Timeout waiting for EEPROM done");
+}
+
/* Offset 0x01: Switch MAC Address Register Bytes 0 & 1
* Offset 0x02: Switch MAC Address Register Bytes 2 & 3
* Offset 0x03: Switch MAC Address Register Bytes 4 & 5
int mv88e6185_g1_reset(struct mv88e6xxx_chip *chip);
int mv88e6352_g1_reset(struct mv88e6xxx_chip *chip);
int mv88e6250_g1_reset(struct mv88e6xxx_chip *chip);
+void mv88e6xxx_g1_wait_eeprom_done(struct mv88e6xxx_chip *chip);
int mv88e6185_g1_ppu_enable(struct mv88e6xxx_chip *chip);
int mv88e6185_g1_ppu_disable(struct mv88e6xxx_chip *chip);
* Offset 0x08: VTU/STU Data Register 2
* Offset 0x09: VTU/STU Data Register 3
*/
-
-static int mv88e6185_g1_vtu_data_read(struct mv88e6xxx_chip *chip,
- struct mv88e6xxx_vtu_entry *entry)
+static int mv88e6185_g1_vtu_stu_data_read(struct mv88e6xxx_chip *chip,
+ u16 *regs)
{
- u16 regs[3];
int i;
/* Read all 3 VTU/STU Data registers */
return err;
}
- /* Extract MemberTag and PortState data */
+ return 0;
+}
+
+static int mv88e6185_g1_vtu_data_read(struct mv88e6xxx_chip *chip,
+ struct mv88e6xxx_vtu_entry *entry)
+{
+ u16 regs[3];
+ int err;
+ int i;
+
+ err = mv88e6185_g1_vtu_stu_data_read(chip, regs);
+ if (err)
+ return err;
+
+ /* Extract MemberTag data */
for (i = 0; i < mv88e6xxx_num_ports(chip); ++i) {
unsigned int member_offset = (i % 4) * 4;
- unsigned int state_offset = member_offset + 2;
entry->member[i] = (regs[i / 4] >> member_offset) & 0x3;
+ }
+
+ return 0;
+}
+
+static int mv88e6185_g1_stu_data_read(struct mv88e6xxx_chip *chip,
+ struct mv88e6xxx_vtu_entry *entry)
+{
+ u16 regs[3];
+ int err;
+ int i;
+
+ err = mv88e6185_g1_vtu_stu_data_read(chip, regs);
+ if (err)
+ return err;
+
+ /* Extract PortState data */
+ for (i = 0; i < mv88e6xxx_num_ports(chip); ++i) {
+ unsigned int state_offset = (i % 4) * 4 + 2;
+
entry->state[i] = (regs[i / 4] >> state_offset) & 0x3;
}
if (err)
return err;
+ err = mv88e6185_g1_stu_data_read(chip, entry);
+ if (err)
+ return err;
+
/* VTU DBNum[3:0] are located in VTU Operation 3:0
* VTU DBNum[7:4] are located in VTU Operation 11:8
*/
return err;
if (entry->valid) {
- /* Fetch (and mask) VLAN PortState data from the STU */
- err = mv88e6xxx_g1_vtu_stu_get(chip, entry);
+ err = mv88e6185_g1_vtu_data_read(chip, entry);
if (err)
return err;
- err = mv88e6185_g1_vtu_data_read(chip, entry);
+ err = mv88e6xxx_g1_vtu_fid_read(chip, entry);
if (err)
return err;
- err = mv88e6xxx_g1_vtu_fid_read(chip, entry);
+ /* Fetch VLAN PortState data from the STU */
+ err = mv88e6xxx_g1_vtu_stu_get(chip, entry);
+ if (err)
+ return err;
+
+ err = mv88e6185_g1_stu_data_read(chip, entry);
if (err)
return err;
}
return 0;
}
+int mv88e6xxx_port_sync_link(struct mv88e6xxx_chip *chip, int port, unsigned int mode, bool isup)
+{
+ const struct mv88e6xxx_ops *ops = chip->info->ops;
+ int err = 0;
+ int link;
+
+ if (isup)
+ link = LINK_FORCED_UP;
+ else
+ link = LINK_FORCED_DOWN;
+
+ if (ops->port_set_link)
+ err = ops->port_set_link(chip, port, link);
+
+ return err;
+}
+
+int mv88e6185_port_sync_link(struct mv88e6xxx_chip *chip, int port, unsigned int mode, bool isup)
+{
+ const struct mv88e6xxx_ops *ops = chip->info->ops;
+ int err = 0;
+ int link;
+
+ if (mode == MLO_AN_INBAND)
+ link = LINK_UNFORCED;
+ else if (isup)
+ link = LINK_FORCED_UP;
+ else
+ link = LINK_FORCED_DOWN;
+
+ if (ops->port_set_link)
+ err = ops->port_set_link(chip, port, link);
+
+ return err;
+}
+
static int mv88e6xxx_port_set_speed_duplex(struct mv88e6xxx_chip *chip,
int port, int speed, bool alt_bit,
bool force_bit, int duplex)
int mv88e6xxx_port_set_link(struct mv88e6xxx_chip *chip, int port, int link);
+int mv88e6xxx_port_sync_link(struct mv88e6xxx_chip *chip, int port, unsigned int mode, bool isup);
+int mv88e6185_port_sync_link(struct mv88e6xxx_chip *chip, int port, unsigned int mode, bool isup);
+
int mv88e6065_port_set_speed_duplex(struct mv88e6xxx_chip *chip, int port,
int speed, int duplex);
int mv88e6185_port_set_speed_duplex(struct mv88e6xxx_chip *chip, int port,
{
u16 *p = _p;
u16 reg;
+ int err;
int i;
if (!mv88e6352_port_has_serdes(chip, port))
return;
for (i = 0 ; i < 32; i++) {
- mv88e6352_serdes_read(chip, i, ®);
- p[i] = reg;
+ err = mv88e6352_serdes_read(chip, i, ®);
+ if (!err)
+ p[i] = reg;
}
}
return lane;
}
+int mv88e6185_serdes_power(struct mv88e6xxx_chip *chip, int port, u8 lane,
+ bool up)
+{
+ /* The serdes power can't be controlled on this switch chip but we need
+ * to supply this function to avoid returning -EOPNOTSUPP in
+ * mv88e6xxx_serdes_power_up/mv88e6xxx_serdes_power_down
+ */
+ return 0;
+}
+
+u8 mv88e6185_serdes_get_lane(struct mv88e6xxx_chip *chip, int port)
+{
+ /* There are no configurable serdes lanes on this switch chip but we
+ * need to return non-zero so that callers of
+ * mv88e6xxx_serdes_get_lane() know this is a serdes port.
+ */
+ switch (chip->ports[port].cmode) {
+ case MV88E6185_PORT_STS_CMODE_SERDES:
+ case MV88E6185_PORT_STS_CMODE_1000BASE_X:
+ return 0xff;
+ default:
+ return 0;
+ }
+}
+
+int mv88e6185_serdes_pcs_get_state(struct mv88e6xxx_chip *chip, int port,
+ u8 lane, struct phylink_link_state *state)
+{
+ int err;
+ u16 status;
+
+ err = mv88e6xxx_port_read(chip, port, MV88E6XXX_PORT_STS, &status);
+ if (err)
+ return err;
+
+ state->link = !!(status & MV88E6XXX_PORT_STS_LINK);
+
+ if (state->link) {
+ state->duplex = status & MV88E6XXX_PORT_STS_DUPLEX ? DUPLEX_FULL : DUPLEX_HALF;
+
+ switch (status & MV88E6XXX_PORT_STS_SPEED_MASK) {
+ case MV88E6XXX_PORT_STS_SPEED_1000:
+ state->speed = SPEED_1000;
+ break;
+ case MV88E6XXX_PORT_STS_SPEED_100:
+ state->speed = SPEED_100;
+ break;
+ case MV88E6XXX_PORT_STS_SPEED_10:
+ state->speed = SPEED_10;
+ break;
+ default:
+ dev_err(chip->dev, "invalid PHY speed\n");
+ return -EINVAL;
+ }
+ } else {
+ state->duplex = DUPLEX_UNKNOWN;
+ state->speed = SPEED_UNKNOWN;
+ }
+
+ return 0;
+}
+
+int mv88e6097_serdes_irq_enable(struct mv88e6xxx_chip *chip, int port, u8 lane,
+ bool enable)
+{
+ u8 cmode = chip->ports[port].cmode;
+
+ /* The serdes interrupts are enabled in the G2_INT_MASK register. We
+ * need to return 0 to avoid returning -EOPNOTSUPP in
+ * mv88e6xxx_serdes_irq_enable/mv88e6xxx_serdes_irq_disable
+ */
+ switch (cmode) {
+ case MV88E6185_PORT_STS_CMODE_SERDES:
+ case MV88E6185_PORT_STS_CMODE_1000BASE_X:
+ return 0;
+ }
+
+ return -EOPNOTSUPP;
+}
+
+static void mv88e6097_serdes_irq_link(struct mv88e6xxx_chip *chip, int port)
+{
+ u16 status;
+ int err;
+
+ err = mv88e6xxx_port_read(chip, port, MV88E6XXX_PORT_STS, &status);
+ if (err) {
+ dev_err(chip->dev, "can't read port status: %d\n", err);
+ return;
+ }
+
+ dsa_port_phylink_mac_change(chip->ds, port, !!(status & MV88E6XXX_PORT_STS_LINK));
+}
+
+irqreturn_t mv88e6097_serdes_irq_status(struct mv88e6xxx_chip *chip, int port,
+ u8 lane)
+{
+ u8 cmode = chip->ports[port].cmode;
+
+ switch (cmode) {
+ case MV88E6185_PORT_STS_CMODE_SERDES:
+ case MV88E6185_PORT_STS_CMODE_1000BASE_X:
+ mv88e6097_serdes_irq_link(chip, port);
+ return IRQ_HANDLED;
+ }
+
+ return IRQ_NONE;
+}
+
u8 mv88e6390_serdes_get_lane(struct mv88e6xxx_chip *chip, int port)
{
u8 cmode = chip->ports[port].cmode;
u16 *p = _p;
int lane;
u16 reg;
+ int err;
int i;
lane = mv88e6xxx_serdes_get_lane(chip, port);
return;
for (i = 0 ; i < ARRAY_SIZE(mv88e6390_serdes_regs); i++) {
- mv88e6390_serdes_read(chip, lane, MDIO_MMD_PHYXS,
- mv88e6390_serdes_regs[i], ®);
- p[i] = reg;
+ err = mv88e6390_serdes_read(chip, lane, MDIO_MMD_PHYXS,
+ mv88e6390_serdes_regs[i], ®);
+ if (!err)
+ p[i] = reg;
}
}
#define MV88E6390_PG_CONTROL 0xf010
#define MV88E6390_PG_CONTROL_ENABLE_PC BIT(0)
+u8 mv88e6185_serdes_get_lane(struct mv88e6xxx_chip *chip, int port);
u8 mv88e6341_serdes_get_lane(struct mv88e6xxx_chip *chip, int port);
u8 mv88e6352_serdes_get_lane(struct mv88e6xxx_chip *chip, int port);
u8 mv88e6390_serdes_get_lane(struct mv88e6xxx_chip *chip, int port);
u8 lane, unsigned int mode,
phy_interface_t interface,
const unsigned long *advertise);
+int mv88e6185_serdes_pcs_get_state(struct mv88e6xxx_chip *chip, int port,
+ u8 lane, struct phylink_link_state *state);
int mv88e6352_serdes_pcs_get_state(struct mv88e6xxx_chip *chip, int port,
u8 lane, struct phylink_link_state *state);
int mv88e6390_serdes_pcs_get_state(struct mv88e6xxx_chip *chip, int port,
int port);
unsigned int mv88e6390_serdes_irq_mapping(struct mv88e6xxx_chip *chip,
int port);
+int mv88e6185_serdes_power(struct mv88e6xxx_chip *chip, int port, u8 lane,
+ bool up);
int mv88e6352_serdes_power(struct mv88e6xxx_chip *chip, int port, u8 lane,
bool on);
int mv88e6390_serdes_power(struct mv88e6xxx_chip *chip, int port, u8 lane,
bool on);
+int mv88e6097_serdes_irq_enable(struct mv88e6xxx_chip *chip, int port, u8 lane,
+ bool enable);
int mv88e6352_serdes_irq_enable(struct mv88e6xxx_chip *chip, int port, u8 lane,
bool enable);
int mv88e6390_serdes_irq_enable(struct mv88e6xxx_chip *chip, int port, u8 lane,
bool enable);
+irqreturn_t mv88e6097_serdes_irq_status(struct mv88e6xxx_chip *chip, int port,
+ u8 lane);
irqreturn_t mv88e6352_serdes_irq_status(struct mv88e6xxx_chip *chip, int port,
u8 lane);
irqreturn_t mv88e6390_serdes_irq_status(struct mv88e6xxx_chip *chip, int port,
{
struct ena_com_rx_buf_info *ena_buf = &ena_rx_ctx->ena_bufs[0];
struct ena_eth_io_rx_cdesc_base *cdesc = NULL;
+ u16 q_depth = io_cq->q_depth;
u16 cdesc_idx = 0;
u16 nb_hw_desc;
u16 i = 0;
do {
ena_buf[i].len = cdesc->length;
ena_buf[i].req_id = cdesc->req_id;
+ if (unlikely(ena_buf[i].req_id >= q_depth))
+ return -EIO;
if (++i >= nb_hw_desc)
break;
* Copyright 2015-2020 Amazon.com, Inc. or its affiliates. All rights reserved.
*/
+#include <linux/ethtool.h>
#include <linux/pci.h>
#include "ena_netdev.h"
adapter->num_io_queues);
}
-static int validate_rx_req_id(struct ena_ring *rx_ring, u16 req_id)
-{
- if (likely(req_id < rx_ring->ring_size))
- return 0;
-
- netif_err(rx_ring->adapter, rx_err, rx_ring->netdev,
- "Invalid rx req_id: %hu\n", req_id);
-
- u64_stats_update_begin(&rx_ring->syncp);
- rx_ring->rx_stats.bad_req_id++;
- u64_stats_update_end(&rx_ring->syncp);
-
- /* Trigger device reset */
- rx_ring->adapter->reset_reason = ENA_REGS_RESET_INV_RX_REQ_ID;
- set_bit(ENA_FLAG_TRIGGER_RESET, &rx_ring->adapter->flags);
- return -EFAULT;
-}
-
/* ena_setup_rx_resources - allocate I/O Rx resources (Descriptors)
* @adapter: network interface device structure
* @qid: queue index
static int ena_alloc_rx_page(struct ena_ring *rx_ring,
struct ena_rx_buffer *rx_info, gfp_t gfp)
{
+ int headroom = rx_ring->rx_headroom;
struct ena_com_buf *ena_buf;
struct page *page;
dma_addr_t dma;
+ /* restore page offset value in case it has been changed by device */
+ rx_info->page_offset = headroom;
+
/* if previous allocated page is not used */
if (unlikely(rx_info->page))
return 0;
"Allocate page %p, rx_info %p\n", page, rx_info);
rx_info->page = page;
- rx_info->page_offset = 0;
ena_buf = &rx_info->ena_buf;
- ena_buf->paddr = dma + rx_ring->rx_headroom;
- ena_buf->len = ENA_PAGE_SIZE - rx_ring->rx_headroom;
+ ena_buf->paddr = dma + headroom;
+ ena_buf->len = ENA_PAGE_SIZE - headroom;
return 0;
}
struct ena_rx_buffer *rx_info;
u16 len, req_id, buf = 0;
void *va;
- int rc;
len = ena_bufs[buf].len;
req_id = ena_bufs[buf].req_id;
- rc = validate_rx_req_id(rx_ring, req_id);
- if (unlikely(rc < 0))
- return NULL;
-
rx_info = &rx_ring->rx_buffer_info[req_id];
if (unlikely(!rx_info->page)) {
/* save virt address of first buffer */
va = page_address(rx_info->page) + rx_info->page_offset;
- prefetch(va + NET_IP_ALIGN);
+
+ prefetch(va);
if (len <= rx_ring->rx_copybreak) {
skb = ena_alloc_skb(rx_ring, false);
skb_add_rx_frag(skb, skb_shinfo(skb)->nr_frags, rx_info->page,
rx_info->page_offset, len, ENA_PAGE_SIZE);
- /* The offset is non zero only for the first buffer */
- rx_info->page_offset = 0;
netif_dbg(rx_ring->adapter, rx_status, rx_ring->netdev,
"RX skb updated. len %d. data_len %d\n",
len = ena_bufs[buf].len;
req_id = ena_bufs[buf].req_id;
- rc = validate_rx_req_id(rx_ring, req_id);
- if (unlikely(rc < 0))
- return NULL;
-
rx_info = &rx_ring->rx_buffer_info[req_id];
} while (1);
int ret;
rx_info = &rx_ring->rx_buffer_info[rx_ring->ena_bufs[0].req_id];
- xdp->data = page_address(rx_info->page) +
- rx_info->page_offset + rx_ring->rx_headroom;
+ xdp->data = page_address(rx_info->page) + rx_info->page_offset;
xdp_set_data_meta_invalid(xdp);
xdp->data_hard_start = page_address(rx_info->page);
xdp->data_end = xdp->data + rx_ring->ena_bufs[0].len;
if (unlikely(ena_rx_ctx.descs == 0))
break;
+ /* First descriptor might have an offset set by the device */
rx_info = &rx_ring->rx_buffer_info[rx_ring->ena_bufs[0].req_id];
- rx_info->page_offset = ena_rx_ctx.pkt_offset;
+ rx_info->page_offset += ena_rx_ctx.pkt_offset;
netif_dbg(rx_ring->adapter, rx_status, rx_ring->netdev,
"rx_poll: q %d got packet from ena. descs #: %d l3 proto %d l4 proto %d hash: %x\n",
error:
adapter = netdev_priv(rx_ring->netdev);
- u64_stats_update_begin(&rx_ring->syncp);
- rx_ring->rx_stats.bad_desc_num++;
- u64_stats_update_end(&rx_ring->syncp);
+ if (rc == -ENOSPC) {
+ u64_stats_update_begin(&rx_ring->syncp);
+ rx_ring->rx_stats.bad_desc_num++;
+ u64_stats_update_end(&rx_ring->syncp);
+ adapter->reset_reason = ENA_REGS_RESET_TOO_MANY_RX_DESCS;
+ } else {
+ u64_stats_update_begin(&rx_ring->syncp);
+ rx_ring->rx_stats.bad_req_id++;
+ u64_stats_update_end(&rx_ring->syncp);
+ adapter->reset_reason = ENA_REGS_RESET_INV_RX_REQ_ID;
+ }
- /* Too many desc from the device. Trigger reset */
- adapter->reset_reason = ENA_REGS_RESET_TOO_MANY_RX_DESCS;
set_bit(ENA_FLAG_TRIGGER_RESET, &adapter->flags);
return 0;
goto err_mmio_read_less;
}
- rc = pci_set_dma_mask(pdev, DMA_BIT_MASK(dma_width));
+ rc = dma_set_mask_and_coherent(dev, DMA_BIT_MASK(dma_width));
if (rc) {
- dev_err(dev, "pci_set_dma_mask failed 0x%x\n", rc);
- goto err_mmio_read_less;
- }
-
- rc = pci_set_consistent_dma_mask(pdev, DMA_BIT_MASK(dma_width));
- if (rc) {
- dev_err(dev, "err_pci_set_consistent_dma_mask failed 0x%x\n",
- rc);
+ dev_err(dev, "dma_set_mask_and_coherent failed %d\n", rc);
goto err_mmio_read_less;
}
return rc;
}
+ rc = dma_set_mask_and_coherent(&pdev->dev, DMA_BIT_MASK(ENA_MAX_PHYS_ADDR_SIZE_BITS));
+ if (rc) {
+ dev_err(&pdev->dev, "dma_set_mask_and_coherent failed %d\n", rc);
+ goto err_disable_device;
+ }
+
pci_set_master(pdev);
ena_dev = vzalloc(sizeof(*ena_dev));
#ifndef AQ_NIC_H
#define AQ_NIC_H
+#include <linux/ethtool.h>
+
#include "aq_common.h"
#include "aq_rss.h"
#include "aq_hw.h"
buff->rxdata.pg_off,
buff->len, DMA_FROM_DEVICE);
- /* for single fragment packets use build_skb() */
- if (buff->is_eop &&
- buff->len <= AQ_CFG_RX_FRAME_MAX - AQ_SKB_ALIGN) {
- skb = build_skb(aq_buf_vaddr(&buff->rxdata),
+ skb = napi_alloc_skb(napi, AQ_CFG_RX_HDR_SIZE);
+ if (unlikely(!skb)) {
+ u64_stats_update_begin(&self->stats.rx.syncp);
+ self->stats.rx.skb_alloc_fails++;
+ u64_stats_update_end(&self->stats.rx.syncp);
+ err = -ENOMEM;
+ goto err_exit;
+ }
+ if (is_ptp_ring)
+ buff->len -=
+ aq_ptp_extract_ts(self->aq_nic, skb,
+ aq_buf_vaddr(&buff->rxdata),
+ buff->len);
+
+ hdr_len = buff->len;
+ if (hdr_len > AQ_CFG_RX_HDR_SIZE)
+ hdr_len = eth_get_headlen(skb->dev,
+ aq_buf_vaddr(&buff->rxdata),
+ AQ_CFG_RX_HDR_SIZE);
+
+ memcpy(__skb_put(skb, hdr_len), aq_buf_vaddr(&buff->rxdata),
+ ALIGN(hdr_len, sizeof(long)));
+
+ if (buff->len - hdr_len > 0) {
+ skb_add_rx_frag(skb, 0, buff->rxdata.page,
+ buff->rxdata.pg_off + hdr_len,
+ buff->len - hdr_len,
AQ_CFG_RX_FRAME_MAX);
- if (unlikely(!skb)) {
- u64_stats_update_begin(&self->stats.rx.syncp);
- self->stats.rx.skb_alloc_fails++;
- u64_stats_update_end(&self->stats.rx.syncp);
- err = -ENOMEM;
- goto err_exit;
- }
- if (is_ptp_ring)
- buff->len -=
- aq_ptp_extract_ts(self->aq_nic, skb,
- aq_buf_vaddr(&buff->rxdata),
- buff->len);
- skb_put(skb, buff->len);
page_ref_inc(buff->rxdata.page);
- } else {
- skb = napi_alloc_skb(napi, AQ_CFG_RX_HDR_SIZE);
- if (unlikely(!skb)) {
- u64_stats_update_begin(&self->stats.rx.syncp);
- self->stats.rx.skb_alloc_fails++;
- u64_stats_update_end(&self->stats.rx.syncp);
- err = -ENOMEM;
- goto err_exit;
- }
- if (is_ptp_ring)
- buff->len -=
- aq_ptp_extract_ts(self->aq_nic, skb,
- aq_buf_vaddr(&buff->rxdata),
- buff->len);
-
- hdr_len = buff->len;
- if (hdr_len > AQ_CFG_RX_HDR_SIZE)
- hdr_len = eth_get_headlen(skb->dev,
- aq_buf_vaddr(&buff->rxdata),
- AQ_CFG_RX_HDR_SIZE);
-
- memcpy(__skb_put(skb, hdr_len), aq_buf_vaddr(&buff->rxdata),
- ALIGN(hdr_len, sizeof(long)));
-
- if (buff->len - hdr_len > 0) {
- skb_add_rx_frag(skb, 0, buff->rxdata.page,
- buff->rxdata.pg_off + hdr_len,
- buff->len - hdr_len,
- AQ_CFG_RX_FRAME_MAX);
- page_ref_inc(buff->rxdata.page);
- }
+ }
- if (!buff->is_eop) {
- buff_ = buff;
- i = 1U;
- do {
- next_ = buff_->next,
- buff_ = &self->buff_ring[next_];
+ if (!buff->is_eop) {
+ buff_ = buff;
+ i = 1U;
+ do {
+ next_ = buff_->next;
+ buff_ = &self->buff_ring[next_];
- dma_sync_single_range_for_cpu(
- aq_nic_get_dev(self->aq_nic),
- buff_->rxdata.daddr,
- buff_->rxdata.pg_off,
- buff_->len,
- DMA_FROM_DEVICE);
- skb_add_rx_frag(skb, i++,
- buff_->rxdata.page,
- buff_->rxdata.pg_off,
- buff_->len,
- AQ_CFG_RX_FRAME_MAX);
- page_ref_inc(buff_->rxdata.page);
- buff_->is_cleaned = 1;
-
- buff->is_ip_cso &= buff_->is_ip_cso;
- buff->is_udp_cso &= buff_->is_udp_cso;
- buff->is_tcp_cso &= buff_->is_tcp_cso;
- buff->is_cso_err |= buff_->is_cso_err;
+ dma_sync_single_range_for_cpu(aq_nic_get_dev(self->aq_nic),
+ buff_->rxdata.daddr,
+ buff_->rxdata.pg_off,
+ buff_->len,
+ DMA_FROM_DEVICE);
+ skb_add_rx_frag(skb, i++,
+ buff_->rxdata.page,
+ buff_->rxdata.pg_off,
+ buff_->len,
+ AQ_CFG_RX_FRAME_MAX);
+ page_ref_inc(buff_->rxdata.page);
+ buff_->is_cleaned = 1;
- } while (!buff_->is_eop);
- }
+ buff->is_ip_cso &= buff_->is_ip_cso;
+ buff->is_udp_cso &= buff_->is_udp_cso;
+ buff->is_tcp_cso &= buff_->is_tcp_cso;
+ buff->is_cso_err |= buff_->is_cso_err;
+
+ } while (!buff_->is_eop);
}
if (buff->is_vlan)
* various kernel subsystems to support the mechanics required by a
* fixed-high-32-bit system.
*/
- if ((dma_set_mask(&pdev->dev, DMA_BIT_MASK(32)) != 0) ||
- (dma_set_coherent_mask(&pdev->dev, DMA_BIT_MASK(32)) != 0)) {
+ err = dma_set_mask_and_coherent(&pdev->dev, DMA_BIT_MASK(32));
+ if (err) {
dev_err(&pdev->dev, "No usable DMA configuration,aborting\n");
goto err_dma;
}
* various kernel subsystems to support the mechanics required by a
* fixed-high-32-bit system.
*/
- if ((dma_set_mask(&pdev->dev, DMA_BIT_MASK(32)) != 0) ||
- (dma_set_coherent_mask(&pdev->dev, DMA_BIT_MASK(32)) != 0)) {
+ err = dma_set_mask_and_coherent(&pdev->dev, DMA_BIT_MASK(32));
+ if (err) {
dev_err(&pdev->dev, "No usable DMA configuration,aborting\n");
goto err_dma;
}
config CNIC
tristate "QLogic CNIC support"
depends on PCI && (IPV6 || IPV6=n)
+ depends on MMU
select BNX2
select UIO
help
goto err_out_free_dev;
}
- if (dma_set_mask_and_coherent(sdev->dma_dev, DMA_BIT_MASK(30))) {
+ err = dma_set_mask_and_coherent(sdev->dma_dev, DMA_BIT_MASK(30));
+ if (err) {
dev_err(sdev->dev,
"Required 30BIT DMA mask unsupported by the system\n");
goto err_out_powerdown;
bnxt_free_ntp_fltrs(bp, irq_re_init);
if (irq_re_init) {
bnxt_free_ring_stats(bp);
- if (!(bp->fw_cap & BNXT_FW_CAP_PORT_STATS_NO_RESET))
+ if (!(bp->fw_cap & BNXT_FW_CAP_PORT_STATS_NO_RESET) ||
+ test_bit(BNXT_STATE_IN_FW_RESET, &bp->state))
bnxt_free_port_stats(bp);
bnxt_free_ring_grps(bp);
bnxt_free_vnics(bp);
{
u64 sw_tmp;
+ hw &= mask;
sw_tmp = (*sw & ~mask) | hw;
if (hw < (*sw & mask))
sw_tmp += mask + 1;
if (dma_set_mask_and_coherent(&pdev->dev, DMA_BIT_MASK(64)) != 0 &&
dma_set_mask_and_coherent(&pdev->dev, DMA_BIT_MASK(32)) != 0) {
dev_err(&pdev->dev, "System does not support DMA, aborting\n");
- goto init_err_disable;
+ rc = -EIO;
+ goto init_err_release;
}
pci_set_master(pdev);
create_singlethread_workqueue("bnxt_pf_wq");
if (!bnxt_pf_wq) {
dev_err(&pdev->dev, "Unable to create workqueue.\n");
+ rc = -ENOMEM;
goto init_err_pci_clean;
}
}
#define DRV_VER_MIN 10
#define DRV_VER_UPD 1
+#include <linux/ethtool.h>
#include <linux/interrupt.h>
#include <linux/rhashtable.h>
#include <linux/crash_dump.h>
return -EPERM;
}
- devlink_flash_update_begin_notify(dl);
devlink_flash_update_status_notify(dl, "Preparing to flash", NULL, 0, 0);
- rc = bnxt_flash_package_from_file(bp->dev, params->file_name, 0);
+ rc = bnxt_flash_package_from_fw_obj(bp->dev, params->fw, 0);
if (!rc)
devlink_flash_update_status_notify(dl, "Flashing done", NULL, 0, 0);
else
devlink_flash_update_status_notify(dl, "Flashing failed", NULL, 0, 0);
- devlink_flash_update_end_notify(dl);
return rc;
}
struct hwrm_nvm_get_dev_info_input req = {0};
int rc;
+ if (BNXT_VF(bp))
+ return -EOPNOTSUPP;
+
bnxt_hwrm_cmd_hdr_init(bp, &req, HWRM_NVM_GET_DEV_INFO, -1, -1);
mutex_lock(&bp->hwrm_cmd_lock);
rc = _hwrm_send_message(bp, &req, sizeof(req), HWRM_CMD_TIMEOUT);
return rc;
}
-int bnxt_flash_package_from_file(struct net_device *dev, const char *filename,
- u32 install_type)
+int bnxt_flash_package_from_fw_obj(struct net_device *dev, const struct firmware *fw,
+ u32 install_type)
{
struct bnxt *bp = netdev_priv(dev);
struct hwrm_nvm_install_update_output *resp = bp->hwrm_cmd_resp_addr;
struct hwrm_nvm_install_update_input install = {0};
- const struct firmware *fw;
u32 item_len;
int rc = 0;
u16 index;
return rc;
}
- rc = request_firmware(&fw, filename, &dev->dev);
- if (rc != 0) {
- netdev_err(dev, "PKG error %d requesting file: %s\n",
- rc, filename);
- return rc;
- }
-
if (fw->size > item_len) {
netdev_err(dev, "PKG insufficient update area in nvram: %lu\n",
(unsigned long)fw->size);
dma_handle);
}
}
- release_firmware(fw);
if (rc)
goto err_exit;
return rc;
}
+static int bnxt_flash_package_from_file(struct net_device *dev, const char *filename,
+ u32 install_type)
+{
+ const struct firmware *fw;
+ int rc;
+
+ rc = request_firmware(&fw, filename, &dev->dev);
+ if (rc != 0) {
+ netdev_err(dev, "PKG error %d requesting file: %s\n",
+ rc, filename);
+ return rc;
+ }
+
+ rc = bnxt_flash_package_from_fw_obj(dev, fw, install_type);
+
+ release_firmware(fw);
+
+ return rc;
+}
+
static int bnxt_flash_device(struct net_device *dev,
struct ethtool_flash *flash)
{
/* Read A2 portion of the EEPROM */
if (length) {
start -= ETH_MODULE_SFF_8436_LEN;
- rc = bnxt_read_sfp_module_eeprom_info(bp, I2C_DEV_ADDR_A2, 1,
+ rc = bnxt_read_sfp_module_eeprom_info(bp, I2C_DEV_ADDR_A2, 0,
start, length, data);
}
return rc;
u16 bnxt_get_fw_auto_link_speeds(u32);
int bnxt_hwrm_nvm_get_dev_info(struct bnxt *bp,
struct hwrm_nvm_get_dev_info_output *nvm_dev_info);
-int bnxt_flash_package_from_file(struct net_device *dev, const char *filename,
- u32 install_type);
+int bnxt_flash_package_from_fw_obj(struct net_device *dev, const struct firmware *fw,
+ u32 install_type);
void bnxt_ethtool_init(struct bnxt *bp);
void bnxt_ethtool_free(struct bnxt *bp);
* the Free Software Foundation.
*/
+#include <linux/ethtool.h>
#include <linux/module.h>
#include <linux/pci.h>
#include <linux/netdevice.h>
* of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE, TITLE, or
* NONINFRINGEMENT. See the GNU General Public License for more details.
***********************************************************************/
+#include <linux/ethtool.h>
#include <linux/netdevice.h>
#include <linux/net_tstamp.h>
#include <linux/pci.h>
/* ETHTOOL Support for VNIC_VF Device*/
+#include <linux/ethtool.h>
#include <linux/pci.h>
#include <linux/net_tstamp.h>
config CHELSIO_T4
tristate "Chelsio Communications T4/T5/T6 Ethernet support"
- depends on PCI && (IPV6 || IPV6=n)
+ depends on PCI && (IPV6 || IPV6=n) && (TLS || TLS=n)
select FW_LOADER
select MDIO
select ZLIB_DEFLATE
GFP_KERNEL | __GFP_COMP);
if (!avail) {
CH_ALERT(adapter, "free list queue 0 initialization failed\n");
+ ret = -ENOMEM;
goto err;
}
if (avail < q->fl[0].size)
#include <linux/bitops.h>
#include <linux/cache.h>
+#include <linux/ethtool.h>
#include <linux/interrupt.h>
#include <linux/list.h>
#include <linux/netdevice.h>
FW_FILTER_WR_OVLAN_VLD_V(f->fs.val.ovlan_vld) |
FW_FILTER_WR_IVLAN_VLDM_V(f->fs.mask.ivlan_vld) |
FW_FILTER_WR_OVLAN_VLDM_V(f->fs.mask.ovlan_vld));
- fwr->smac_sel = f->smt->idx;
+ if (f->fs.newsmac)
+ fwr->smac_sel = f->smt->idx;
fwr->rx_chan_rx_rpl_iq =
htons(FW_FILTER_WR_RX_CHAN_V(0) |
FW_FILTER_WR_RX_RPL_IQ_V(adapter->sge.fw_evtq.abs_id));
* SOFTWARE.
*/
+#include <linux/ethtool.h>
#include <linux/pci.h>
#include "t4vf_common.h"
/* need to wait for hw response, can't free tx_info yet. */
if (tx_info->open_state == CH_KTLS_OPEN_PENDING)
tx_info->pending_close = true;
- /* free the lock after the cleanup */
+ else
+ spin_unlock_bh(&tx_info->lock);
+ /* if in pending close, free the lock after the cleanup */
goto put_module;
}
spin_unlock_bh(&tx_info->lock);
sk_setup_caps(newsk, dst);
ctx = tls_get_ctx(lsk);
newsk->sk_destruct = ctx->sk_destruct;
+ newsk->sk_prot_creator = lsk->sk_prot_creator;
csk->sk = newsk;
csk->passive_reap_next = oreq;
csk->tx_chan = cxgb4_port_chan(ndev);
csk->wr_unacked += DIV_ROUND_UP(len, 16);
enqueue_wr(csk, skb);
cxgb4_ofld_send(csk->egress_dev, skb);
+ skb = NULL;
chtls_set_scmd(csk);
/* Clear quiesce for Rx key */
err_phy_connect:
ftgmac100_phy_disconnect(netdev);
err_ncsi_dev:
+ if (priv->ndev)
+ ncsi_unregister_dev(priv->ndev);
ftgmac100_destroy_mdio(netdev);
err_setup_mdio:
iounmap(priv->base);
netdev = platform_get_drvdata(pdev);
priv = netdev_priv(netdev);
+ if (priv->ndev)
+ ncsi_unregister_dev(priv->ndev);
unregister_netdev(netdev);
clk_disable_unprepare(priv->rclk);
#include <linux/dma-mapping.h>
#include <linux/sort.h>
#include <linux/phy_fixed.h>
+#include <linux/bpf.h>
+#include <linux/bpf_trace.h>
#include <soc/fsl/bman.h>
#include <soc/fsl/qman.h>
#include "fman.h"
#define DPAA_HWA_SIZE (DPAA_PARSE_RESULTS_SIZE + DPAA_TIME_STAMP_SIZE \
+ DPAA_HASH_RESULTS_SIZE)
#define DPAA_RX_PRIV_DATA_DEFAULT_SIZE (DPAA_TX_PRIV_DATA_SIZE + \
- dpaa_rx_extra_headroom)
+ XDP_PACKET_HEADROOM - DPAA_HWA_SIZE)
#ifdef CONFIG_DPAA_ERRATUM_A050385
#define DPAA_RX_PRIV_DATA_A050385_SIZE (DPAA_A050385_ALIGN - DPAA_HWA_SIZE)
#define DPAA_RX_PRIV_DATA_SIZE (fman_has_errata_a050385() ? \
dpaa_fq->fqid = qman_fq_fqid(fq);
+ if (dpaa_fq->fq_type == FQ_TYPE_RX_DEFAULT ||
+ dpaa_fq->fq_type == FQ_TYPE_RX_PCD) {
+ err = xdp_rxq_info_reg(&dpaa_fq->xdp_rxq, dpaa_fq->net_dev,
+ dpaa_fq->fqid);
+ if (err) {
+ dev_err(dev, "xdp_rxq_info_reg() = %d\n", err);
+ return err;
+ }
+
+ err = xdp_rxq_info_reg_mem_model(&dpaa_fq->xdp_rxq,
+ MEM_TYPE_PAGE_ORDER0, NULL);
+ if (err) {
+ dev_err(dev, "xdp_rxq_info_reg_mem_model() = %d\n",
+ err);
+ xdp_rxq_info_unreg(&dpaa_fq->xdp_rxq);
+ return err;
+ }
+ }
+
return 0;
}
}
}
+ if ((dpaa_fq->fq_type == FQ_TYPE_RX_DEFAULT ||
+ dpaa_fq->fq_type == FQ_TYPE_RX_PCD) &&
+ xdp_rxq_info_is_reg(&dpaa_fq->xdp_rxq))
+ xdp_rxq_info_unreg(&dpaa_fq->xdp_rxq);
+
qman_destroy_fq(fq);
list_del(&dpaa_fq->list);
*
* Return the skb backpointer, since for S/G frames the buffer containing it
* gets freed here.
+ *
+ * No skb backpointer is set when transmitting XDP frames. Cleanup the buffer
+ * and return NULL in this case.
*/
static struct sk_buff *dpaa_cleanup_tx_fd(const struct dpaa_priv *priv,
const struct qm_fd *fd, bool ts)
dma_addr_t addr = qm_fd_addr(fd);
void *vaddr = phys_to_virt(addr);
const struct qm_sg_entry *sgt;
+ struct dpaa_eth_swbp *swbp;
struct sk_buff *skb;
u64 ns;
int i;
}
} else {
dma_unmap_single(priv->tx_dma_dev, addr,
- priv->tx_headroom + qm_fd_get_length(fd),
+ qm_fd_get_offset(fd) + qm_fd_get_length(fd),
dma_dir);
}
- skb = *(struct sk_buff **)vaddr;
+ swbp = (struct dpaa_eth_swbp *)vaddr;
+ skb = swbp->skb;
+
+ /* No skb backpointer is set when running XDP. An xdp_frame
+ * backpointer is saved instead.
+ */
+ if (!skb) {
+ xdp_return_frame(swbp->xdpf);
+ return NULL;
+ }
/* DMA unmapping is required before accessing the HW provided info */
if (ts && priv->tx_tstamp &&
SKB_DATA_ALIGN(sizeof(struct skb_shared_info)));
if (WARN_ONCE(!skb, "Build skb failure on Rx\n"))
goto free_buffer;
- WARN_ON(fd_off != priv->rx_headroom);
skb_reserve(skb, fd_off);
skb_put(skb, qm_fd_get_length(fd));
{
struct net_device *net_dev = priv->net_dev;
enum dma_data_direction dma_dir;
+ struct dpaa_eth_swbp *swbp;
unsigned char *buff_start;
- struct sk_buff **skbh;
dma_addr_t addr;
int err;
buff_start = skb->data - priv->tx_headroom;
dma_dir = DMA_TO_DEVICE;
- skbh = (struct sk_buff **)buff_start;
- *skbh = skb;
+ swbp = (struct dpaa_eth_swbp *)buff_start;
+ swbp->skb = skb;
/* Enable L3/L4 hardware checksum computation.
*
const enum dma_data_direction dma_dir = DMA_TO_DEVICE;
const int nr_frags = skb_shinfo(skb)->nr_frags;
struct net_device *net_dev = priv->net_dev;
+ struct dpaa_eth_swbp *swbp;
struct qm_sg_entry *sgt;
- struct sk_buff **skbh;
void *buff_start;
skb_frag_t *frag;
dma_addr_t addr;
qm_fd_set_sg(fd, priv->tx_headroom, skb->len);
/* DMA map the SGT page */
- skbh = (struct sk_buff **)buff_start;
- *skbh = skb;
+ swbp = (struct dpaa_eth_swbp *)buff_start;
+ swbp->skb = skb;
addr = dma_map_page(priv->tx_dma_dev, p, 0,
priv->tx_headroom + DPAA_SGT_SIZE, dma_dir);
}
#ifdef CONFIG_DPAA_ERRATUM_A050385
-static int dpaa_a050385_wa(struct net_device *net_dev, struct sk_buff **s)
+static int dpaa_a050385_wa_skb(struct net_device *net_dev, struct sk_buff **s)
{
struct dpaa_priv *priv = netdev_priv(net_dev);
struct sk_buff *new_skb, *skb = *s;
skb_copy_header(new_skb, skb);
new_skb->dev = skb->dev;
+ /* Copy relevant timestamp info from the old skb to the new */
+ if (priv->tx_tstamp) {
+ skb_shinfo(new_skb)->tx_flags = skb_shinfo(skb)->tx_flags;
+ skb_shinfo(new_skb)->hwtstamps = skb_shinfo(skb)->hwtstamps;
+ skb_shinfo(new_skb)->tskey = skb_shinfo(skb)->tskey;
+ if (skb->sk)
+ skb_set_owner_w(new_skb, skb->sk);
+ }
+
/* We move the headroom when we align it so we have to reset the
* network and transport header offsets relative to the new data
* pointer. The checksum offload relies on these offsets.
skb_set_network_header(new_skb, skb_network_offset(skb));
skb_set_transport_header(new_skb, skb_transport_offset(skb));
- /* TODO: does timestamping need the result in the old skb? */
dev_kfree_skb(skb);
*s = new_skb;
return 0;
}
+
+static int dpaa_a050385_wa_xdpf(struct dpaa_priv *priv,
+ struct xdp_frame **init_xdpf)
+{
+ struct xdp_frame *new_xdpf, *xdpf = *init_xdpf;
+ void *new_buff;
+ struct page *p;
+
+ /* Check the data alignment and make sure the headroom is large
+ * enough to store the xdpf backpointer. Use an aligned headroom
+ * value.
+ *
+ * Due to alignment constraints, we give XDP access to the full 256
+ * byte frame headroom. If the XDP program uses all of it, copy the
+ * data to a new buffer and make room for storing the backpointer.
+ */
+ if (PTR_IS_ALIGNED(xdpf->data, DPAA_A050385_ALIGN) &&
+ xdpf->headroom >= priv->tx_headroom) {
+ xdpf->headroom = priv->tx_headroom;
+ return 0;
+ }
+
+ p = dev_alloc_pages(0);
+ if (unlikely(!p))
+ return -ENOMEM;
+
+ /* Copy the data to the new buffer at a properly aligned offset */
+ new_buff = page_address(p);
+ memcpy(new_buff + priv->tx_headroom, xdpf->data, xdpf->len);
+
+ /* Create an XDP frame around the new buffer in a similar fashion
+ * to xdp_convert_buff_to_frame.
+ */
+ new_xdpf = new_buff;
+ new_xdpf->data = new_buff + priv->tx_headroom;
+ new_xdpf->len = xdpf->len;
+ new_xdpf->headroom = priv->tx_headroom;
+ new_xdpf->frame_sz = DPAA_BP_RAW_SIZE;
+ new_xdpf->mem.type = MEM_TYPE_PAGE_ORDER0;
+
+ /* Release the initial buffer */
+ xdp_return_frame_rx_napi(xdpf);
+
+ *init_xdpf = new_xdpf;
+ return 0;
+}
#endif
static netdev_tx_t
#ifdef CONFIG_DPAA_ERRATUM_A050385
if (unlikely(fman_has_errata_a050385())) {
- if (dpaa_a050385_wa(net_dev, &skb))
+ if (dpaa_a050385_wa_skb(net_dev, &skb))
goto enomem;
nonlinear = skb_is_nonlinear(skb);
}
{
struct dpaa_napi_portal *np =
container_of(napi, struct dpaa_napi_portal, napi);
+ int cleaned;
+
+ np->xdp_act = 0;
- int cleaned = qman_p_poll_dqrr(np->p, budget);
+ cleaned = qman_p_poll_dqrr(np->p, budget);
if (cleaned < budget) {
napi_complete_done(napi, cleaned);
qman_p_irqsource_add(np->p, QM_PIRQ_DQRI);
}
+ if (np->xdp_act & XDP_REDIRECT)
+ xdp_do_flush();
+
return cleaned;
}
return qman_cb_dqrr_consume;
}
+static int dpaa_xdp_xmit_frame(struct net_device *net_dev,
+ struct xdp_frame *xdpf)
+{
+ struct dpaa_priv *priv = netdev_priv(net_dev);
+ struct rtnl_link_stats64 *percpu_stats;
+ struct dpaa_percpu_priv *percpu_priv;
+ struct dpaa_eth_swbp *swbp;
+ struct netdev_queue *txq;
+ void *buff_start;
+ struct qm_fd fd;
+ dma_addr_t addr;
+ int err;
+
+ percpu_priv = this_cpu_ptr(priv->percpu_priv);
+ percpu_stats = &percpu_priv->stats;
+
+#ifdef CONFIG_DPAA_ERRATUM_A050385
+ if (unlikely(fman_has_errata_a050385())) {
+ if (dpaa_a050385_wa_xdpf(priv, &xdpf)) {
+ err = -ENOMEM;
+ goto out_error;
+ }
+ }
+#endif
+
+ if (xdpf->headroom < DPAA_TX_PRIV_DATA_SIZE) {
+ err = -EINVAL;
+ goto out_error;
+ }
+
+ buff_start = xdpf->data - xdpf->headroom;
+
+ /* Leave empty the skb backpointer at the start of the buffer.
+ * Save the XDP frame for easy cleanup on confirmation.
+ */
+ swbp = (struct dpaa_eth_swbp *)buff_start;
+ swbp->skb = NULL;
+ swbp->xdpf = xdpf;
+
+ qm_fd_clear_fd(&fd);
+ fd.bpid = FSL_DPAA_BPID_INV;
+ fd.cmd |= cpu_to_be32(FM_FD_CMD_FCO);
+ qm_fd_set_contig(&fd, xdpf->headroom, xdpf->len);
+
+ addr = dma_map_single(priv->tx_dma_dev, buff_start,
+ xdpf->headroom + xdpf->len,
+ DMA_TO_DEVICE);
+ if (unlikely(dma_mapping_error(priv->tx_dma_dev, addr))) {
+ err = -EINVAL;
+ goto out_error;
+ }
+
+ qm_fd_addr_set64(&fd, addr);
+
+ /* Bump the trans_start */
+ txq = netdev_get_tx_queue(net_dev, smp_processor_id());
+ txq->trans_start = jiffies;
+
+ err = dpaa_xmit(priv, percpu_stats, smp_processor_id(), &fd);
+ if (err) {
+ dma_unmap_single(priv->tx_dma_dev, addr,
+ qm_fd_get_offset(&fd) + qm_fd_get_length(&fd),
+ DMA_TO_DEVICE);
+ goto out_error;
+ }
+
+ return 0;
+
+out_error:
+ percpu_stats->tx_errors++;
+ return err;
+}
+
+static u32 dpaa_run_xdp(struct dpaa_priv *priv, struct qm_fd *fd, void *vaddr,
+ struct dpaa_fq *dpaa_fq, unsigned int *xdp_meta_len)
+{
+ ssize_t fd_off = qm_fd_get_offset(fd);
+ struct bpf_prog *xdp_prog;
+ struct xdp_frame *xdpf;
+ struct xdp_buff xdp;
+ u32 xdp_act;
+ int err;
+
+ rcu_read_lock();
+
+ xdp_prog = READ_ONCE(priv->xdp_prog);
+ if (!xdp_prog) {
+ rcu_read_unlock();
+ return XDP_PASS;
+ }
+
+ xdp.data = vaddr + fd_off;
+ xdp.data_meta = xdp.data;
+ xdp.data_hard_start = xdp.data - XDP_PACKET_HEADROOM;
+ xdp.data_end = xdp.data + qm_fd_get_length(fd);
+ xdp.frame_sz = DPAA_BP_RAW_SIZE - DPAA_TX_PRIV_DATA_SIZE;
+ xdp.rxq = &dpaa_fq->xdp_rxq;
+
+ /* We reserve a fixed headroom of 256 bytes under the erratum and we
+ * offer it all to XDP programs to use. If no room is left for the
+ * xdpf backpointer on TX, we will need to copy the data.
+ * Disable metadata support since data realignments might be required
+ * and the information can be lost.
+ */
+#ifdef CONFIG_DPAA_ERRATUM_A050385
+ if (unlikely(fman_has_errata_a050385())) {
+ xdp_set_data_meta_invalid(&xdp);
+ xdp.data_hard_start = vaddr;
+ xdp.frame_sz = DPAA_BP_RAW_SIZE;
+ }
+#endif
+
+ xdp_act = bpf_prog_run_xdp(xdp_prog, &xdp);
+
+ /* Update the length and the offset of the FD */
+ qm_fd_set_contig(fd, xdp.data - vaddr, xdp.data_end - xdp.data);
+
+ switch (xdp_act) {
+ case XDP_PASS:
+#ifdef CONFIG_DPAA_ERRATUM_A050385
+ *xdp_meta_len = xdp_data_meta_unsupported(&xdp) ? 0 :
+ xdp.data - xdp.data_meta;
+#else
+ *xdp_meta_len = xdp.data - xdp.data_meta;
+#endif
+ break;
+ case XDP_TX:
+ /* We can access the full headroom when sending the frame
+ * back out
+ */
+ xdp.data_hard_start = vaddr;
+ xdp.frame_sz = DPAA_BP_RAW_SIZE;
+ xdpf = xdp_convert_buff_to_frame(&xdp);
+ if (unlikely(!xdpf)) {
+ free_pages((unsigned long)vaddr, 0);
+ break;
+ }
+
+ if (dpaa_xdp_xmit_frame(priv->net_dev, xdpf))
+ xdp_return_frame_rx_napi(xdpf);
+
+ break;
+ case XDP_REDIRECT:
+ /* Allow redirect to use the full headroom */
+ xdp.data_hard_start = vaddr;
+ xdp.frame_sz = DPAA_BP_RAW_SIZE;
+
+ err = xdp_do_redirect(priv->net_dev, &xdp, xdp_prog);
+ if (err) {
+ trace_xdp_exception(priv->net_dev, xdp_prog, xdp_act);
+ free_pages((unsigned long)vaddr, 0);
+ }
+ break;
+ default:
+ bpf_warn_invalid_xdp_action(xdp_act);
+ fallthrough;
+ case XDP_ABORTED:
+ trace_xdp_exception(priv->net_dev, xdp_prog, xdp_act);
+ fallthrough;
+ case XDP_DROP:
+ /* Free the buffer */
+ free_pages((unsigned long)vaddr, 0);
+ break;
+ }
+
+ rcu_read_unlock();
+
+ return xdp_act;
+}
+
static enum qman_cb_dqrr_result rx_default_dqrr(struct qman_portal *portal,
struct qman_fq *fq,
const struct qm_dqrr_entry *dq,
bool sched_napi)
{
+ bool ts_valid = false, hash_valid = false;
struct skb_shared_hwtstamps *shhwtstamps;
+ unsigned int skb_len, xdp_meta_len = 0;
struct rtnl_link_stats64 *percpu_stats;
struct dpaa_percpu_priv *percpu_priv;
const struct qm_fd *fd = &dq->fd;
dma_addr_t addr = qm_fd_addr(fd);
+ struct dpaa_napi_portal *np;
enum qm_fd_format fd_format;
struct net_device *net_dev;
u32 fd_status, hash_offset;
+ struct qm_sg_entry *sgt;
struct dpaa_bp *dpaa_bp;
+ struct dpaa_fq *dpaa_fq;
struct dpaa_priv *priv;
- unsigned int skb_len;
struct sk_buff *skb;
int *count_ptr;
+ u32 xdp_act;
void *vaddr;
+ u32 hash;
u64 ns;
+ np = container_of(&portal, struct dpaa_napi_portal, p);
+ dpaa_fq = container_of(fq, struct dpaa_fq, fq_base);
fd_status = be32_to_cpu(fd->status);
fd_format = qm_fd_get_format(fd);
- net_dev = ((struct dpaa_fq *)fq)->net_dev;
+ net_dev = dpaa_fq->net_dev;
priv = netdev_priv(net_dev);
dpaa_bp = dpaa_bpid2pool(dq->fd.bpid);
if (!dpaa_bp)
count_ptr = this_cpu_ptr(dpaa_bp->percpu_count);
(*count_ptr)--;
- if (likely(fd_format == qm_fd_contig))
+ /* Extract the timestamp stored in the headroom before running XDP */
+ if (priv->rx_tstamp) {
+ if (!fman_port_get_tstamp(priv->mac_dev->port[RX], vaddr, &ns))
+ ts_valid = true;
+ else
+ WARN_ONCE(1, "fman_port_get_tstamp failed!\n");
+ }
+
+ /* Extract the hash stored in the headroom before running XDP */
+ if (net_dev->features & NETIF_F_RXHASH && priv->keygen_in_use &&
+ !fman_port_get_hash_result_offset(priv->mac_dev->port[RX],
+ &hash_offset)) {
+ hash = be32_to_cpu(*(u32 *)(vaddr + hash_offset));
+ hash_valid = true;
+ }
+
+ if (likely(fd_format == qm_fd_contig)) {
+ xdp_act = dpaa_run_xdp(priv, (struct qm_fd *)fd, vaddr,
+ dpaa_fq, &xdp_meta_len);
+ np->xdp_act |= xdp_act;
+ if (xdp_act != XDP_PASS) {
+ percpu_stats->rx_packets++;
+ percpu_stats->rx_bytes += qm_fd_get_length(fd);
+ return qman_cb_dqrr_consume;
+ }
skb = contig_fd_to_skb(priv, fd);
- else
+ } else {
+ /* XDP doesn't support S/G frames. Return the fragments to the
+ * buffer pool and release the SGT.
+ */
+ if (READ_ONCE(priv->xdp_prog)) {
+ WARN_ONCE(1, "S/G frames not supported under XDP\n");
+ sgt = vaddr + qm_fd_get_offset(fd);
+ dpaa_release_sgt_members(sgt);
+ free_pages((unsigned long)vaddr, 0);
+ return qman_cb_dqrr_consume;
+ }
skb = sg_fd_to_skb(priv, fd);
+ }
if (!skb)
return qman_cb_dqrr_consume;
- if (priv->rx_tstamp) {
+ if (xdp_meta_len)
+ skb_metadata_set(skb, xdp_meta_len);
+
+ /* Set the previously extracted timestamp */
+ if (ts_valid) {
shhwtstamps = skb_hwtstamps(skb);
memset(shhwtstamps, 0, sizeof(*shhwtstamps));
-
- if (!fman_port_get_tstamp(priv->mac_dev->port[RX], vaddr, &ns))
- shhwtstamps->hwtstamp = ns_to_ktime(ns);
- else
- dev_warn(net_dev->dev.parent, "fman_port_get_tstamp failed!\n");
+ shhwtstamps->hwtstamp = ns_to_ktime(ns);
}
skb->protocol = eth_type_trans(skb, net_dev);
- if (net_dev->features & NETIF_F_RXHASH && priv->keygen_in_use &&
- !fman_port_get_hash_result_offset(priv->mac_dev->port[RX],
- &hash_offset)) {
+ /* Set the previously extracted hash */
+ if (hash_valid) {
enum pkt_hash_types type;
/* if L4 exists, it was used in the hash generation */
type = be32_to_cpu(fd->status) & FM_FD_STAT_L4CV ?
PKT_HASH_TYPE_L4 : PKT_HASH_TYPE_L3;
- skb_set_hash(skb, be32_to_cpu(*(u32 *)(vaddr + hash_offset)),
- type);
+ skb_set_hash(skb, hash, type);
}
skb_len = skb->len;
return err;
}
+static bool xdp_validate_mtu(struct dpaa_priv *priv, int mtu)
+{
+ int max_contig_data = priv->dpaa_bp->size - priv->rx_headroom;
+
+ /* We do not support S/G fragments when XDP is enabled.
+ * Limit the MTU in relation to the buffer size.
+ */
+ if (mtu + VLAN_ETH_HLEN + ETH_FCS_LEN > max_contig_data) {
+ dev_warn(priv->net_dev->dev.parent,
+ "The maximum MTU for XDP is %d\n",
+ max_contig_data - VLAN_ETH_HLEN - ETH_FCS_LEN);
+ return false;
+ }
+
+ return true;
+}
+
+static int dpaa_change_mtu(struct net_device *net_dev, int new_mtu)
+{
+ struct dpaa_priv *priv = netdev_priv(net_dev);
+
+ if (priv->xdp_prog && !xdp_validate_mtu(priv, new_mtu))
+ return -EINVAL;
+
+ net_dev->mtu = new_mtu;
+ return 0;
+}
+
+static int dpaa_setup_xdp(struct net_device *net_dev, struct netdev_bpf *bpf)
+{
+ struct dpaa_priv *priv = netdev_priv(net_dev);
+ struct bpf_prog *old_prog;
+ int err;
+ bool up;
+
+ /* S/G fragments are not supported in XDP-mode */
+ if (bpf->prog && !xdp_validate_mtu(priv, net_dev->mtu)) {
+ NL_SET_ERR_MSG_MOD(bpf->extack, "MTU too large for XDP");
+ return -EINVAL;
+ }
+
+ up = netif_running(net_dev);
+
+ if (up)
+ dpaa_eth_stop(net_dev);
+
+ old_prog = xchg(&priv->xdp_prog, bpf->prog);
+ if (old_prog)
+ bpf_prog_put(old_prog);
+
+ if (up) {
+ err = dpaa_open(net_dev);
+ if (err) {
+ NL_SET_ERR_MSG_MOD(bpf->extack, "dpaa_open() failed");
+ return err;
+ }
+ }
+
+ return 0;
+}
+
+static int dpaa_xdp(struct net_device *net_dev, struct netdev_bpf *xdp)
+{
+ switch (xdp->command) {
+ case XDP_SETUP_PROG:
+ return dpaa_setup_xdp(net_dev, xdp);
+ default:
+ return -EINVAL;
+ }
+}
+
+static int dpaa_xdp_xmit(struct net_device *net_dev, int n,
+ struct xdp_frame **frames, u32 flags)
+{
+ struct xdp_frame *xdpf;
+ int i, err, drops = 0;
+
+ if (unlikely(flags & ~XDP_XMIT_FLAGS_MASK))
+ return -EINVAL;
+
+ if (!netif_running(net_dev))
+ return -ENETDOWN;
+
+ for (i = 0; i < n; i++) {
+ xdpf = frames[i];
+ err = dpaa_xdp_xmit_frame(net_dev, xdpf);
+ if (err) {
+ xdp_return_frame_rx_napi(xdpf);
+ drops++;
+ }
+ }
+
+ return n - drops;
+}
+
static int dpaa_ts_ioctl(struct net_device *dev, struct ifreq *rq, int cmd)
{
struct dpaa_priv *priv = netdev_priv(dev);
.ndo_set_rx_mode = dpaa_set_rx_mode,
.ndo_do_ioctl = dpaa_ioctl,
.ndo_setup_tc = dpaa_setup_tc,
+ .ndo_change_mtu = dpaa_change_mtu,
+ .ndo_bpf = dpaa_xdp,
+ .ndo_xdp_xmit = dpaa_xdp_xmit,
};
static int dpaa_napi_add(struct net_device *net_dev)
*/
headroom = (u16)(bl[port].priv_data_size + DPAA_HWA_SIZE);
- if (port == RX)
+ if (port == RX) {
+#ifdef CONFIG_DPAA_ERRATUM_A050385
+ if (unlikely(fman_has_errata_a050385()))
+ headroom = XDP_PACKET_HEADROOM;
+#endif
+
return ALIGN(headroom, DPAA_FD_RX_DATA_ALIGNMENT);
- else
+ } else {
return ALIGN(headroom, DPAA_FD_DATA_ALIGNMENT);
+ }
}
static int dpaa_eth_probe(struct platform_device *pdev)
u16 channel;
u8 wq;
enum dpaa_fq_type fq_type;
+ struct xdp_rxq_info xdp_rxq;
};
struct dpaa_fq_cbs {
struct napi_struct napi;
struct qman_portal *p;
bool down;
+ int xdp_act;
};
struct dpaa_percpu_priv {
u16 priv_data_size;
};
+/* Information to be used on the Tx confirmation path. Stored just
+ * before the start of the transmit buffer. Maximum size allowed
+ * is DPAA_TX_PRIV_DATA_SIZE bytes.
+ */
+struct dpaa_eth_swbp {
+ struct sk_buff *skb;
+ struct xdp_frame *xdpf;
+};
+
struct dpaa_priv {
struct dpaa_percpu_priv __percpu *percpu_priv;
struct dpaa_bp *dpaa_bp;
bool tx_tstamp; /* Tx timestamping enabled */
bool rx_tstamp; /* Rx timestamping enabled */
+
+ struct bpf_prog *xdp_prog;
};
/* from dpaa_ethtool.c */
depends on FSL_MC_BUS && FSL_MC_DPIO
select PHYLINK
select PCS_LYNX
+ select FSL_XGMAC_MDIO
+ select NET_DEVLINK
help
This is the DPAA2 Ethernet driver supporting Freescale SoCs
with DPAA2 (DataPath Acceleration Architecture v2).
if (skb->cb[0] == TX_TSTAMP_ONESTEP_SYNC) {
if (dpaa2_eth_ptp_parse(skb, &msgtype, &twostep, &udp,
&offset1, &offset2) ||
- msgtype != 0 || twostep) {
+ msgtype != PTP_MSGTYPE_SYNC || twostep) {
WARN_ONCE(1, "Bad packet for one-step timestamping\n");
return;
}
if (skb->cb[0] == TX_TSTAMP_ONESTEP_SYNC) {
if (!dpaa2_eth_ptp_parse(skb, &msgtype, &twostep, &udp,
&offset1, &offset2))
- if (msgtype == 0 && twostep == 0) {
+ if (msgtype == PTP_MSGTYPE_SYNC && twostep == 0) {
skb_queue_tail(&priv->tx_skbs, skb);
queue_work(priv->dpaa2_ptp_wq,
&priv->tx_onestep_tstamp);
config FSL_ENETC_VF
tristate "ENETC VF driver"
depends on PCI && PCI_MSI
+ select FSL_ENETC_MDIO
select PHYLINK
select DIMLIB
help
return NETDEV_TX_BUSY;
}
+ enetc_lock_mdio();
count = enetc_map_tx_buffs(tx_ring, skb, priv->active_offloads);
+ enetc_unlock_mdio();
+
if (unlikely(!count))
goto drop_packet_err;
skb_tx_timestamp(skb);
/* let H/W know BD ring has been updated */
- enetc_wr_reg(tx_ring->tpir, i); /* includes wmb() */
+ enetc_wr_reg_hot(tx_ring->tpir, i); /* includes wmb() */
return count;
struct enetc_int_vector *v = data;
int i;
+ enetc_lock_mdio();
+
/* disable interrupts */
- enetc_wr_reg(v->rbier, 0);
- enetc_wr_reg(v->ricr1, v->rx_ictt);
+ enetc_wr_reg_hot(v->rbier, 0);
+ enetc_wr_reg_hot(v->ricr1, v->rx_ictt);
for_each_set_bit(i, &v->tx_rings_map, ENETC_MAX_NUM_TXQS)
- enetc_wr_reg(v->tbier_base + ENETC_BDR_OFF(i), 0);
+ enetc_wr_reg_hot(v->tbier_base + ENETC_BDR_OFF(i), 0);
+
+ enetc_unlock_mdio();
napi_schedule(&v->napi);
v->rx_napi_work = false;
+ enetc_lock_mdio();
+
/* enable interrupts */
- enetc_wr_reg(v->rbier, ENETC_RBIER_RXTIE);
+ enetc_wr_reg_hot(v->rbier, ENETC_RBIER_RXTIE);
for_each_set_bit(i, &v->tx_rings_map, ENETC_MAX_NUM_TXQS)
- enetc_wr_reg(v->tbier_base + ENETC_BDR_OFF(i),
- ENETC_TBIER_TXTIE);
+ enetc_wr_reg_hot(v->tbier_base + ENETC_BDR_OFF(i),
+ ENETC_TBIER_TXTIE);
+
+ enetc_unlock_mdio();
return work_done;
}
static int enetc_bd_ready_count(struct enetc_bdr *tx_ring, int ci)
{
- int pi = enetc_rd_reg(tx_ring->tcir) & ENETC_TBCIR_IDX_MASK;
+ int pi = enetc_rd_reg_hot(tx_ring->tcir) & ENETC_TBCIR_IDX_MASK;
return pi >= ci ? pi - ci : tx_ring->bd_count - ci + pi;
}
i = tx_ring->next_to_clean;
tx_swbd = &tx_ring->tx_swbd[i];
+
+ enetc_lock_mdio();
bds_to_clean = enetc_bd_ready_count(tx_ring, i);
+ enetc_unlock_mdio();
do_tstamp = false;
tx_swbd = tx_ring->tx_swbd;
}
+ enetc_lock_mdio();
+
/* BD iteration loop end */
if (is_eof) {
tx_frm_cnt++;
/* re-arm interrupt source */
- enetc_wr_reg(tx_ring->idr, BIT(tx_ring->index) |
- BIT(16 + tx_ring->index));
+ enetc_wr_reg_hot(tx_ring->idr, BIT(tx_ring->index) |
+ BIT(16 + tx_ring->index));
}
if (unlikely(!bds_to_clean))
bds_to_clean = enetc_bd_ready_count(tx_ring, i);
+
+ enetc_unlock_mdio();
}
tx_ring->next_to_clean = i;
if (likely(j)) {
rx_ring->next_to_alloc = i; /* keep track from page reuse */
rx_ring->next_to_use = i;
- /* update ENETC's consumer index */
- enetc_wr_reg(rx_ring->rcir, i);
}
return j;
u64 tstamp;
if (le16_to_cpu(rxbd->r.flags) & ENETC_RXBD_FLAG_TSTMP) {
- lo = enetc_rd(hw, ENETC_SICTR0);
- hi = enetc_rd(hw, ENETC_SICTR1);
+ lo = enetc_rd_reg_hot(hw->reg + ENETC_SICTR0);
+ hi = enetc_rd_reg_hot(hw->reg + ENETC_SICTR1);
rxbd = enetc_rxbd_ext(rxbd);
tstamp_lo = le32_to_cpu(rxbd->ext.tstamp);
if (lo <= tstamp_lo)
u32 bd_status;
u16 size;
+ enetc_lock_mdio();
+
if (cleaned_cnt >= ENETC_RXBD_BUNDLE) {
int count = enetc_refill_rx_ring(rx_ring, cleaned_cnt);
+ /* update ENETC's consumer index */
+ enetc_wr_reg_hot(rx_ring->rcir, rx_ring->next_to_use);
cleaned_cnt -= count;
}
rxbd = enetc_rxbd(rx_ring, i);
bd_status = le32_to_cpu(rxbd->r.lstatus);
- if (!bd_status)
+ if (!bd_status) {
+ enetc_unlock_mdio();
break;
+ }
- enetc_wr_reg(rx_ring->idr, BIT(rx_ring->index));
+ enetc_wr_reg_hot(rx_ring->idr, BIT(rx_ring->index));
dma_rmb(); /* for reading other rxbd fields */
size = le16_to_cpu(rxbd->r.buf_len);
skb = enetc_map_rx_buff_to_skb(rx_ring, i, size);
- if (!skb)
+ if (!skb) {
+ enetc_unlock_mdio();
break;
+ }
enetc_get_offloads(rx_ring, rxbd, skb);
if (unlikely(bd_status &
ENETC_RXBD_LSTATUS(ENETC_RXBD_ERR_MASK))) {
+ enetc_unlock_mdio();
dev_kfree_skb(skb);
while (!(bd_status & ENETC_RXBD_LSTATUS_F)) {
dma_rmb();
enetc_process_skb(rx_ring, skb);
+ enetc_unlock_mdio();
+
napi_gro_receive(napi, skb);
rx_frm_cnt++;
rx_ring->idr = hw->reg + ENETC_SIRXIDR;
enetc_refill_rx_ring(rx_ring, enetc_bd_unused(rx_ring));
+ enetc_wr(hw, ENETC_SIRXIDR, rx_ring->next_to_use);
/* enable ring */
enetc_rxbdr_wr(hw, idx, ENETC_RBMR, rbmr);
void __iomem *global;
};
-/* general register accessors */
-#define enetc_rd_reg(reg) ioread32((reg))
-#define enetc_wr_reg(reg, val) iowrite32((val), (reg))
+/* ENETC register accessors */
+
+/* MDIO issue workaround (on LS1028A) -
+ * Due to a hardware issue, an access to MDIO registers
+ * that is concurrent with other ENETC register accesses
+ * may lead to the MDIO access being dropped or corrupted.
+ * To protect the MDIO accesses a readers-writers locking
+ * scheme is used, where the MDIO register accesses are
+ * protected by write locks to insure exclusivity, while
+ * the remaining ENETC registers are accessed under read
+ * locks since they only compete with MDIO accesses.
+ */
+extern rwlock_t enetc_mdio_lock;
+
+/* use this locking primitive only on the fast datapath to
+ * group together multiple non-MDIO register accesses to
+ * minimize the overhead of the lock
+ */
+static inline void enetc_lock_mdio(void)
+{
+ read_lock(&enetc_mdio_lock);
+}
+
+static inline void enetc_unlock_mdio(void)
+{
+ read_unlock(&enetc_mdio_lock);
+}
+
+/* use these accessors only on the fast datapath under
+ * the enetc_lock_mdio() locking primitive to minimize
+ * the overhead of the lock
+ */
+static inline u32 enetc_rd_reg_hot(void __iomem *reg)
+{
+ lockdep_assert_held(&enetc_mdio_lock);
+
+ return ioread32(reg);
+}
+
+static inline void enetc_wr_reg_hot(void __iomem *reg, u32 val)
+{
+ lockdep_assert_held(&enetc_mdio_lock);
+
+ iowrite32(val, reg);
+}
+
+/* internal helpers for the MDIO w/a */
+static inline u32 _enetc_rd_reg_wa(void __iomem *reg)
+{
+ u32 val;
+
+ enetc_lock_mdio();
+ val = ioread32(reg);
+ enetc_unlock_mdio();
+
+ return val;
+}
+
+static inline void _enetc_wr_reg_wa(void __iomem *reg, u32 val)
+{
+ enetc_lock_mdio();
+ iowrite32(val, reg);
+ enetc_unlock_mdio();
+}
+
+static inline u32 _enetc_rd_mdio_reg_wa(void __iomem *reg)
+{
+ unsigned long flags;
+ u32 val;
+
+ write_lock_irqsave(&enetc_mdio_lock, flags);
+ val = ioread32(reg);
+ write_unlock_irqrestore(&enetc_mdio_lock, flags);
+
+ return val;
+}
+
+static inline void _enetc_wr_mdio_reg_wa(void __iomem *reg, u32 val)
+{
+ unsigned long flags;
+
+ write_lock_irqsave(&enetc_mdio_lock, flags);
+ iowrite32(val, reg);
+ write_unlock_irqrestore(&enetc_mdio_lock, flags);
+}
+
#ifdef ioread64
-#define enetc_rd_reg64(reg) ioread64((reg))
+static inline u64 _enetc_rd_reg64(void __iomem *reg)
+{
+ return ioread64(reg);
+}
#else
/* using this to read out stats on 32b systems */
-static inline u64 enetc_rd_reg64(void __iomem *reg)
+static inline u64 _enetc_rd_reg64(void __iomem *reg)
{
u32 low, high, tmp;
}
#endif
+static inline u64 _enetc_rd_reg64_wa(void __iomem *reg)
+{
+ u64 val;
+
+ enetc_lock_mdio();
+ val = _enetc_rd_reg64(reg);
+ enetc_unlock_mdio();
+
+ return val;
+}
+
+/* general register accessors */
+#define enetc_rd_reg(reg) _enetc_rd_reg_wa((reg))
+#define enetc_wr_reg(reg, val) _enetc_wr_reg_wa((reg), (val))
#define enetc_rd(hw, off) enetc_rd_reg((hw)->reg + (off))
#define enetc_wr(hw, off, val) enetc_wr_reg((hw)->reg + (off), val)
-#define enetc_rd64(hw, off) enetc_rd_reg64((hw)->reg + (off))
+#define enetc_rd64(hw, off) _enetc_rd_reg64_wa((hw)->reg + (off))
/* port register accessors - PF only */
#define enetc_port_rd(hw, off) enetc_rd_reg((hw)->port + (off))
#define enetc_port_wr(hw, off, val) enetc_wr_reg((hw)->port + (off), val)
+#define enetc_port_rd_mdio(hw, off) _enetc_rd_mdio_reg_wa((hw)->port + (off))
+#define enetc_port_wr_mdio(hw, off, val) _enetc_wr_mdio_reg_wa(\
+ (hw)->port + (off), val)
/* global register accessors - PF only */
#define enetc_global_rd(hw, off) enetc_rd_reg((hw)->global + (off))
#define enetc_global_wr(hw, off, val) enetc_wr_reg((hw)->global + (off), val)
u8 smac_m[6];
u8 dmac_h[6];
u8 dmac_m[6];
- u32 sip_h[4];
- u32 sip_m[4];
- u32 dip_h[4];
- u32 dip_m[4];
+ __be32 sip_h[4];
+ __be32 sip_m[4];
+ __be32 dip_h[4];
+ __be32 dip_m[4];
u16 ethtype_h;
u16 ethtype_m;
u16 ethtype4_h;
static inline u32 _enetc_mdio_rd(struct enetc_mdio_priv *mdio_priv, int off)
{
- return enetc_port_rd(mdio_priv->hw, mdio_priv->mdio_base + off);
+ return enetc_port_rd_mdio(mdio_priv->hw, mdio_priv->mdio_base + off);
}
static inline void _enetc_mdio_wr(struct enetc_mdio_priv *mdio_priv, int off,
u32 val)
{
- enetc_port_wr(mdio_priv->hw, mdio_priv->mdio_base + off, val);
+ enetc_port_wr_mdio(mdio_priv->hw, mdio_priv->mdio_base + off, val);
}
#define enetc_mdio_rd(mdio_priv, off) \
return hw;
}
EXPORT_SYMBOL_GPL(enetc_hw_alloc);
+
+/* Lock for MDIO access errata on LS1028A */
+DEFINE_RWLOCK(enetc_mdio_lock);
+EXPORT_SYMBOL_GPL(enetc_mdio_lock);
gcl_config->atc = 0xff;
gcl_config->acl_len = cpu_to_le16(gcl_len);
- if (!admin_conf->base_time) {
- gcl_data->btl =
- cpu_to_le32(enetc_rd(&priv->si->hw, ENETC_SICTR0));
- gcl_data->bth =
- cpu_to_le32(enetc_rd(&priv->si->hw, ENETC_SICTR1));
- } else {
- gcl_data->btl =
- cpu_to_le32(lower_32_bits(admin_conf->base_time));
- gcl_data->bth =
- cpu_to_le32(upper_32_bits(admin_conf->base_time));
- }
-
+ gcl_data->btl = cpu_to_le32(lower_32_bits(admin_conf->base_time));
+ gcl_data->bth = cpu_to_le32(upper_32_bits(admin_conf->base_time));
gcl_data->ct = cpu_to_le32(admin_conf->cycle_time);
gcl_data->cte = cpu_to_le32(admin_conf->cycle_time_extension);
return -ENOMEM;
}
- cbd.addr[0] = lower_32_bits(dma);
- cbd.addr[1] = upper_32_bits(dma);
+ cbd.addr[0] = cpu_to_le32(lower_32_bits(dma));
+ cbd.addr[1] = cpu_to_le32(upper_32_bits(dma));
cbd.cls = BDCR_CMD_PORT_GCL;
cbd.status_flags = 0;
return -ENOMEM;
}
- cbd.addr[0] = lower_32_bits(dma);
- cbd.addr[1] = upper_32_bits(dma);
+ cbd.addr[0] = cpu_to_le32(lower_32_bits(dma));
+ cbd.addr[1] = cpu_to_le32(upper_32_bits(dma));
eth_broadcast_addr(si_data->dmac);
- si_data->vid_vidm_tg =
- cpu_to_le16(ENETC_CBDR_SID_VID_MASK
- + ((0x3 << 14) | ENETC_CBDR_SID_VIDM));
+ si_data->vid_vidm_tg = (ENETC_CBDR_SID_VID_MASK
+ + ((0x3 << 14) | ENETC_CBDR_SID_VIDM));
si_conf = &cbd.sid_set;
/* Only one port supported for one entry, set itself */
- si_conf->iports = 1 << enetc_get_port(priv);
+ si_conf->iports = cpu_to_le32(1 << enetc_get_port(priv));
si_conf->id_type = 1;
si_conf->oui[2] = 0x0;
si_conf->oui[1] = 0x80;
si_conf->en = 0x80;
si_conf->stream_handle = cpu_to_le32(sid->handle);
- si_conf->iports = 1 << enetc_get_port(priv);
+ si_conf->iports = cpu_to_le32(1 << enetc_get_port(priv));
si_conf->id_type = sid->filtertype;
si_conf->oui[2] = 0x0;
si_conf->oui[1] = 0x80;
cbd.length = cpu_to_le16(data_size);
- cbd.addr[0] = lower_32_bits(dma);
- cbd.addr[1] = upper_32_bits(dma);
+ cbd.addr[0] = cpu_to_le32(lower_32_bits(dma));
+ cbd.addr[1] = cpu_to_le32(upper_32_bits(dma));
/* VIDM default to be 1.
* VID Match. If set (b1) then the VID must match, otherwise
*/
if (si_conf->id_type == STREAMID_TYPE_NULL) {
ether_addr_copy(si_data->dmac, sid->dst_mac);
- si_data->vid_vidm_tg =
- cpu_to_le16((sid->vid & ENETC_CBDR_SID_VID_MASK) +
- ((((u16)(sid->tagged) & 0x3) << 14)
- | ENETC_CBDR_SID_VIDM));
+ si_data->vid_vidm_tg = (sid->vid & ENETC_CBDR_SID_VID_MASK) +
+ ((((u16)(sid->tagged) & 0x3) << 14)
+ | ENETC_CBDR_SID_VIDM);
} else if (si_conf->id_type == STREAMID_TYPE_SMAC) {
ether_addr_copy(si_data->smac, sid->src_mac);
- si_data->vid_vidm_tg =
- cpu_to_le16((sid->vid & ENETC_CBDR_SID_VID_MASK) +
- ((((u16)(sid->tagged) & 0x3) << 14)
- | ENETC_CBDR_SID_VIDM));
+ si_data->vid_vidm_tg = (sid->vid & ENETC_CBDR_SID_VID_MASK) +
+ ((((u16)(sid->tagged) & 0x3) << 14)
+ | ENETC_CBDR_SID_VIDM);
}
err = enetc_send_cmd(priv->si, &cbd);
}
sfi_config->sg_inst_table_index = cpu_to_le16(sfi->gate_id);
- sfi_config->input_ports = 1 << enetc_get_port(priv);
+ sfi_config->input_ports = cpu_to_le32(1 << enetc_get_port(priv));
/* The priority value which may be matched against the
* frame’s priority value to determine a match for this entry.
err = -ENOMEM;
goto exit;
}
- cbd.addr[0] = lower_32_bits(dma);
- cbd.addr[1] = upper_32_bits(dma);
+ cbd.addr[0] = cpu_to_le32(lower_32_bits(dma));
+ cbd.addr[1] = cpu_to_le32(upper_32_bits(dma));
cbd.length = cpu_to_le16(data_size);
if (err)
goto exit;
- cnt->matching_frames_count =
- ((u64)le32_to_cpu(data_buf->matchh) << 32)
- + data_buf->matchl;
+ cnt->matching_frames_count = ((u64)data_buf->matchh << 32) +
+ data_buf->matchl;
- cnt->not_passing_sdu_count =
- ((u64)le32_to_cpu(data_buf->msdu_droph) << 32)
- + data_buf->msdu_dropl;
+ cnt->not_passing_sdu_count = ((u64)data_buf->msdu_droph << 32) +
+ data_buf->msdu_dropl;
cnt->passing_sdu_count = cnt->matching_frames_count
- cnt->not_passing_sdu_count;
cnt->not_passing_frames_count =
- ((u64)le32_to_cpu(data_buf->stream_gate_droph) << 32)
- + le32_to_cpu(data_buf->stream_gate_dropl);
+ ((u64)data_buf->stream_gate_droph << 32) +
+ data_buf->stream_gate_dropl;
- cnt->passing_frames_count = cnt->matching_frames_count
- - cnt->not_passing_sdu_count
- - cnt->not_passing_frames_count;
+ cnt->passing_frames_count = cnt->matching_frames_count -
+ cnt->not_passing_sdu_count -
+ cnt->not_passing_frames_count;
- cnt->red_frames_count =
- ((u64)le32_to_cpu(data_buf->flow_meter_droph) << 32)
- + le32_to_cpu(data_buf->flow_meter_dropl);
+ cnt->red_frames_count = ((u64)data_buf->flow_meter_droph << 32) +
+ data_buf->flow_meter_dropl;
exit:
kfree(data_buf);
return -ENOMEM;
}
- cbd.addr[0] = lower_32_bits(dma);
- cbd.addr[1] = upper_32_bits(dma);
+ cbd.addr[0] = cpu_to_le32(lower_32_bits(dma));
+ cbd.addr[1] = cpu_to_le32(upper_32_bits(dma));
sgce = &sgcl_data->sgcl[0];
sgcl_config->agtst = 0x80;
- sgcl_data->ct = cpu_to_le32(sgi->cycletime);
- sgcl_data->cte = cpu_to_le32(sgi->cycletimext);
+ sgcl_data->ct = sgi->cycletime;
+ sgcl_data->cte = sgi->cycletimext;
if (sgi->init_ipv >= 0)
sgcl_config->aipv = (sgi->init_ipv & 0x7) | 0x8;
to->msdu[2] = (from->maxoctets >> 16) & 0xFF;
}
- to->interval = cpu_to_le32(from->interval);
+ to->interval = from->interval;
}
/* If basetime is less than now, calculate start time */
err = get_start_ns(now, sgi->cycletime, &start);
if (err)
goto exit;
- sgcl_data->btl = cpu_to_le32(lower_32_bits(start));
- sgcl_data->bth = cpu_to_le32(upper_32_bits(start));
+ sgcl_data->btl = lower_32_bits(start);
+ sgcl_data->bth = upper_32_bits(start);
} else {
u32 hi, lo;
hi = upper_32_bits(sgi->basetime);
lo = lower_32_bits(sgi->basetime);
- sgcl_data->bth = cpu_to_le32(hi);
- sgcl_data->btl = cpu_to_le32(lo);
+ sgcl_data->bth = hi;
+ sgcl_data->btl = lo;
}
err = enetc_send_cmd(priv->si, &cbd);
int ret = 0, frame_start, frame_addr, frame_op;
bool is_c45 = !!(regnum & MII_ADDR_C45);
- ret = pm_runtime_get_sync(dev);
+ ret = pm_runtime_resume_and_get(dev);
if (ret < 0)
return ret;
int ret, frame_start, frame_addr;
bool is_c45 = !!(regnum & MII_ADDR_C45);
- ret = pm_runtime_get_sync(dev);
+ ret = pm_runtime_resume_and_get(dev);
if (ret < 0)
return ret;
- else
- ret = 0;
if (is_c45) {
frame_start = FEC_MMFR_ST_C45;
u32 i, off;
int ret;
- ret = pm_runtime_get_sync(dev);
+ ret = pm_runtime_resume_and_get(dev);
if (ret < 0)
return;
int ret;
bool reset_again;
- ret = pm_runtime_get_sync(&fep->pdev->dev);
+ ret = pm_runtime_resume_and_get(&fep->pdev->dev);
if (ret < 0)
return ret;
struct device_node *np = pdev->dev.of_node;
int ret;
- ret = pm_runtime_get_sync(&pdev->dev);
+ ret = pm_runtime_resume_and_get(&pdev->dev);
if (ret < 0)
return ret;
addresses */
#define TX_TIMEOUT (1*HZ)
-#define SKB_ALLOC_TIMEOUT 100000
#define PHY_INIT_TIMEOUT 100000
#define PHY_CHANGE_TIME 2
* Copyright (C) 2015-2019 Google, Inc.
*/
+#include <linux/ethtool.h>
#include <linux/rtnetlink.h>
#include "gve.h"
#include "gve_adminq.h"
#include <linux/dcbnl.h>
#include <linux/delay.h>
#include <linux/device.h>
+#include <linux/ethtool.h>
#include <linux/module.h>
#include <linux/netdevice.h>
#include <linux/pci.h>
HNAE3_DEV_SUPPORT_FD_FORWARD_TC_B,
HNAE3_DEV_SUPPORT_PTP_B,
HNAE3_DEV_SUPPORT_INT_QL_B,
- HNAE3_DEV_SUPPORT_SIMPLE_BD_B,
+ HNAE3_DEV_SUPPORT_HW_TX_CSUM_B,
HNAE3_DEV_SUPPORT_TX_PUSH_B,
HNAE3_DEV_SUPPORT_PHY_IMP_B,
HNAE3_DEV_SUPPORT_TQP_TXRX_INDEP_B,
HNAE3_DEV_SUPPORT_HW_PAD_B,
HNAE3_DEV_SUPPORT_STASH_B,
+ HNAE3_DEV_SUPPORT_UDP_TUNNEL_CSUM_B,
};
#define hnae3_dev_fd_supported(hdev) \
#define hnae3_dev_int_ql_supported(hdev) \
test_bit(HNAE3_DEV_SUPPORT_INT_QL_B, (hdev)->ae_dev->caps)
-#define hnae3_dev_simple_bd_supported(hdev) \
- test_bit(HNAE3_DEV_SUPPORT_SIMPLE_BD_B, (hdev)->ae_dev->caps)
+#define hnae3_dev_hw_csum_supported(hdev) \
+ test_bit(HNAE3_DEV_SUPPORT_HW_TX_CSUM_B, (hdev)->ae_dev->caps)
#define hnae3_dev_tx_push_supported(hdev) \
test_bit(HNAE3_DEV_SUPPORT_TX_PUSH_B, (hdev)->ae_dev->caps)
u16 rss_ind_tbl_size;
u16 rss_key_size;
u16 int_ql_max; /* max value of interrupt coalesce based on INT_QL */
+ u16 max_int_gl; /* max value of interrupt coalesce based on INT_GL */
u8 max_non_tso_bd_num; /* max BD number of one non-TSO packet */
};
struct hnae3_roce_private_info {
struct net_device *netdev;
void __iomem *roce_io_base;
+ void __iomem *roce_mem_base;
int base_vector;
int num_vectors;
u32 tx_index, rx_index;
u32 q_num, value;
dma_addr_t addr;
+ u16 mss_hw_csum;
+ u32 l234info;
int cnt;
cnt = sscanf(&cmd_buf[8], "%u %u", &q_num, &tx_index);
tx_desc = &ring->desc[tx_index];
addr = le64_to_cpu(tx_desc->addr);
+ mss_hw_csum = le16_to_cpu(tx_desc->tx.mss_hw_csum);
dev_info(dev, "TX Queue Num: %u, BD Index: %u\n", q_num, tx_index);
dev_info(dev, "(TX)addr: %pad\n", &addr);
dev_info(dev, "(TX)vlan_tag: %u\n", le16_to_cpu(tx_desc->tx.vlan_tag));
dev_info(dev, "(TX)send_size: %u\n",
le16_to_cpu(tx_desc->tx.send_size));
- dev_info(dev, "(TX)vlan_tso: %u\n", tx_desc->tx.type_cs_vlan_tso);
- dev_info(dev, "(TX)l2_len: %u\n", tx_desc->tx.l2_len);
- dev_info(dev, "(TX)l3_len: %u\n", tx_desc->tx.l3_len);
- dev_info(dev, "(TX)l4_len: %u\n", tx_desc->tx.l4_len);
+
+ if (mss_hw_csum & BIT(HNS3_TXD_HW_CS_B)) {
+ u32 offset = le32_to_cpu(tx_desc->tx.ol_type_vlan_len_msec);
+ u32 start = le32_to_cpu(tx_desc->tx.type_cs_vlan_tso_len);
+
+ dev_info(dev, "(TX)csum start: %u\n",
+ hnae3_get_field(start,
+ HNS3_TXD_CSUM_START_M,
+ HNS3_TXD_CSUM_START_S));
+ dev_info(dev, "(TX)csum offset: %u\n",
+ hnae3_get_field(offset,
+ HNS3_TXD_CSUM_OFFSET_M,
+ HNS3_TXD_CSUM_OFFSET_S));
+ } else {
+ dev_info(dev, "(TX)vlan_tso: %u\n",
+ tx_desc->tx.type_cs_vlan_tso);
+ dev_info(dev, "(TX)l2_len: %u\n", tx_desc->tx.l2_len);
+ dev_info(dev, "(TX)l3_len: %u\n", tx_desc->tx.l3_len);
+ dev_info(dev, "(TX)l4_len: %u\n", tx_desc->tx.l4_len);
+ dev_info(dev, "(TX)vlan_msec: %u\n",
+ tx_desc->tx.ol_type_vlan_msec);
+ dev_info(dev, "(TX)ol2_len: %u\n", tx_desc->tx.ol2_len);
+ dev_info(dev, "(TX)ol3_len: %u\n", tx_desc->tx.ol3_len);
+ dev_info(dev, "(TX)ol4_len: %u\n", tx_desc->tx.ol4_len);
+ }
+
dev_info(dev, "(TX)vlan_tag: %u\n",
le16_to_cpu(tx_desc->tx.outer_vlan_tag));
dev_info(dev, "(TX)tv: %u\n", le16_to_cpu(tx_desc->tx.tv));
- dev_info(dev, "(TX)vlan_msec: %u\n", tx_desc->tx.ol_type_vlan_msec);
- dev_info(dev, "(TX)ol2_len: %u\n", tx_desc->tx.ol2_len);
- dev_info(dev, "(TX)ol3_len: %u\n", tx_desc->tx.ol3_len);
- dev_info(dev, "(TX)ol4_len: %u\n", tx_desc->tx.ol4_len);
- dev_info(dev, "(TX)paylen: %u\n", le32_to_cpu(tx_desc->tx.paylen));
+ dev_info(dev, "(TX)paylen_ol4cs: %u\n",
+ le32_to_cpu(tx_desc->tx.paylen_ol4cs));
dev_info(dev, "(TX)vld_ra_ri: %u\n",
le16_to_cpu(tx_desc->tx.bdtp_fe_sc_vld_ra_ri));
- dev_info(dev, "(TX)mss: %u\n", le16_to_cpu(tx_desc->tx.mss));
+ dev_info(dev, "(TX)mss_hw_csum: %u\n", mss_hw_csum);
ring = &priv->ring[q_num + h->kinfo.num_tqps];
value = readl_relaxed(ring->tqp->io_base + HNS3_RING_RX_RING_TAIL_REG);
rx_desc = &ring->desc[rx_index];
addr = le64_to_cpu(rx_desc->addr);
+ l234info = le32_to_cpu(rx_desc->rx.l234_info);
dev_info(dev, "RX Queue Num: %u, BD Index: %u\n", q_num, rx_index);
dev_info(dev, "(RX)addr: %pad\n", &addr);
- dev_info(dev, "(RX)l234_info: %u\n",
- le32_to_cpu(rx_desc->rx.l234_info));
+ dev_info(dev, "(RX)l234_info: %u\n", l234info);
+
+ if (l234info & BIT(HNS3_RXD_L2_CSUM_B)) {
+ u32 lo, hi;
+
+ lo = hnae3_get_field(l234info, HNS3_RXD_L2_CSUM_L_M,
+ HNS3_RXD_L2_CSUM_L_S);
+ hi = hnae3_get_field(l234info, HNS3_RXD_L2_CSUM_H_M,
+ HNS3_RXD_L2_CSUM_H_S);
+ dev_info(dev, "(RX)csum: %u\n", lo | hi << 8);
+ }
+
dev_info(dev, "(RX)pkt_len: %u\n", le16_to_cpu(rx_desc->rx.pkt_len));
dev_info(dev, "(RX)size: %u\n", le16_to_cpu(rx_desc->rx.size));
dev_info(dev, "(RX)rss_hash: %u\n", le32_to_cpu(rx_desc->rx.rss_hash));
test_bit(HNAE3_DEV_SUPPORT_PTP_B, caps) ? "yes" : "no");
dev_info(&h->pdev->dev, "support INT QL: %s\n",
test_bit(HNAE3_DEV_SUPPORT_INT_QL_B, caps) ? "yes" : "no");
+ dev_info(&h->pdev->dev, "support HW TX csum: %s\n",
+ test_bit(HNAE3_DEV_SUPPORT_HW_TX_CSUM_B, caps) ? "yes" : "no");
+ dev_info(&h->pdev->dev, "support UDP tunnel csum: %s\n",
+ test_bit(HNAE3_DEV_SUPPORT_UDP_TUNNEL_CSUM_B, caps) ?
+ "yes" : "no");
}
static void hns3_dbg_dev_specs(struct hnae3_handle *h)
dev_info(priv->dev, "Desc num per RX queue: %u\n", kinfo->num_rx_desc);
dev_info(priv->dev, "Total number of enabled TCs: %u\n", kinfo->num_tc);
dev_info(priv->dev, "MAX INT QL: %u\n", dev_specs->int_ql_max);
+ dev_info(priv->dev, "MAX INT GL: %u\n", dev_specs->max_int_gl);
}
static ssize_t hns3_dbg_cmd_read(struct file *filp, char __user *buffer,
* GL and RL(Rate Limiter) are 2 ways to acheive interrupt coalescing
*/
- if (rl_reg > 0 && !tqp_vector->tx_group.coal.gl_adapt_enable &&
- !tqp_vector->rx_group.coal.gl_adapt_enable)
+ if (rl_reg > 0 && !tqp_vector->tx_group.coal.adapt_enable &&
+ !tqp_vector->rx_group.coal.adapt_enable)
/* According to the hardware, the range of rl_reg is
* 0-59 and the unit is 4.
*/
void hns3_set_vector_coalesce_rx_gl(struct hns3_enet_tqp_vector *tqp_vector,
u32 gl_value)
{
- u32 rx_gl_reg = hns3_gl_usec_to_reg(gl_value);
+ u32 new_val;
- writel(rx_gl_reg, tqp_vector->mask_addr + HNS3_VECTOR_GL0_OFFSET);
+ if (tqp_vector->rx_group.coal.unit_1us)
+ new_val = gl_value | HNS3_INT_GL_1US;
+ else
+ new_val = hns3_gl_usec_to_reg(gl_value);
+
+ writel(new_val, tqp_vector->mask_addr + HNS3_VECTOR_GL0_OFFSET);
}
void hns3_set_vector_coalesce_tx_gl(struct hns3_enet_tqp_vector *tqp_vector,
u32 gl_value)
{
- u32 tx_gl_reg = hns3_gl_usec_to_reg(gl_value);
+ u32 new_val;
+
+ if (tqp_vector->tx_group.coal.unit_1us)
+ new_val = gl_value | HNS3_INT_GL_1US;
+ else
+ new_val = hns3_gl_usec_to_reg(gl_value);
+
+ writel(new_val, tqp_vector->mask_addr + HNS3_VECTOR_GL1_OFFSET);
+}
+
+void hns3_set_vector_coalesce_tx_ql(struct hns3_enet_tqp_vector *tqp_vector,
+ u32 ql_value)
+{
+ writel(ql_value, tqp_vector->mask_addr + HNS3_VECTOR_TX_QL_OFFSET);
+}
- writel(tx_gl_reg, tqp_vector->mask_addr + HNS3_VECTOR_GL1_OFFSET);
+void hns3_set_vector_coalesce_rx_ql(struct hns3_enet_tqp_vector *tqp_vector,
+ u32 ql_value)
+{
+ writel(ql_value, tqp_vector->mask_addr + HNS3_VECTOR_RX_QL_OFFSET);
}
-static void hns3_vector_gl_rl_init(struct hns3_enet_tqp_vector *tqp_vector,
- struct hns3_nic_priv *priv)
+static void hns3_vector_coalesce_init(struct hns3_enet_tqp_vector *tqp_vector,
+ struct hns3_nic_priv *priv)
{
+ struct hnae3_ae_dev *ae_dev = pci_get_drvdata(priv->ae_handle->pdev);
+ struct hns3_enet_coalesce *tx_coal = &tqp_vector->tx_group.coal;
+ struct hns3_enet_coalesce *rx_coal = &tqp_vector->rx_group.coal;
+
/* initialize the configuration for interrupt coalescing.
* 1. GL (Interrupt Gap Limiter)
* 2. RL (Interrupt Rate Limiter)
+ * 3. QL (Interrupt Quantity Limiter)
*
* Default: enable interrupt coalescing self-adaptive and GL
*/
- tqp_vector->tx_group.coal.gl_adapt_enable = 1;
- tqp_vector->rx_group.coal.gl_adapt_enable = 1;
+ tx_coal->adapt_enable = 1;
+ rx_coal->adapt_enable = 1;
+
+ tx_coal->int_gl = HNS3_INT_GL_50K;
+ rx_coal->int_gl = HNS3_INT_GL_50K;
- tqp_vector->tx_group.coal.int_gl = HNS3_INT_GL_50K;
- tqp_vector->rx_group.coal.int_gl = HNS3_INT_GL_50K;
+ rx_coal->flow_level = HNS3_FLOW_LOW;
+ tx_coal->flow_level = HNS3_FLOW_LOW;
- tqp_vector->rx_group.coal.flow_level = HNS3_FLOW_LOW;
- tqp_vector->tx_group.coal.flow_level = HNS3_FLOW_LOW;
+ /* device version above V3(include V3), GL can configure 1us
+ * unit, so uses 1us unit.
+ */
+ if (ae_dev->dev_version >= HNAE3_DEVICE_VERSION_V3) {
+ tx_coal->unit_1us = 1;
+ rx_coal->unit_1us = 1;
+ }
+
+ if (ae_dev->dev_specs.int_ql_max) {
+ tx_coal->ql_enable = 1;
+ rx_coal->ql_enable = 1;
+ tx_coal->int_ql_max = ae_dev->dev_specs.int_ql_max;
+ rx_coal->int_ql_max = ae_dev->dev_specs.int_ql_max;
+ tx_coal->int_ql = HNS3_INT_QL_DEFAULT_CFG;
+ rx_coal->int_ql = HNS3_INT_QL_DEFAULT_CFG;
+ }
}
-static void hns3_vector_gl_rl_init_hw(struct hns3_enet_tqp_vector *tqp_vector,
- struct hns3_nic_priv *priv)
+static void
+hns3_vector_coalesce_init_hw(struct hns3_enet_tqp_vector *tqp_vector,
+ struct hns3_nic_priv *priv)
{
+ struct hns3_enet_coalesce *tx_coal = &tqp_vector->tx_group.coal;
+ struct hns3_enet_coalesce *rx_coal = &tqp_vector->rx_group.coal;
struct hnae3_handle *h = priv->ae_handle;
- hns3_set_vector_coalesce_tx_gl(tqp_vector,
- tqp_vector->tx_group.coal.int_gl);
- hns3_set_vector_coalesce_rx_gl(tqp_vector,
- tqp_vector->rx_group.coal.int_gl);
+ hns3_set_vector_coalesce_tx_gl(tqp_vector, tx_coal->int_gl);
+ hns3_set_vector_coalesce_rx_gl(tqp_vector, rx_coal->int_gl);
hns3_set_vector_coalesce_rl(tqp_vector, h->kinfo.int_rl_setting);
+
+ if (tx_coal->ql_enable)
+ hns3_set_vector_coalesce_tx_ql(tqp_vector, tx_coal->int_ql);
+
+ if (rx_coal->ql_enable)
+ hns3_set_vector_coalesce_rx_ql(tqp_vector, rx_coal->int_ql);
}
static int hns3_nic_set_real_num_queue(struct net_device *netdev)
}
}
-static int hns3_set_tso(struct sk_buff *skb, u32 *paylen,
+static int hns3_set_tso(struct sk_buff *skb, u32 *paylen_fdop_ol4cs,
u16 *mss, u32 *type_cs_vlan_tso)
{
u32 l4_offset, hdr_len;
SKB_GSO_GRE_CSUM |
SKB_GSO_UDP_TUNNEL |
SKB_GSO_UDP_TUNNEL_CSUM)) {
- if ((!(skb_shinfo(skb)->gso_type &
- SKB_GSO_PARTIAL)) &&
- (skb_shinfo(skb)->gso_type &
- SKB_GSO_UDP_TUNNEL_CSUM)) {
- /* Software should clear the udp's checksum
- * field when tso is needed.
- */
- l4.udp->check = 0;
- }
/* reset l3&l4 pointers from outer to inner headers */
l3.hdr = skb_inner_network_header(skb);
l4.hdr = skb_inner_transport_header(skb);
}
/* find the txbd field values */
- *paylen = skb->len - hdr_len;
+ *paylen_fdop_ol4cs = skb->len - hdr_len;
hns3_set_field(*type_cs_vlan_tso, HNS3_TXD_TSO_B, 1);
+ /* offload outer UDP header checksum */
+ if (skb_shinfo(skb)->gso_type & SKB_GSO_UDP_TUNNEL_CSUM)
+ hns3_set_field(*paylen_fdop_ol4cs, HNS3_TXD_OL4CS_B, 1);
+
/* get MSS for TSO */
*mss = skb_shinfo(skb)->gso_size;
*/
static bool hns3_tunnel_csum_bug(struct sk_buff *skb)
{
+ struct hns3_nic_priv *priv = netdev_priv(skb->dev);
+ struct hnae3_ae_dev *ae_dev = pci_get_drvdata(priv->ae_handle->pdev);
union l4_hdr_info l4;
+ /* device version above V3(include V3), the hardware can
+ * do this checksum offload.
+ */
+ if (ae_dev->dev_version >= HNAE3_DEVICE_VERSION_V3)
+ return false;
+
l4.hdr = skb_transport_header(skb);
if (!(!skb->encapsulation &&
return 0;
}
+/* check if the hardware is capable of checksum offloading */
+static bool hns3_check_hw_tx_csum(struct sk_buff *skb)
+{
+ struct hns3_nic_priv *priv = netdev_priv(skb->dev);
+
+ /* Kindly note, due to backward compatibility of the TX descriptor,
+ * HW checksum of the non-IP packets and GSO packets is handled at
+ * different place in the following code
+ */
+ if (skb->csum_not_inet || skb_is_gso(skb) ||
+ !test_bit(HNS3_NIC_STATE_HW_TX_CSUM_ENABLE, &priv->state))
+ return false;
+
+ return true;
+}
+
static int hns3_fill_skb_desc(struct hns3_enet_ring *ring,
struct sk_buff *skb, struct hns3_desc *desc)
{
u32 ol_type_vlan_len_msec = 0;
+ u32 paylen_ol4cs = skb->len;
u32 type_cs_vlan_tso = 0;
- u32 paylen = skb->len;
+ u16 mss_hw_csum = 0;
u16 inner_vtag = 0;
u16 out_vtag = 0;
- u16 mss = 0;
int ret;
ret = hns3_handle_vtags(ring, skb);
if (skb->ip_summed == CHECKSUM_PARTIAL) {
u8 ol4_proto, il4_proto;
+ if (hns3_check_hw_tx_csum(skb)) {
+ /* set checksum start and offset, defined in 2 Bytes */
+ hns3_set_field(type_cs_vlan_tso, HNS3_TXD_CSUM_START_S,
+ skb_checksum_start_offset(skb) >> 1);
+ hns3_set_field(ol_type_vlan_len_msec,
+ HNS3_TXD_CSUM_OFFSET_S,
+ skb->csum_offset >> 1);
+ mss_hw_csum |= BIT(HNS3_TXD_HW_CS_B);
+ goto out_hw_tx_csum;
+ }
+
skb_reset_mac_len(skb);
ret = hns3_get_l4_protocol(skb, &ol4_proto, &il4_proto);
return ret;
}
- ret = hns3_set_tso(skb, &paylen, &mss,
+ ret = hns3_set_tso(skb, &paylen_ol4cs, &mss_hw_csum,
&type_cs_vlan_tso);
if (unlikely(ret < 0)) {
u64_stats_update_begin(&ring->syncp);
}
}
+out_hw_tx_csum:
/* Set txbd */
desc->tx.ol_type_vlan_len_msec =
cpu_to_le32(ol_type_vlan_len_msec);
desc->tx.type_cs_vlan_tso_len = cpu_to_le32(type_cs_vlan_tso);
- desc->tx.paylen = cpu_to_le32(paylen);
- desc->tx.mss = cpu_to_le16(mss);
+ desc->tx.paylen_ol4cs = cpu_to_le32(paylen_ol4cs);
+ desc->tx.mss_hw_csum = cpu_to_le16(mss_hw_csum);
desc->tx.vlan_tag = cpu_to_le16(inner_vtag);
desc->tx.outer_vlan_tag = cpu_to_le16(out_vtag);
netdev->priv_flags |= IFF_UNICAST_FLT;
- netdev->hw_enc_features |= NETIF_F_IP_CSUM | NETIF_F_IPV6_CSUM |
- NETIF_F_RXCSUM | NETIF_F_SG | NETIF_F_GSO |
+ netdev->hw_enc_features |= NETIF_F_RXCSUM | NETIF_F_SG | NETIF_F_GSO |
NETIF_F_GRO | NETIF_F_TSO | NETIF_F_TSO6 | NETIF_F_GSO_GRE |
NETIF_F_GSO_GRE_CSUM | NETIF_F_GSO_UDP_TUNNEL |
- NETIF_F_GSO_UDP_TUNNEL_CSUM | NETIF_F_SCTP_CRC |
- NETIF_F_TSO_MANGLEID | NETIF_F_FRAGLIST;
+ NETIF_F_SCTP_CRC | NETIF_F_TSO_MANGLEID | NETIF_F_FRAGLIST;
netdev->gso_partial_features |= NETIF_F_GSO_GRE_CSUM;
- netdev->features |= NETIF_F_IP_CSUM | NETIF_F_IPV6_CSUM |
- NETIF_F_HW_VLAN_CTAG_FILTER |
+ netdev->features |= NETIF_F_HW_VLAN_CTAG_FILTER |
NETIF_F_HW_VLAN_CTAG_TX | NETIF_F_HW_VLAN_CTAG_RX |
NETIF_F_RXCSUM | NETIF_F_SG | NETIF_F_GSO |
NETIF_F_GRO | NETIF_F_TSO | NETIF_F_TSO6 | NETIF_F_GSO_GRE |
NETIF_F_GSO_GRE_CSUM | NETIF_F_GSO_UDP_TUNNEL |
- NETIF_F_GSO_UDP_TUNNEL_CSUM | NETIF_F_SCTP_CRC |
- NETIF_F_FRAGLIST;
+ NETIF_F_SCTP_CRC | NETIF_F_FRAGLIST;
- netdev->vlan_features |=
- NETIF_F_IP_CSUM | NETIF_F_IPV6_CSUM | NETIF_F_RXCSUM |
+ netdev->vlan_features |= NETIF_F_RXCSUM |
NETIF_F_SG | NETIF_F_GSO | NETIF_F_GRO |
NETIF_F_TSO | NETIF_F_TSO6 | NETIF_F_GSO_GRE |
NETIF_F_GSO_GRE_CSUM | NETIF_F_GSO_UDP_TUNNEL |
- NETIF_F_GSO_UDP_TUNNEL_CSUM | NETIF_F_SCTP_CRC |
- NETIF_F_FRAGLIST;
+ NETIF_F_SCTP_CRC | NETIF_F_FRAGLIST;
- netdev->hw_features |= NETIF_F_IP_CSUM | NETIF_F_IPV6_CSUM |
- NETIF_F_HW_VLAN_CTAG_TX | NETIF_F_HW_VLAN_CTAG_RX |
+ netdev->hw_features |= NETIF_F_HW_VLAN_CTAG_TX |
+ NETIF_F_HW_VLAN_CTAG_RX |
NETIF_F_RXCSUM | NETIF_F_SG | NETIF_F_GSO |
NETIF_F_GRO | NETIF_F_TSO | NETIF_F_TSO6 | NETIF_F_GSO_GRE |
NETIF_F_GSO_GRE_CSUM | NETIF_F_GSO_UDP_TUNNEL |
- NETIF_F_GSO_UDP_TUNNEL_CSUM | NETIF_F_SCTP_CRC |
- NETIF_F_FRAGLIST;
+ NETIF_F_SCTP_CRC | NETIF_F_FRAGLIST;
if (ae_dev->dev_version >= HNAE3_DEVICE_VERSION_V2) {
netdev->hw_features |= NETIF_F_GRO_HW;
netdev->vlan_features |= NETIF_F_GSO_UDP_L4;
netdev->hw_enc_features |= NETIF_F_GSO_UDP_L4;
}
+
+ if (test_bit(HNAE3_DEV_SUPPORT_HW_TX_CSUM_B, ae_dev->caps)) {
+ netdev->hw_features |= NETIF_F_HW_CSUM;
+ netdev->features |= NETIF_F_HW_CSUM;
+ netdev->vlan_features |= NETIF_F_HW_CSUM;
+ netdev->hw_enc_features |= NETIF_F_HW_CSUM;
+ } else {
+ netdev->hw_features |= NETIF_F_IP_CSUM | NETIF_F_IPV6_CSUM;
+ netdev->features |= NETIF_F_IP_CSUM | NETIF_F_IPV6_CSUM;
+ netdev->vlan_features |= NETIF_F_IP_CSUM | NETIF_F_IPV6_CSUM;
+ netdev->hw_enc_features |= NETIF_F_IP_CSUM | NETIF_F_IPV6_CSUM;
+ }
+
+ if (test_bit(HNAE3_DEV_SUPPORT_UDP_TUNNEL_CSUM_B, ae_dev->caps)) {
+ netdev->hw_features |= NETIF_F_GSO_UDP_TUNNEL_CSUM;
+ netdev->features |= NETIF_F_GSO_UDP_TUNNEL_CSUM;
+ netdev->vlan_features |= NETIF_F_GSO_UDP_TUNNEL_CSUM;
+ netdev->hw_enc_features |= NETIF_F_GSO_UDP_TUNNEL_CSUM;
+ }
}
static int hns3_alloc_buffer(struct hns3_enet_ring *ring,
return 0;
}
+static void hns3_checksum_complete(struct hns3_enet_ring *ring,
+ struct sk_buff *skb, u32 l234info)
+{
+ u32 lo, hi;
+
+ u64_stats_update_begin(&ring->syncp);
+ ring->stats.csum_complete++;
+ u64_stats_update_end(&ring->syncp);
+ skb->ip_summed = CHECKSUM_COMPLETE;
+ lo = hnae3_get_field(l234info, HNS3_RXD_L2_CSUM_L_M,
+ HNS3_RXD_L2_CSUM_L_S);
+ hi = hnae3_get_field(l234info, HNS3_RXD_L2_CSUM_H_M,
+ HNS3_RXD_L2_CSUM_H_S);
+ skb->csum = csum_unfold((__force __sum16)(lo | hi << 8));
+}
+
static void hns3_rx_checksum(struct hns3_enet_ring *ring, struct sk_buff *skb,
u32 l234info, u32 bd_base_info, u32 ol_info)
{
if (!(netdev->features & NETIF_F_RXCSUM))
return;
+ if (l234info & BIT(HNS3_RXD_L2_CSUM_B)) {
+ hns3_checksum_complete(ring, skb, l234info);
+ return;
+ }
+
/* check if hardware has done checksum */
if (!(bd_base_info & BIT(HNS3_RXD_L3L4P_B)))
return;
tqp_vector->last_jiffies + msecs_to_jiffies(1000)))
return;
- if (rx_group->coal.gl_adapt_enable) {
+ if (rx_group->coal.adapt_enable) {
rx_update = hns3_get_new_int_gl(rx_group);
if (rx_update)
hns3_set_vector_coalesce_rx_gl(tqp_vector,
rx_group->coal.int_gl);
}
- if (tx_group->coal.gl_adapt_enable) {
+ if (tx_group->coal.adapt_enable) {
tx_update = hns3_get_new_int_gl(tx_group);
if (tx_update)
hns3_set_vector_coalesce_tx_gl(tqp_vector,
for (i = 0; i < priv->vector_num; i++) {
tqp_vector = &priv->tqp_vector[i];
- hns3_vector_gl_rl_init_hw(tqp_vector, priv);
+ hns3_vector_coalesce_init_hw(tqp_vector, priv);
tqp_vector->num_tqps = 0;
}
static int hns3_nic_alloc_vector_data(struct hns3_nic_priv *priv)
{
-#define HNS3_VECTOR_PF_MAX_NUM 64
-
struct hnae3_handle *h = priv->ae_handle;
struct hns3_enet_tqp_vector *tqp_vector;
struct hnae3_vector_info *vector;
/* RSS size, cpu online and vector_num should be the same */
/* Should consider 2p/4p later */
vector_num = min_t(u16, num_online_cpus(), tqp_num);
- vector_num = min_t(u16, vector_num, HNS3_VECTOR_PF_MAX_NUM);
vector = devm_kcalloc(&pdev->dev, vector_num, sizeof(*vector),
GFP_KERNEL);
tqp_vector->idx = i;
tqp_vector->mask_addr = vector[i].io_addr;
tqp_vector->vector_irq = vector[i].vector;
- hns3_vector_gl_rl_init(tqp_vector, priv);
+ hns3_vector_coalesce_init(tqp_vector, priv);
}
out:
/* MTU range: (ETH_MIN_MTU(kernel default) - 9702) */
netdev->max_mtu = HNS3_MAX_MTU;
+ if (test_bit(HNAE3_DEV_SUPPORT_HW_TX_CSUM_B, ae_dev->caps))
+ set_bit(HNS3_NIC_STATE_HW_TX_CSUM_ENABLE, &priv->state);
+
set_bit(HNS3_NIC_STATE_INITED, &priv->state);
if (netif_msg_drv(handle))
HNS3_NIC_STATE_SERVICE_INITED,
HNS3_NIC_STATE_SERVICE_SCHED,
HNS3_NIC_STATE2_RESET_REQUESTED,
+ HNS3_NIC_STATE_HW_TX_CSUM_ENABLE,
HNS3_NIC_STATE_MAX
};
#define HNS3_RXD_STRP_TAGP_S 13
#define HNS3_RXD_STRP_TAGP_M (0x3 << HNS3_RXD_STRP_TAGP_S)
+#define HNS3_RXD_L2_CSUM_B 15
+#define HNS3_RXD_L2_CSUM_L_S 4
+#define HNS3_RXD_L2_CSUM_L_M (0xff << HNS3_RXD_L2_CSUM_L_S)
+#define HNS3_RXD_L2_CSUM_H_S 24
+#define HNS3_RXD_L2_CSUM_H_M (0xff << HNS3_RXD_L2_CSUM_H_S)
+
#define HNS3_RXD_L2E_B 16
#define HNS3_RXD_L3E_B 17
#define HNS3_RXD_L4E_B 18
#define HNS3_TXD_L4LEN_S 24
#define HNS3_TXD_L4LEN_M (0xff << HNS3_TXD_L4LEN_S)
+#define HNS3_TXD_CSUM_START_S 8
+#define HNS3_TXD_CSUM_START_M (0xffff << HNS3_TXD_CSUM_START_S)
+
#define HNS3_TXD_OL3T_S 0
#define HNS3_TXD_OL3T_M (0x3 << HNS3_TXD_OL3T_S)
#define HNS3_TXD_OVLAN_B 2
#define HNS3_TXD_TUNTYPE_S 4
#define HNS3_TXD_TUNTYPE_M (0xf << HNS3_TXD_TUNTYPE_S)
+#define HNS3_TXD_CSUM_OFFSET_S 8
+#define HNS3_TXD_CSUM_OFFSET_M (0xffff << HNS3_TXD_CSUM_OFFSET_S)
+
#define HNS3_TXD_BDTYPE_S 0
#define HNS3_TXD_BDTYPE_M (0xf << HNS3_TXD_BDTYPE_S)
#define HNS3_TXD_FE_B 4
#define HNS3_TXD_DECTTL_S 12
#define HNS3_TXD_DECTTL_M (0xf << HNS3_TXD_DECTTL_S)
+#define HNS3_TXD_OL4CS_B 22
+
#define HNS3_TXD_MSS_S 0
#define HNS3_TXD_MSS_M (0x3fff << HNS3_TXD_MSS_S)
+#define HNS3_TXD_HW_CS_B 14
#define HNS3_VECTOR_TX_IRQ BIT_ULL(0)
#define HNS3_VECTOR_RX_IRQ BIT_ULL(1)
#define HNS3_VECTOR_GL2_OFFSET 0x300
#define HNS3_VECTOR_RL_OFFSET 0x900
#define HNS3_VECTOR_RL_EN_B 6
+#define HNS3_VECTOR_TX_QL_OFFSET 0xe00
+#define HNS3_VECTOR_RX_QL_OFFSET 0xf00
#define HNS3_RING_EN_B 0
};
};
- __le32 paylen;
+ __le32 paylen_ol4cs;
__le16 bdtp_fe_sc_vld_ra_ri;
- __le16 mss;
+ __le16 mss_hw_csum;
} tx;
struct {
u64 err_bd_num;
u64 l2_err;
u64 l3l4_csum_err;
+ u64 csum_complete;
u64 rx_multicast;
u64 non_reuse_pg;
};
HNS3_FLOW_ULTRA = 3,
};
-#define HNS3_INT_GL_MAX 0x1FE0
#define HNS3_INT_GL_50K 0x0014
#define HNS3_INT_GL_20K 0x0032
#define HNS3_INT_GL_18K 0x0036
#define HNS3_INT_GL_8K 0x007C
+#define HNS3_INT_GL_1US BIT(31)
+
#define HNS3_INT_RL_MAX 0x00EC
#define HNS3_INT_RL_ENABLE_MASK 0x40
+#define HNS3_INT_QL_DEFAULT_CFG 0x20
+
struct hns3_enet_coalesce {
u16 int_gl;
- u8 gl_adapt_enable;
+ u16 int_ql;
+ u16 int_ql_max;
+ u8 adapt_enable:1;
+ u8 ql_enable:1;
+ u8 unit_1us:1;
enum hns3_flow_level_range flow_level;
};
u32 gl_value);
void hns3_set_vector_coalesce_rl(struct hns3_enet_tqp_vector *tqp_vector,
u32 rl_value);
+void hns3_set_vector_coalesce_rx_ql(struct hns3_enet_tqp_vector *tqp_vector,
+ u32 ql_value);
+void hns3_set_vector_coalesce_tx_ql(struct hns3_enet_tqp_vector *tqp_vector,
+ u32 ql_value);
void hns3_enable_vlan_filter(struct net_device *netdev, bool enable);
void hns3_request_update_promisc_mode(struct hnae3_handle *handle);
HNS3_TQP_STAT("err_bd_num", err_bd_num),
HNS3_TQP_STAT("l2_err", l2_err),
HNS3_TQP_STAT("l3l4_csum_err", l3l4_csum_err),
+ HNS3_TQP_STAT("csum_complete", csum_complete),
HNS3_TQP_STAT("multicast", rx_multicast),
HNS3_TQP_STAT("non_reuse_pg", non_reuse_pg),
};
rx_vector = priv->ring[queue_num + queue].tqp_vector;
cmd->use_adaptive_tx_coalesce =
- tx_vector->tx_group.coal.gl_adapt_enable;
+ tx_vector->tx_group.coal.adapt_enable;
cmd->use_adaptive_rx_coalesce =
- rx_vector->rx_group.coal.gl_adapt_enable;
+ rx_vector->rx_group.coal.adapt_enable;
cmd->tx_coalesce_usecs = tx_vector->tx_group.coal.int_gl;
cmd->rx_coalesce_usecs = rx_vector->rx_group.coal.int_gl;
cmd->tx_coalesce_usecs_high = h->kinfo.int_rl_setting;
cmd->rx_coalesce_usecs_high = h->kinfo.int_rl_setting;
+ cmd->tx_max_coalesced_frames = tx_vector->tx_group.coal.int_ql;
+ cmd->rx_max_coalesced_frames = rx_vector->rx_group.coal.int_ql;
+
return 0;
}
static int hns3_check_gl_coalesce_para(struct net_device *netdev,
struct ethtool_coalesce *cmd)
{
+ struct hnae3_handle *handle = hns3_get_handle(netdev);
+ struct hnae3_ae_dev *ae_dev = pci_get_drvdata(handle->pdev);
u32 rx_gl, tx_gl;
- if (cmd->rx_coalesce_usecs > HNS3_INT_GL_MAX) {
+ if (cmd->rx_coalesce_usecs > ae_dev->dev_specs.max_int_gl) {
netdev_err(netdev,
- "Invalid rx-usecs value, rx-usecs range is 0-%d\n",
- HNS3_INT_GL_MAX);
+ "invalid rx-usecs value, rx-usecs range is 0-%u\n",
+ ae_dev->dev_specs.max_int_gl);
return -EINVAL;
}
- if (cmd->tx_coalesce_usecs > HNS3_INT_GL_MAX) {
+ if (cmd->tx_coalesce_usecs > ae_dev->dev_specs.max_int_gl) {
netdev_err(netdev,
- "Invalid tx-usecs value, tx-usecs range is 0-%d\n",
- HNS3_INT_GL_MAX);
+ "invalid tx-usecs value, tx-usecs range is 0-%u\n",
+ ae_dev->dev_specs.max_int_gl);
return -EINVAL;
}
+ /* device version above V3(include V3), GL uses 1us unit,
+ * so the round down is not needed.
+ */
+ if (ae_dev->dev_version >= HNAE3_DEVICE_VERSION_V3)
+ return 0;
+
rx_gl = hns3_gl_round_down(cmd->rx_coalesce_usecs);
if (rx_gl != cmd->rx_coalesce_usecs) {
netdev_info(netdev,
return 0;
}
+static int hns3_check_ql_coalesce_param(struct net_device *netdev,
+ struct ethtool_coalesce *cmd)
+{
+ struct hnae3_handle *handle = hns3_get_handle(netdev);
+ struct hnae3_ae_dev *ae_dev = pci_get_drvdata(handle->pdev);
+
+ if ((cmd->tx_max_coalesced_frames || cmd->rx_max_coalesced_frames) &&
+ !ae_dev->dev_specs.int_ql_max) {
+ netdev_err(netdev, "coalesced frames is not supported\n");
+ return -EOPNOTSUPP;
+ }
+
+ if (cmd->tx_max_coalesced_frames > ae_dev->dev_specs.int_ql_max ||
+ cmd->rx_max_coalesced_frames > ae_dev->dev_specs.int_ql_max) {
+ netdev_err(netdev,
+ "invalid coalesced_frames value, range is 0-%u\n",
+ ae_dev->dev_specs.int_ql_max);
+ return -ERANGE;
+ }
+
+ return 0;
+}
+
static int hns3_check_coalesce_para(struct net_device *netdev,
struct ethtool_coalesce *cmd)
{
return ret;
}
+ ret = hns3_check_ql_coalesce_param(netdev, cmd);
+ if (ret)
+ return ret;
+
if (cmd->use_adaptive_tx_coalesce == 1 ||
cmd->use_adaptive_rx_coalesce == 1) {
netdev_info(netdev,
tx_vector = priv->ring[queue].tqp_vector;
rx_vector = priv->ring[queue_num + queue].tqp_vector;
- tx_vector->tx_group.coal.gl_adapt_enable =
+ tx_vector->tx_group.coal.adapt_enable =
cmd->use_adaptive_tx_coalesce;
- rx_vector->rx_group.coal.gl_adapt_enable =
+ rx_vector->rx_group.coal.adapt_enable =
cmd->use_adaptive_rx_coalesce;
tx_vector->tx_group.coal.int_gl = cmd->tx_coalesce_usecs;
rx_vector->rx_group.coal.int_gl = cmd->rx_coalesce_usecs;
+ tx_vector->tx_group.coal.int_ql = cmd->tx_max_coalesced_frames;
+ rx_vector->rx_group.coal.int_ql = cmd->rx_max_coalesced_frames;
+
hns3_set_vector_coalesce_tx_gl(tx_vector,
tx_vector->tx_group.coal.int_gl);
hns3_set_vector_coalesce_rx_gl(rx_vector,
hns3_set_vector_coalesce_rl(tx_vector, h->kinfo.int_rl_setting);
hns3_set_vector_coalesce_rl(rx_vector, h->kinfo.int_rl_setting);
+
+ if (tx_vector->tx_group.coal.ql_enable)
+ hns3_set_vector_coalesce_tx_ql(tx_vector,
+ tx_vector->tx_group.coal.int_ql);
+ if (rx_vector->rx_group.coal.ql_enable)
+ hns3_set_vector_coalesce_rx_ql(rx_vector,
+ rx_vector->rx_group.coal.int_ql);
}
static int hns3_set_coalesce(struct net_device *netdev,
#define HNS3_ETHTOOL_COALESCE (ETHTOOL_COALESCE_USECS | \
ETHTOOL_COALESCE_USE_ADAPTIVE | \
ETHTOOL_COALESCE_RX_USECS_HIGH | \
- ETHTOOL_COALESCE_TX_USECS_HIGH)
+ ETHTOOL_COALESCE_TX_USECS_HIGH | \
+ ETHTOOL_COALESCE_MAX_FRAMES)
static const struct ethtool_ops hns3vf_ethtool_ops = {
.supported_coalesce_params = HNS3_ETHTOOL_COALESCE,
set_bit(HNAE3_DEV_SUPPORT_INT_QL_B, ae_dev->caps);
if (hnae3_get_bit(caps, HCLGE_CAP_TQP_TXRX_INDEP_B))
set_bit(HNAE3_DEV_SUPPORT_TQP_TXRX_INDEP_B, ae_dev->caps);
+ if (hnae3_get_bit(caps, HCLGE_CAP_HW_TX_CSUM_B))
+ set_bit(HNAE3_DEV_SUPPORT_HW_TX_CSUM_B, ae_dev->caps);
+ if (hnae3_get_bit(caps, HCLGE_CAP_UDP_TUNNEL_CSUM_B))
+ set_bit(HNAE3_DEV_SUPPORT_UDP_TUNNEL_CSUM_B, ae_dev->caps);
}
static enum hclge_cmd_status
#define HCLGE_TQP_REG_OFFSET 0x80000
#define HCLGE_TQP_REG_SIZE 0x200
+#define HCLGE_TQP_MAX_SIZE_DEV_V2 1024
+#define HCLGE_TQP_EXT_REG_OFFSET 0x100
+
#define HCLGE_RCB_INIT_QUERY_TIMEOUT 10
#define HCLGE_RCB_INIT_FLAG_EN_B 0
#define HCLGE_RCB_INIT_FLAG_FINI_B 8
};
struct hclge_ctrl_vector_chain_cmd {
- u8 int_vector_id;
+#define HCLGE_VECTOR_ID_L_S 0
+#define HCLGE_VECTOR_ID_L_M GENMASK(7, 0)
+ u8 int_vector_id_l;
u8 int_cause_num;
#define HCLGE_INT_TYPE_S 0
#define HCLGE_INT_TYPE_M GENMASK(1, 0)
#define HCLGE_INT_GL_IDX_M GENMASK(14, 13)
__le16 tqp_type_and_id[HCLGE_VECTOR_ELEMENTS_PER_CMD];
u8 vfid;
- u8 rsv;
+#define HCLGE_VECTOR_ID_H_S 8
+#define HCLGE_VECTOR_ID_H_M GENMASK(15, 8)
+ u8 int_vector_id_h;
};
#define HCLGE_MAX_TC_NUM 8
HCLGE_CAP_FD_FORWARD_TC_B,
HCLGE_CAP_PTP_B,
HCLGE_CAP_INT_QL_B,
- HCLGE_CAP_SIMPLE_BD_B,
+ HCLGE_CAP_HW_TX_CSUM_B,
HCLGE_CAP_TX_PUSH_B,
HCLGE_CAP_PHY_IMP_B,
HCLGE_CAP_TQP_TXRX_INDEP_B,
HCLGE_CAP_HW_PAD_B,
HCLGE_CAP_STASH_B,
+ HCLGE_CAP_UDP_TUNNEL_CSUM_B,
};
#define HCLGE_QUERY_CAP_LENGTH 3
__le16 tqp_num;
__le16 buf_size;
__le16 msixcap_localid_ba_nic;
- __le16 msixcap_localid_ba_rocee;
-#define HCLGE_MSIX_OFT_ROCEE_S 0
-#define HCLGE_MSIX_OFT_ROCEE_M GENMASK(15, 0)
-#define HCLGE_PF_VEC_NUM_S 0
-#define HCLGE_PF_VEC_NUM_M GENMASK(7, 0)
- __le16 pf_intr_vector_number;
+ __le16 msixcap_localid_number_nic;
+ __le16 pf_intr_vector_number_roce;
__le16 pf_own_fun_number;
__le16 tx_buf_size;
__le16 dv_buf_size;
- __le32 rsv[2];
+ __le16 ext_tqp_num;
+ u8 rsv[6];
};
#define HCLGE_CFG_OFFSET_S 0
u8 rsv[22];
};
-#define HCLGE_RING_ID_MASK GENMASK(9, 0)
#define HCLGE_TQP_ENABLE_B 0
#define HCLGE_MAC_CFG_AN_EN_B 0
__le32 max_tm_rate;
};
+#define HCLGE_DEF_MAX_INT_GL 0x1FE0U
+
+struct hclge_dev_specs_1_cmd {
+ __le32 rsv0;
+ __le16 max_int_gl;
+ u8 rsv1[18];
+};
+
int hclge_cmd_init(struct hclge_dev *hdev);
static inline void hclge_write_reg(void __iomem *base, u32 reg, u32 value)
{
dev_info(&hdev->pdev->dev, "PG_P pg_id: %u\n", pg_shap_cfg_cmd->pg_id);
dev_info(&hdev->pdev->dev, "PG_P pg_shapping: 0x%x\n",
le32_to_cpu(pg_shap_cfg_cmd->pg_shapping_para));
+ dev_info(&hdev->pdev->dev, "PG_P flag: %#x\n", pg_shap_cfg_cmd->flag);
+ dev_info(&hdev->pdev->dev, "PG_P pg_rate: %u(Mbps)\n",
+ le32_to_cpu(pg_shap_cfg_cmd->pg_rate));
cmd = HCLGE_OPC_TM_PORT_SHAPPING;
hclge_cmd_setup_basic_desc(&desc, cmd, true);
port_shap_cfg_cmd = (struct hclge_port_shapping_cmd *)desc.data;
dev_info(&hdev->pdev->dev, "PORT port_shapping: 0x%x\n",
le32_to_cpu(port_shap_cfg_cmd->port_shapping_para));
+ dev_info(&hdev->pdev->dev, "PORT flag: %#x\n", port_shap_cfg_cmd->flag);
+ dev_info(&hdev->pdev->dev, "PORT port_rate: %u(Mbps)\n",
+ le32_to_cpu(port_shap_cfg_cmd->port_rate));
cmd = HCLGE_OPC_TM_PG_SCH_MODE_CFG;
hclge_cmd_setup_basic_desc(&desc, cmd, true);
dev_info(&hdev->pdev->dev, "PRI_C pri_id: %u\n", shap_cfg_cmd->pri_id);
dev_info(&hdev->pdev->dev, "PRI_C pri_shapping: 0x%x\n",
le32_to_cpu(shap_cfg_cmd->pri_shapping_para));
+ dev_info(&hdev->pdev->dev, "PRI_C flag: %#x\n", shap_cfg_cmd->flag);
+ dev_info(&hdev->pdev->dev, "PRI_C pri_rate: %u(Mbps)\n",
+ le32_to_cpu(shap_cfg_cmd->pri_rate));
cmd = HCLGE_OPC_TM_PRI_P_SHAPPING;
hclge_cmd_setup_basic_desc(&desc, cmd, true);
dev_info(&hdev->pdev->dev, "PRI_P pri_id: %u\n", shap_cfg_cmd->pri_id);
dev_info(&hdev->pdev->dev, "PRI_P pri_shapping: 0x%x\n",
le32_to_cpu(shap_cfg_cmd->pri_shapping_para));
+ dev_info(&hdev->pdev->dev, "PRI_P flag: %#x\n", shap_cfg_cmd->flag);
+ dev_info(&hdev->pdev->dev, "PRI_P pri_rate: %u(Mbps)\n",
+ le32_to_cpu(shap_cfg_cmd->pri_rate));
hclge_dbg_dump_tm_pg(hdev);
{
struct hclge_bp_to_qs_map_cmd *bp_to_qs_map_cmd;
struct hclge_nq_to_qs_link_cmd *nq_to_qs_map;
+ u32 qset_mapping[HCLGE_BP_EXT_GRP_NUM];
struct hclge_qs_to_pri_link_cmd *map;
struct hclge_tqp_tx_queue_tc_cmd *tc;
enum hclge_opcode_type cmd;
struct hclge_desc desc;
int queue_id, group_id;
- u32 qset_mapping[32];
int tc_id, qset_id;
int pri_id, ret;
+ u16 qs_id_l;
+ u16 qs_id_h;
+ u8 grp_num;
u32 i;
ret = kstrtouint(cmd_buf, 0, &queue_id);
ret = hclge_cmd_send(&hdev->hw, &desc, 1);
if (ret)
goto err_tm_map_cmd_send;
- qset_id = le16_to_cpu(nq_to_qs_map->qset_id) & 0x3FF;
+ qset_id = le16_to_cpu(nq_to_qs_map->qset_id);
+
+ /* convert qset_id to the following format, drop the vld bit
+ * | qs_id_h | vld | qs_id_l |
+ * qset_id: | 15 ~ 11 | 10 | 9 ~ 0 |
+ * \ \ / /
+ * \ \ / /
+ * qset_id: | 15 | 14 ~ 10 | 9 ~ 0 |
+ */
+ qs_id_l = hnae3_get_field(qset_id, HCLGE_TM_QS_ID_L_MSK,
+ HCLGE_TM_QS_ID_L_S);
+ qs_id_h = hnae3_get_field(qset_id, HCLGE_TM_QS_ID_H_EXT_MSK,
+ HCLGE_TM_QS_ID_H_EXT_S);
+ qset_id = 0;
+ hnae3_set_field(qset_id, HCLGE_TM_QS_ID_L_MSK, HCLGE_TM_QS_ID_L_S,
+ qs_id_l);
+ hnae3_set_field(qset_id, HCLGE_TM_QS_ID_H_MSK, HCLGE_TM_QS_ID_H_S,
+ qs_id_h);
cmd = HCLGE_OPC_TM_QS_TO_PRI_LINK;
map = (struct hclge_qs_to_pri_link_cmd *)desc.data;
return;
}
+ grp_num = hdev->num_tqps <= HCLGE_TQP_MAX_SIZE_DEV_V2 ?
+ HCLGE_BP_GRP_NUM : HCLGE_BP_EXT_GRP_NUM;
cmd = HCLGE_OPC_TM_BP_TO_QSET_MAPPING;
bp_to_qs_map_cmd = (struct hclge_bp_to_qs_map_cmd *)desc.data;
- for (group_id = 0; group_id < 32; group_id++) {
+ for (group_id = 0; group_id < grp_num; group_id++) {
hclge_cmd_setup_basic_desc(&desc, cmd, true);
bp_to_qs_map_cmd->tc_id = tc_id;
bp_to_qs_map_cmd->qs_group_id = group_id;
dev_info(&hdev->pdev->dev, "index | tm bp qset maping:\n");
i = 0;
- for (group_id = 0; group_id < 4; group_id++) {
+ for (group_id = 0; group_id < grp_num / 8; group_id++) {
dev_info(&hdev->pdev->dev,
"%04d | %08x:%08x:%08x:%08x:%08x:%08x:%08x:%08x\n",
group_id * 256, qset_mapping[(u32)(i + 7)],
u8 ir_u, ir_b, ir_s, bs_b, bs_s;
struct hclge_desc desc;
u32 shapping_para;
+ u32 rate;
int ret;
hclge_cmd_setup_basic_desc(&desc, HCLGE_OPC_QCN_SHAPPING_CFG, true);
ir_s = hclge_tm_get_field(shapping_para, IR_S);
bs_b = hclge_tm_get_field(shapping_para, BS_B);
bs_s = hclge_tm_get_field(shapping_para, BS_S);
+ rate = le32_to_cpu(shap_cfg_cmd->qs_rate);
dev_info(&hdev->pdev->dev,
- "qs%u ir_b:%u, ir_u:%u, ir_s:%u, bs_b:%u, bs_s:%u\n",
- qsid, ir_b, ir_u, ir_s, bs_b, bs_s);
+ "qs%u ir_b:%u, ir_u:%u, ir_s:%u, bs_b:%u, bs_s:%u, flag:%#x, rate:%u(Mbps)\n",
+ qsid, ir_b, ir_u, ir_s, bs_b, bs_s, shap_cfg_cmd->flag, rate);
}
static void hclge_dbg_dump_qs_shaper_all(struct hclge_dev *hdev)
hclge_cmd_setup_basic_desc(&desc[0], HCLGE_OPC_QUERY_RX_STATS,
true);
- desc[0].data[0] = cpu_to_le32((tqp->index & 0x1ff));
+ desc[0].data[0] = cpu_to_le32(tqp->index);
ret = hclge_cmd_send(&hdev->hw, desc, 1);
if (ret) {
dev_err(&hdev->pdev->dev,
HCLGE_OPC_QUERY_TX_STATS,
true);
- desc[0].data[0] = cpu_to_le32((tqp->index & 0x1ff));
+ desc[0].data[0] = cpu_to_le32(tqp->index);
ret = hclge_cmd_send(&hdev->hw, desc, 1);
if (ret) {
dev_err(&hdev->pdev->dev,
}
req = (struct hclge_pf_res_cmd *)desc.data;
- hdev->num_tqps = le16_to_cpu(req->tqp_num);
+ hdev->num_tqps = le16_to_cpu(req->tqp_num) +
+ le16_to_cpu(req->ext_tqp_num);
hdev->pkt_buf_size = le16_to_cpu(req->buf_size) << HCLGE_BUF_UNIT_S;
if (req->tx_buf_size)
hdev->dv_buf_size = roundup(hdev->dv_buf_size, HCLGE_BUF_SIZE_UNIT);
+ hdev->num_nic_msi = le16_to_cpu(req->msixcap_localid_number_nic);
+ if (hdev->num_nic_msi < HNAE3_MIN_VECTOR_NUM) {
+ dev_err(&hdev->pdev->dev,
+ "only %u msi resources available, not enough for pf(min:2).\n",
+ hdev->num_nic_msi);
+ return -EINVAL;
+ }
+
if (hnae3_dev_roce_supported(hdev)) {
- hdev->roce_base_msix_offset =
- hnae3_get_field(le16_to_cpu(req->msixcap_localid_ba_rocee),
- HCLGE_MSIX_OFT_ROCEE_M, HCLGE_MSIX_OFT_ROCEE_S);
hdev->num_roce_msi =
- hnae3_get_field(le16_to_cpu(req->pf_intr_vector_number),
- HCLGE_PF_VEC_NUM_M, HCLGE_PF_VEC_NUM_S);
-
- /* nic's msix numbers is always equals to the roce's. */
- hdev->num_nic_msi = hdev->num_roce_msi;
+ le16_to_cpu(req->pf_intr_vector_number_roce);
/* PF should have NIC vectors and Roce vectors,
* NIC vectors are queued before Roce vectors.
*/
- hdev->num_msi = hdev->num_roce_msi +
- hdev->roce_base_msix_offset;
+ hdev->num_msi = hdev->num_nic_msi + hdev->num_roce_msi;
} else {
- hdev->num_msi =
- hnae3_get_field(le16_to_cpu(req->pf_intr_vector_number),
- HCLGE_PF_VEC_NUM_M, HCLGE_PF_VEC_NUM_S);
-
- hdev->num_nic_msi = hdev->num_msi;
- }
-
- if (hdev->num_nic_msi < HNAE3_MIN_VECTOR_NUM) {
- dev_err(&hdev->pdev->dev,
- "Just %u msi resources, not enough for pf(min:2).\n",
- hdev->num_nic_msi);
- return -EINVAL;
+ hdev->num_msi = hdev->num_nic_msi;
}
return 0;
ae_dev->dev_specs.rss_ind_tbl_size = HCLGE_RSS_IND_TBL_SIZE;
ae_dev->dev_specs.rss_key_size = HCLGE_RSS_KEY_SIZE;
ae_dev->dev_specs.max_tm_rate = HCLGE_ETHER_MAX_RATE;
+ ae_dev->dev_specs.max_int_gl = HCLGE_DEF_MAX_INT_GL;
}
static void hclge_parse_dev_specs(struct hclge_dev *hdev,
{
struct hnae3_ae_dev *ae_dev = pci_get_drvdata(hdev->pdev);
struct hclge_dev_specs_0_cmd *req0;
+ struct hclge_dev_specs_1_cmd *req1;
req0 = (struct hclge_dev_specs_0_cmd *)desc[0].data;
+ req1 = (struct hclge_dev_specs_1_cmd *)desc[1].data;
ae_dev->dev_specs.max_non_tso_bd_num = req0->max_non_tso_bd_num;
ae_dev->dev_specs.rss_ind_tbl_size =
le16_to_cpu(req0->rss_ind_tbl_size);
+ ae_dev->dev_specs.int_ql_max = le16_to_cpu(req0->int_ql_max);
ae_dev->dev_specs.rss_key_size = le16_to_cpu(req0->rss_key_size);
ae_dev->dev_specs.max_tm_rate = le32_to_cpu(req0->max_tm_rate);
+ ae_dev->dev_specs.max_int_gl = le16_to_cpu(req1->max_int_gl);
}
static void hclge_check_dev_specs(struct hclge_dev *hdev)
dev_specs->rss_key_size = HCLGE_RSS_KEY_SIZE;
if (!dev_specs->max_tm_rate)
dev_specs->max_tm_rate = HCLGE_ETHER_MAX_RATE;
+ if (!dev_specs->max_int_gl)
+ dev_specs->max_int_gl = HCLGE_DEF_MAX_INT_GL;
}
static int hclge_query_dev_specs(struct hclge_dev *hdev)
tqp->q.buf_size = hdev->rx_buf_len;
tqp->q.tx_desc_num = hdev->num_tx_desc;
tqp->q.rx_desc_num = hdev->num_rx_desc;
- tqp->q.io_base = hdev->hw.io_base + HCLGE_TQP_REG_OFFSET +
- i * HCLGE_TQP_REG_SIZE;
+
+ /* need an extended offset to configure queues >=
+ * HCLGE_TQP_MAX_SIZE_DEV_V2
+ */
+ if (i < HCLGE_TQP_MAX_SIZE_DEV_V2)
+ tqp->q.io_base = hdev->hw.io_base +
+ HCLGE_TQP_REG_OFFSET +
+ i * HCLGE_TQP_REG_SIZE;
+ else
+ tqp->q.io_base = hdev->hw.io_base +
+ HCLGE_TQP_REG_OFFSET +
+ HCLGE_TQP_EXT_REG_OFFSET +
+ (i - HCLGE_TQP_MAX_SIZE_DEV_V2) *
+ HCLGE_TQP_REG_SIZE;
tqp++;
}
{
struct hnae3_handle *roce = &vport->roce;
struct hnae3_handle *nic = &vport->nic;
+ struct hclge_dev *hdev = vport->back;
roce->rinfo.num_vectors = vport->back->num_roce_msi;
- if (vport->back->num_msi_left < vport->roce.rinfo.num_vectors ||
- vport->back->num_msi_left == 0)
+ if (hdev->num_msi < hdev->num_nic_msi + hdev->num_roce_msi)
return -EINVAL;
- roce->rinfo.base_vector = vport->back->roce_base_vector;
+ roce->rinfo.base_vector = hdev->roce_base_vector;
roce->rinfo.netdev = nic->kinfo.netdev;
- roce->rinfo.roce_io_base = vport->back->hw.io_base;
+ roce->rinfo.roce_io_base = hdev->hw.io_base;
+ roce->rinfo.roce_mem_base = hdev->hw.mem_base;
roce->pdev = nic->pdev;
roce->ae_algo = nic->ae_algo;
hdev->base_msi_vector = pdev->irq;
hdev->roce_base_vector = hdev->base_msi_vector +
- hdev->roce_base_msix_offset;
+ hdev->num_nic_msi;
hdev->vector_status = devm_kcalloc(&pdev->dev, hdev->num_msi,
sizeof(u16), GFP_KERNEL);
return container_of(handle, struct hclge_vport, nic);
}
+static void hclge_get_vector_info(struct hclge_dev *hdev, u16 idx,
+ struct hnae3_vector_info *vector_info)
+{
+#define HCLGE_PF_MAX_VECTOR_NUM_DEV_V2 64
+
+ vector_info->vector = pci_irq_vector(hdev->pdev, idx);
+
+ /* need an extend offset to config vector >= 64 */
+ if (idx - 1 < HCLGE_PF_MAX_VECTOR_NUM_DEV_V2)
+ vector_info->io_addr = hdev->hw.io_base +
+ HCLGE_VECTOR_REG_BASE +
+ (idx - 1) * HCLGE_VECTOR_REG_OFFSET;
+ else
+ vector_info->io_addr = hdev->hw.io_base +
+ HCLGE_VECTOR_EXT_REG_BASE +
+ (idx - 1) / HCLGE_PF_MAX_VECTOR_NUM_DEV_V2 *
+ HCLGE_VECTOR_REG_OFFSET_H +
+ (idx - 1) % HCLGE_PF_MAX_VECTOR_NUM_DEV_V2 *
+ HCLGE_VECTOR_REG_OFFSET;
+
+ hdev->vector_status[idx] = hdev->vport[0].vport_id;
+ hdev->vector_irq[idx] = vector_info->vector;
+}
+
static int hclge_get_vector(struct hnae3_handle *handle, u16 vector_num,
struct hnae3_vector_info *vector_info)
{
struct hnae3_vector_info *vector = vector_info;
struct hclge_dev *hdev = vport->back;
int alloc = 0;
- int i, j;
+ u16 i = 0;
+ u16 j;
vector_num = min_t(u16, hdev->num_nic_msi - 1, vector_num);
vector_num = min(hdev->num_msi_left, vector_num);
for (j = 0; j < vector_num; j++) {
- for (i = 1; i < hdev->num_msi; i++) {
+ while (++i < hdev->num_nic_msi) {
if (hdev->vector_status[i] == HCLGE_INVALID_VPORT) {
- vector->vector = pci_irq_vector(hdev->pdev, i);
- vector->io_addr = hdev->hw.io_base +
- HCLGE_VECTOR_REG_BASE +
- (i - 1) * HCLGE_VECTOR_REG_OFFSET +
- vport->vport_id *
- HCLGE_VECTOR_VF_OFFSET;
- hdev->vector_status[i] = vport->vport_id;
- hdev->vector_irq[i] = vector->vector;
-
+ hclge_get_vector_info(hdev, i, vector);
vector++;
alloc++;
op = en ? HCLGE_OPC_ADD_RING_TO_VECTOR : HCLGE_OPC_DEL_RING_TO_VECTOR;
hclge_cmd_setup_basic_desc(&desc, op, false);
- req->int_vector_id = vector_id;
+ req->int_vector_id_l = hnae3_get_field(vector_id,
+ HCLGE_VECTOR_ID_L_M,
+ HCLGE_VECTOR_ID_L_S);
+ req->int_vector_id_h = hnae3_get_field(vector_id,
+ HCLGE_VECTOR_ID_H_M,
+ HCLGE_VECTOR_ID_H_S);
i = 0;
for (node = ring_chain; node; node = node->next) {
hclge_cmd_setup_basic_desc(&desc,
op,
false);
- req->int_vector_id = vector_id;
+ req->int_vector_id_l =
+ hnae3_get_field(vector_id,
+ HCLGE_VECTOR_ID_L_M,
+ HCLGE_VECTOR_ID_L_S);
+ req->int_vector_id_h =
+ hnae3_get_field(vector_id,
+ HCLGE_VECTOR_ID_H_M,
+ HCLGE_VECTOR_ID_H_S);
}
}
int ret;
hclge_cmd_setup_basic_desc(&desc, HCLGE_OPC_CFG_COM_TQP_QUEUE, false);
- req->tqp_id = cpu_to_le16(tqp_id & HCLGE_RING_ID_MASK);
+ req->tqp_id = cpu_to_le16(tqp_id);
req->stream_id = cpu_to_le16(stream_id);
if (enable)
req->enable |= 1U << HCLGE_TQP_ENABLE_B;
hclge_cmd_setup_basic_desc(&desc, HCLGE_OPC_RESET_TQP_QUEUE, false);
req = (struct hclge_reset_tqp_queue_cmd *)desc.data;
- req->tqp_id = cpu_to_le16(queue_id & HCLGE_RING_ID_MASK);
+ req->tqp_id = cpu_to_le16(queue_id);
if (enable)
hnae3_set_bit(req->reset_req, HCLGE_TQP_RESET_B, 1U);
hclge_cmd_setup_basic_desc(&desc, HCLGE_OPC_RESET_TQP_QUEUE, true);
req = (struct hclge_reset_tqp_queue_cmd *)desc.data;
- req->tqp_id = cpu_to_le16(queue_id & HCLGE_RING_ID_MASK);
+ req->tqp_id = cpu_to_le16(queue_id);
ret = hclge_cmd_send(&hdev->hw, &desc, 1);
if (ret) {
}
}
+static int hclge_dev_mem_map(struct hclge_dev *hdev)
+{
+#define HCLGE_MEM_BAR 4
+
+ struct pci_dev *pdev = hdev->pdev;
+ struct hclge_hw *hw = &hdev->hw;
+
+ /* for device does not have device memory, return directly */
+ if (!(pci_select_bars(pdev, IORESOURCE_MEM) & BIT(HCLGE_MEM_BAR)))
+ return 0;
+
+ hw->mem_base = devm_ioremap_wc(&pdev->dev,
+ pci_resource_start(pdev, HCLGE_MEM_BAR),
+ pci_resource_len(pdev, HCLGE_MEM_BAR));
+ if (!hw->mem_base) {
+ dev_err(&pdev->dev, "failed to map device memory\n");
+ return -EFAULT;
+ }
+
+ return 0;
+}
+
static int hclge_pci_init(struct hclge_dev *hdev)
{
struct pci_dev *pdev = hdev->pdev;
goto err_clr_master;
}
+ ret = hclge_dev_mem_map(hdev);
+ if (ret)
+ goto err_unmap_io_base;
+
hdev->num_req_vfs = pci_sriov_get_totalvfs(pdev);
return 0;
+
+err_unmap_io_base:
+ pcim_iounmap(pdev, hdev->hw.io_base);
err_clr_master:
pci_clear_master(pdev);
pci_release_regions(pdev);
{
struct pci_dev *pdev = hdev->pdev;
+ if (hdev->hw.mem_base)
+ devm_iounmap(&pdev->dev, hdev->hw.mem_base);
+
pcim_iounmap(pdev, hdev->hw.io_base);
pci_free_irq_vectors(pdev);
pci_clear_master(pdev);
(HCLGE_PF_CFG_BLOCK_SIZE / HCLGE_CFG_RD_LEN_BYTES)
#define HCLGE_VECTOR_REG_BASE 0x20000
+#define HCLGE_VECTOR_EXT_REG_BASE 0x30000
#define HCLGE_MISC_VECTOR_REG_BASE 0x20400
#define HCLGE_VECTOR_REG_OFFSET 0x4
+#define HCLGE_VECTOR_REG_OFFSET_H 0x1000
#define HCLGE_VECTOR_VF_OFFSET 0x100000
#define HCLGE_CMDQ_TX_ADDR_L_REG 0x27000
struct hclge_hw {
void __iomem *io_base;
+ void __iomem *mem_base;
struct hclge_mac mac;
int num_vec;
struct hclge_cmq cmq;
u16 num_msi;
u16 num_msi_left;
u16 num_msi_used;
- u16 roce_base_msix_offset;
u32 base_msi_vector;
u16 *vector_status;
int *vector_irq;
{
struct hclge_nq_to_qs_link_cmd *map;
struct hclge_desc desc;
+ u16 qs_id_l;
+ u16 qs_id_h;
hclge_cmd_setup_basic_desc(&desc, HCLGE_OPC_TM_NQ_TO_QS_LINK, false);
map = (struct hclge_nq_to_qs_link_cmd *)desc.data;
map->nq_id = cpu_to_le16(q_id);
+
+ /* convert qs_id to the following format to support qset_id >= 1024
+ * qs_id: | 15 | 14 ~ 10 | 9 ~ 0 |
+ * / / \ \
+ * / / \ \
+ * qset_id: | 15 ~ 11 | 10 | 9 ~ 0 |
+ * | qs_id_h | vld | qs_id_l |
+ */
+ qs_id_l = hnae3_get_field(qs_id, HCLGE_TM_QS_ID_L_MSK,
+ HCLGE_TM_QS_ID_L_S);
+ qs_id_h = hnae3_get_field(qs_id, HCLGE_TM_QS_ID_H_MSK,
+ HCLGE_TM_QS_ID_H_S);
+ hnae3_set_field(qs_id, HCLGE_TM_QS_ID_L_MSK, HCLGE_TM_QS_ID_L_S,
+ qs_id_l);
+ hnae3_set_field(qs_id, HCLGE_TM_QS_ID_H_EXT_MSK, HCLGE_TM_QS_ID_H_EXT_S,
+ qs_id_h);
map->qset_id = cpu_to_le16(qs_id | HCLGE_TM_Q_QS_LINK_VLD_MSK);
return hclge_cmd_send(&hdev->hw, &desc, 1);
static int hclge_tm_pg_shapping_cfg(struct hclge_dev *hdev,
enum hclge_shap_bucket bucket, u8 pg_id,
- u32 shapping_para)
+ u32 shapping_para, u32 rate)
{
struct hclge_pg_shapping_cmd *shap_cfg_cmd;
enum hclge_opcode_type opcode;
shap_cfg_cmd->pg_shapping_para = cpu_to_le32(shapping_para);
+ hnae3_set_bit(shap_cfg_cmd->flag, HCLGE_TM_RATE_VLD, 1);
+
+ shap_cfg_cmd->pg_rate = cpu_to_le32(rate);
+
return hclge_cmd_send(&hdev->hw, &desc, 1);
}
shap_cfg_cmd->port_shapping_para = cpu_to_le32(shapping_para);
+ hnae3_set_bit(shap_cfg_cmd->flag, HCLGE_TM_RATE_VLD, 1);
+
+ shap_cfg_cmd->port_rate = cpu_to_le32(hdev->hw.mac.speed);
+
return hclge_cmd_send(&hdev->hw, &desc, 1);
}
static int hclge_tm_pri_shapping_cfg(struct hclge_dev *hdev,
enum hclge_shap_bucket bucket, u8 pri_id,
- u32 shapping_para)
+ u32 shapping_para, u32 rate)
{
struct hclge_pri_shapping_cmd *shap_cfg_cmd;
enum hclge_opcode_type opcode;
shap_cfg_cmd->pri_shapping_para = cpu_to_le32(shapping_para);
+ hnae3_set_bit(shap_cfg_cmd->flag, HCLGE_TM_RATE_VLD, 1);
+
+ shap_cfg_cmd->pri_rate = cpu_to_le32(rate);
+
return hclge_cmd_send(&hdev->hw, &desc, 1);
}
shap_cfg_cmd->qs_id = cpu_to_le16(vport->qs_offset + i);
shap_cfg_cmd->qs_shapping_para = cpu_to_le32(shaper_para);
+ hnae3_set_bit(shap_cfg_cmd->flag, HCLGE_TM_RATE_VLD, 1);
+ shap_cfg_cmd->qs_rate = cpu_to_le32(max_tx_rate);
+
ret = hclge_cmd_send(&hdev->hw, &desc, 1);
if (ret) {
dev_err(&hdev->pdev->dev,
}
}
-static void hclge_pfc_info_init(struct hclge_dev *hdev)
+static void hclge_update_fc_mode_by_dcb_flag(struct hclge_dev *hdev)
{
if (!(hdev->flag & HCLGE_FLAG_DCB_ENABLE)) {
if (hdev->fc_mode_last_time == HCLGE_FC_PFC)
}
}
+static void hclge_update_fc_mode(struct hclge_dev *hdev)
+{
+ if (!hdev->tm_info.pfc_en) {
+ hdev->tm_info.fc_mode = hdev->fc_mode_last_time;
+ return;
+ }
+
+ if (hdev->tm_info.fc_mode != HCLGE_FC_PFC) {
+ hdev->fc_mode_last_time = hdev->tm_info.fc_mode;
+ hdev->tm_info.fc_mode = HCLGE_FC_PFC;
+ }
+}
+
+static void hclge_pfc_info_init(struct hclge_dev *hdev)
+{
+ if (hdev->ae_dev->dev_version >= HNAE3_DEVICE_VERSION_V3)
+ hclge_update_fc_mode(hdev);
+ else
+ hclge_update_fc_mode_by_dcb_flag(hdev);
+}
+
static void hclge_tm_schd_info_init(struct hclge_dev *hdev)
{
hclge_tm_pg_info_init(hdev);
/* Pg to pri */
for (i = 0; i < hdev->tm_info.num_pg; i++) {
+ u32 rate = hdev->tm_info.pg_info[i].bw_limit;
+
/* Calc shaper para */
- ret = hclge_shaper_para_calc(hdev->tm_info.pg_info[i].bw_limit,
- HCLGE_SHAPER_LVL_PG,
+ ret = hclge_shaper_para_calc(rate, HCLGE_SHAPER_LVL_PG,
&ir_para, max_tm_rate);
if (ret)
return ret;
HCLGE_SHAPER_BS_S_DEF);
ret = hclge_tm_pg_shapping_cfg(hdev,
HCLGE_TM_SHAP_C_BUCKET, i,
- shaper_para);
+ shaper_para, rate);
if (ret)
return ret;
HCLGE_SHAPER_BS_S_DEF);
ret = hclge_tm_pg_shapping_cfg(hdev,
HCLGE_TM_SHAP_P_BUCKET, i,
- shaper_para);
+ shaper_para, rate);
if (ret)
return ret;
}
u32 i;
for (i = 0; i < hdev->tm_info.num_tc; i++) {
- ret = hclge_shaper_para_calc(hdev->tm_info.tc_info[i].bw_limit,
- HCLGE_SHAPER_LVL_PRI,
+ u32 rate = hdev->tm_info.tc_info[i].bw_limit;
+
+ ret = hclge_shaper_para_calc(rate, HCLGE_SHAPER_LVL_PRI,
&ir_para, max_tm_rate);
if (ret)
return ret;
HCLGE_SHAPER_BS_U_DEF,
HCLGE_SHAPER_BS_S_DEF);
ret = hclge_tm_pri_shapping_cfg(hdev, HCLGE_TM_SHAP_C_BUCKET, i,
- shaper_para);
+ shaper_para, rate);
if (ret)
return ret;
HCLGE_SHAPER_BS_U_DEF,
HCLGE_SHAPER_BS_S_DEF);
ret = hclge_tm_pri_shapping_cfg(hdev, HCLGE_TM_SHAP_P_BUCKET, i,
- shaper_para);
+ shaper_para, rate);
if (ret)
return ret;
}
HCLGE_SHAPER_BS_U_DEF,
HCLGE_SHAPER_BS_S_DEF);
ret = hclge_tm_pri_shapping_cfg(hdev, HCLGE_TM_SHAP_C_BUCKET,
- vport->vport_id, shaper_para);
+ vport->vport_id, shaper_para,
+ vport->bw_limit);
if (ret)
return ret;
HCLGE_SHAPER_BS_U_DEF,
HCLGE_SHAPER_BS_S_DEF);
ret = hclge_tm_pri_shapping_cfg(hdev, HCLGE_TM_SHAP_P_BUCKET,
- vport->vport_id, shaper_para);
+ vport->vport_id, shaper_para,
+ vport->bw_limit);
if (ret)
return ret;
hdev->tm_info.pfc_en);
}
-/* Each Tc has a 1024 queue sets to backpress, it divides to
- * 32 group, each group contains 32 queue sets, which can be
- * represented by u32 bitmap.
+/* for the queues that use for backpress, divides to several groups,
+ * each group contains 32 queue sets, which can be represented by u32 bitmap.
*/
static int hclge_bp_setup_hw(struct hclge_dev *hdev, u8 tc)
{
+ u16 grp_id_shift = HCLGE_BP_GRP_ID_S;
+ u16 grp_id_mask = HCLGE_BP_GRP_ID_M;
+ u8 grp_num = HCLGE_BP_GRP_NUM;
int i;
- for (i = 0; i < HCLGE_BP_GRP_NUM; i++) {
+ if (hdev->num_tqps > HCLGE_TQP_MAX_SIZE_DEV_V2) {
+ grp_num = HCLGE_BP_EXT_GRP_NUM;
+ grp_id_mask = HCLGE_BP_EXT_GRP_ID_M;
+ grp_id_shift = HCLGE_BP_EXT_GRP_ID_S;
+ }
+
+ for (i = 0; i < grp_num; i++) {
u32 qs_bitmap = 0;
int k, ret;
u16 qs_id = vport->qs_offset + tc;
u8 grp, sub_grp;
- grp = hnae3_get_field(qs_id, HCLGE_BP_GRP_ID_M,
- HCLGE_BP_GRP_ID_S);
+ grp = hnae3_get_field(qs_id, grp_id_mask, grp_id_shift);
sub_grp = hnae3_get_field(qs_id, HCLGE_BP_SUB_GRP_ID_M,
HCLGE_BP_SUB_GRP_ID_S);
if (i == grp)
__le16 nq_id;
__le16 rsvd;
#define HCLGE_TM_Q_QS_LINK_VLD_MSK BIT(10)
+#define HCLGE_TM_QS_ID_L_MSK GENMASK(9, 0)
+#define HCLGE_TM_QS_ID_L_S 0
+#define HCLGE_TM_QS_ID_H_MSK GENMASK(14, 10)
+#define HCLGE_TM_QS_ID_H_S 10
+#define HCLGE_TM_QS_ID_H_EXT_S 11
+#define HCLGE_TM_QS_ID_H_EXT_MSK GENMASK(15, 11)
__le16 qset_id;
};
HCLGE_TM_SHAP_P_BUCKET,
};
+/* set bit HCLGE_TM_RATE_VLD to 1 means use 'rate' to config shaping */
+#define HCLGE_TM_RATE_VLD 0
+
struct hclge_pri_shapping_cmd {
u8 pri_id;
u8 rsvd[3];
__le32 pri_shapping_para;
+ u8 flag;
+ u8 rsvd1[3];
+ __le32 pri_rate;
};
struct hclge_pg_shapping_cmd {
u8 pg_id;
u8 rsvd[3];
__le32 pg_shapping_para;
+ u8 flag;
+ u8 rsvd1[3];
+ __le32 pg_rate;
};
struct hclge_qs_shapping_cmd {
__le16 qs_id;
u8 rsvd[2];
__le32 qs_shapping_para;
+ u8 flag;
+ u8 rsvd1[3];
+ __le32 qs_rate;
};
#define HCLGE_BP_GRP_NUM 32
#define HCLGE_BP_SUB_GRP_ID_M GENMASK(4, 0)
#define HCLGE_BP_GRP_ID_S 5
#define HCLGE_BP_GRP_ID_M GENMASK(9, 5)
+
+#define HCLGE_BP_EXT_GRP_NUM 40
+#define HCLGE_BP_EXT_GRP_ID_S 5
+#define HCLGE_BP_EXT_GRP_ID_M GENMASK(10, 5)
+
struct hclge_bp_to_qs_map_cmd {
u8 tc_id;
u8 rsvd[2];
struct hclge_port_shapping_cmd {
__le32 port_shapping_para;
+ u8 flag;
+ u8 rsvd[3];
+ __le32 port_rate;
};
struct hclge_shaper_ir_para {
set_bit(HNAE3_DEV_SUPPORT_INT_QL_B, ae_dev->caps);
if (hnae3_get_bit(caps, HCLGEVF_CAP_TQP_TXRX_INDEP_B))
set_bit(HNAE3_DEV_SUPPORT_TQP_TXRX_INDEP_B, ae_dev->caps);
+ if (hnae3_get_bit(caps, HCLGEVF_CAP_HW_TX_CSUM_B))
+ set_bit(HNAE3_DEV_SUPPORT_HW_TX_CSUM_B, ae_dev->caps);
+ if (hnae3_get_bit(caps, HCLGEVF_CAP_UDP_TUNNEL_CSUM_B))
+ set_bit(HNAE3_DEV_SUPPORT_UDP_TUNNEL_CSUM_B, ae_dev->caps);
}
static int hclgevf_cmd_query_version_and_capability(struct hclgevf_dev *hdev)
#define HCLGEVF_TQP_REG_OFFSET 0x80000
#define HCLGEVF_TQP_REG_SIZE 0x200
+#define HCLGEVF_TQP_MAX_SIZE_DEV_V2 1024
+#define HCLGEVF_TQP_EXT_REG_OFFSET 0x100
+
struct hclgevf_tqp_map {
__le16 tqp_id; /* Absolute tqp id for in this pf */
u8 tqp_vf; /* VF id */
HCLGEVF_CAP_FD_FORWARD_TC_B,
HCLGEVF_CAP_PTP_B,
HCLGEVF_CAP_INT_QL_B,
- HCLGEVF_CAP_SIMPLE_BD_B,
+ HCLGEVF_CAP_HW_TX_CSUM_B,
HCLGEVF_CAP_TX_PUSH_B,
HCLGEVF_CAP_PHY_IMP_B,
HCLGEVF_CAP_TQP_TXRX_INDEP_B,
HCLGEVF_CAP_HW_PAD_B,
HCLGEVF_CAP_STASH_B,
+ HCLGEVF_CAP_UDP_TUNNEL_CSUM_B,
};
#define HCLGEVF_QUERY_CAP_LENGTH 3
u8 rsv1[5];
};
+#define HCLGEVF_DEF_MAX_INT_GL 0x1FE0U
+
+struct hclgevf_dev_specs_1_cmd {
+ __le32 rsv0;
+ __le16 max_int_gl;
+ u8 rsv1[18];
+};
+
static inline void hclgevf_write_reg(void __iomem *base, u32 reg, u32 value)
{
writel(value, base + reg);
tqp->q.buf_size = hdev->rx_buf_len;
tqp->q.tx_desc_num = hdev->num_tx_desc;
tqp->q.rx_desc_num = hdev->num_rx_desc;
- tqp->q.io_base = hdev->hw.io_base + HCLGEVF_TQP_REG_OFFSET +
- i * HCLGEVF_TQP_REG_SIZE;
+
+ /* need an extended offset to configure queues >=
+ * HCLGEVF_TQP_MAX_SIZE_DEV_V2.
+ */
+ if (i < HCLGEVF_TQP_MAX_SIZE_DEV_V2)
+ tqp->q.io_base = hdev->hw.io_base +
+ HCLGEVF_TQP_REG_OFFSET +
+ i * HCLGEVF_TQP_REG_SIZE;
+ else
+ tqp->q.io_base = hdev->hw.io_base +
+ HCLGEVF_TQP_REG_OFFSET +
+ HCLGEVF_TQP_EXT_REG_OFFSET +
+ (i - HCLGEVF_TQP_MAX_SIZE_DEV_V2) *
+ HCLGEVF_TQP_REG_SIZE;
tqp++;
}
roce->rinfo.netdev = nic->kinfo.netdev;
roce->rinfo.roce_io_base = hdev->hw.io_base;
+ roce->rinfo.roce_mem_base = hdev->hw.mem_base;
roce->pdev = nic->pdev;
roce->ae_algo = nic->ae_algo;
}
}
+static int hclgevf_dev_mem_map(struct hclgevf_dev *hdev)
+{
+#define HCLGEVF_MEM_BAR 4
+
+ struct pci_dev *pdev = hdev->pdev;
+ struct hclgevf_hw *hw = &hdev->hw;
+
+ /* for device does not have device memory, return directly */
+ if (!(pci_select_bars(pdev, IORESOURCE_MEM) & BIT(HCLGEVF_MEM_BAR)))
+ return 0;
+
+ hw->mem_base = devm_ioremap_wc(&pdev->dev,
+ pci_resource_start(pdev,
+ HCLGEVF_MEM_BAR),
+ pci_resource_len(pdev, HCLGEVF_MEM_BAR));
+ if (!hw->mem_base) {
+ dev_err(&pdev->dev, "failed to map device memory\n");
+ return -EFAULT;
+ }
+
+ return 0;
+}
+
static int hclgevf_pci_init(struct hclgevf_dev *hdev)
{
struct pci_dev *pdev = hdev->pdev;
goto err_clr_master;
}
+ ret = hclgevf_dev_mem_map(hdev);
+ if (ret)
+ goto err_unmap_io_base;
+
return 0;
+err_unmap_io_base:
+ pci_iounmap(pdev, hdev->hw.io_base);
err_clr_master:
pci_clear_master(pdev);
pci_release_regions(pdev);
{
struct pci_dev *pdev = hdev->pdev;
+ if (hdev->hw.mem_base)
+ devm_iounmap(&pdev->dev, hdev->hw.mem_base);
+
pci_iounmap(pdev, hdev->hw.io_base);
pci_clear_master(pdev);
pci_release_regions(pdev);
HCLGEVF_MAX_NON_TSO_BD_NUM;
ae_dev->dev_specs.rss_ind_tbl_size = HCLGEVF_RSS_IND_TBL_SIZE;
ae_dev->dev_specs.rss_key_size = HCLGEVF_RSS_KEY_SIZE;
+ ae_dev->dev_specs.max_int_gl = HCLGEVF_DEF_MAX_INT_GL;
}
static void hclgevf_parse_dev_specs(struct hclgevf_dev *hdev,
{
struct hnae3_ae_dev *ae_dev = pci_get_drvdata(hdev->pdev);
struct hclgevf_dev_specs_0_cmd *req0;
+ struct hclgevf_dev_specs_1_cmd *req1;
req0 = (struct hclgevf_dev_specs_0_cmd *)desc[0].data;
+ req1 = (struct hclgevf_dev_specs_1_cmd *)desc[1].data;
ae_dev->dev_specs.max_non_tso_bd_num = req0->max_non_tso_bd_num;
ae_dev->dev_specs.rss_ind_tbl_size =
le16_to_cpu(req0->rss_ind_tbl_size);
+ ae_dev->dev_specs.int_ql_max = le16_to_cpu(req0->int_ql_max);
ae_dev->dev_specs.rss_key_size = le16_to_cpu(req0->rss_key_size);
+ ae_dev->dev_specs.max_int_gl = le16_to_cpu(req1->max_int_gl);
}
static void hclgevf_check_dev_specs(struct hclgevf_dev *hdev)
dev_specs->rss_ind_tbl_size = HCLGEVF_RSS_IND_TBL_SIZE;
if (!dev_specs->rss_key_size)
dev_specs->rss_key_size = HCLGEVF_RSS_KEY_SIZE;
+ if (!dev_specs->max_int_gl)
+ dev_specs->max_int_gl = HCLGEVF_DEF_MAX_INT_GL;
}
static int hclgevf_query_dev_specs(struct hclgevf_dev *hdev)
struct hclgevf_hw {
void __iomem *io_base;
+ void __iomem *mem_base;
int num_vec;
struct hclgevf_cmq cmq;
struct hclgevf_mac mac;
struct netlink_ext_ack *extack)
{
struct hinic_devlink_priv *priv = devlink_priv(devlink);
- const struct firmware *fw;
- int err;
-
- err = request_firmware_direct(&fw, params->file_name,
- &priv->hwdev->hwif->pdev->dev);
- if (err)
- return err;
-
- err = hinic_firmware_update(priv, fw, extack);
- release_firmware(fw);
- return err;
+ return hinic_firmware_update(priv, params->fw, extack);
}
static const struct devlink_ops hinic_devlink_ops = {
#define HINIC_PORT_H
#include <linux/types.h>
+#include <linux/ethtool.h>
#include <linux/etherdevice.h>
#include <linux/bitops.h>
static int ibmvnic_send_crq_init(struct ibmvnic_adapter *);
static int ibmvnic_reenable_crq_queue(struct ibmvnic_adapter *);
static int ibmvnic_send_crq(struct ibmvnic_adapter *, union ibmvnic_crq *);
-static int send_subcrq(struct ibmvnic_adapter *adapter, u64 remote_handle,
- union sub_crq *sub_crq);
static int send_subcrq_indirect(struct ibmvnic_adapter *, u64, u64, u64);
static irqreturn_t ibmvnic_interrupt_rx(int irq, void *instance);
static int enable_scrq_irq(struct ibmvnic_adapter *,
int count = pool->size - atomic_read(&pool->available);
u64 handle = adapter->rx_scrq[pool->index]->handle;
struct device *dev = &adapter->vdev->dev;
+ struct ibmvnic_ind_xmit_queue *ind_bufp;
+ struct ibmvnic_sub_crq_queue *rx_scrq;
+ union sub_crq *sub_crq;
int buffers_added = 0;
unsigned long lpar_rc;
- union sub_crq sub_crq;
struct sk_buff *skb;
unsigned int offset;
dma_addr_t dma_addr;
if (!pool->active)
return;
+ rx_scrq = adapter->rx_scrq[pool->index];
+ ind_bufp = &rx_scrq->ind_buf;
for (i = 0; i < count; ++i) {
- skb = alloc_skb(pool->buff_size, GFP_ATOMIC);
+ skb = netdev_alloc_skb(adapter->netdev, pool->buff_size);
if (!skb) {
dev_err(dev, "Couldn't replenish rx buff\n");
adapter->replenish_no_mem++;
pool->rx_buff[index].pool_index = pool->index;
pool->rx_buff[index].size = pool->buff_size;
- memset(&sub_crq, 0, sizeof(sub_crq));
- sub_crq.rx_add.first = IBMVNIC_CRQ_CMD;
- sub_crq.rx_add.correlator =
+ sub_crq = &ind_bufp->indir_arr[ind_bufp->index++];
+ memset(sub_crq, 0, sizeof(*sub_crq));
+ sub_crq->rx_add.first = IBMVNIC_CRQ_CMD;
+ sub_crq->rx_add.correlator =
cpu_to_be64((u64)&pool->rx_buff[index]);
- sub_crq.rx_add.ioba = cpu_to_be32(dma_addr);
- sub_crq.rx_add.map_id = pool->long_term_buff.map_id;
+ sub_crq->rx_add.ioba = cpu_to_be32(dma_addr);
+ sub_crq->rx_add.map_id = pool->long_term_buff.map_id;
/* The length field of the sCRQ is defined to be 24 bits so the
* buffer size needs to be left shifted by a byte before it is
#ifdef __LITTLE_ENDIAN__
shift = 8;
#endif
- sub_crq.rx_add.len = cpu_to_be32(pool->buff_size << shift);
-
- lpar_rc = send_subcrq(adapter, handle, &sub_crq);
- if (lpar_rc != H_SUCCESS)
- goto failure;
-
- buffers_added++;
- adapter->replenish_add_buff_success++;
+ sub_crq->rx_add.len = cpu_to_be32(pool->buff_size << shift);
pool->next_free = (pool->next_free + 1) % pool->size;
+ if (ind_bufp->index == IBMVNIC_MAX_IND_DESCS ||
+ i == count - 1) {
+ lpar_rc =
+ send_subcrq_indirect(adapter, handle,
+ (u64)ind_bufp->indir_dma,
+ (u64)ind_bufp->index);
+ if (lpar_rc != H_SUCCESS)
+ goto failure;
+ buffers_added += ind_bufp->index;
+ adapter->replenish_add_buff_success += ind_bufp->index;
+ ind_bufp->index = 0;
+ }
}
atomic_add(buffers_added, &pool->available);
return;
failure:
if (lpar_rc != H_PARAMETER && lpar_rc != H_CLOSED)
dev_err_ratelimited(dev, "rx: replenish packet buffer failed\n");
- pool->free_map[pool->next_free] = index;
- pool->rx_buff[index].skb = NULL;
-
- dev_kfree_skb_any(skb);
- adapter->replenish_add_buff_failure++;
- atomic_add(buffers_added, &pool->available);
+ for (i = ind_bufp->index - 1; i >= 0; --i) {
+ struct ibmvnic_rx_buff *rx_buff;
+ pool->next_free = pool->next_free == 0 ?
+ pool->size - 1 : pool->next_free - 1;
+ sub_crq = &ind_bufp->indir_arr[i];
+ rx_buff = (struct ibmvnic_rx_buff *)
+ be64_to_cpu(sub_crq->rx_add.correlator);
+ index = (int)(rx_buff - pool->rx_buff);
+ pool->free_map[pool->next_free] = index;
+ dev_kfree_skb_any(pool->rx_buff[index].skb);
+ pool->rx_buff[index].skb = NULL;
+ }
+ ind_bufp->index = 0;
if (lpar_rc == H_CLOSED || adapter->failover_pending) {
/* Disable buffer pool replenishment and report carrier off if
* queue is closed or pending failover.
if (rx_pool->buff_size != buff_size) {
free_long_term_buff(adapter, &rx_pool->long_term_buff);
- rx_pool->buff_size = buff_size;
+ rx_pool->buff_size = ALIGN(buff_size, L1_CACHE_BYTES);
rc = alloc_long_term_buff(adapter,
&rx_pool->long_term_buff,
rx_pool->size *
rx_pool->size = adapter->req_rx_add_entries_per_subcrq;
rx_pool->index = i;
- rx_pool->buff_size = buff_size;
+ rx_pool->buff_size = ALIGN(buff_size, L1_CACHE_BYTES);
rx_pool->active = 1;
rx_pool->free_map = kcalloc(rx_pool->size, sizeof(int),
{
struct ibmvnic_adapter *adapter = netdev_priv(netdev);
int tx_subcrqs;
+ u64 buff_size;
int i, rc;
tx_subcrqs = adapter->num_active_tx_scrqs;
adapter->num_active_tx_pools = tx_subcrqs;
for (i = 0; i < tx_subcrqs; i++) {
+ buff_size = adapter->req_mtu + VLAN_HLEN;
+ buff_size = ALIGN(buff_size, L1_CACHE_BYTES);
rc = init_one_tx_pool(netdev, &adapter->tx_pool[i],
adapter->req_tx_entries_per_subcrq,
- adapter->req_mtu + VLAN_HLEN);
+ buff_size);
if (rc) {
release_tx_pools(adapter);
return rc;
static int ibmvnic_login(struct net_device *netdev)
{
struct ibmvnic_adapter *adapter = netdev_priv(netdev);
- unsigned long timeout = msecs_to_jiffies(30000);
+ unsigned long timeout = msecs_to_jiffies(20000);
int retry_count = 0;
int retries = 10;
bool retry;
adapter->init_done_rc = 0;
reinit_completion(&adapter->init_done);
rc = send_login(adapter);
- if (rc) {
- netdev_warn(netdev, "Unable to login\n");
+ if (rc)
return rc;
- }
if (!wait_for_completion_timeout(&adapter->init_done,
timeout)) {
static int set_link_state(struct ibmvnic_adapter *adapter, u8 link_state)
{
struct net_device *netdev = adapter->netdev;
- unsigned long timeout = msecs_to_jiffies(30000);
+ unsigned long timeout = msecs_to_jiffies(20000);
union ibmvnic_crq crq;
bool resend;
int rc;
if (prev_state == VNIC_CLOSED)
enable_irq(adapter->tx_scrq[i]->irq);
enable_scrq_irq(adapter, adapter->tx_scrq[i]);
+ netdev_tx_reset_queue(netdev_get_tx_queue(netdev, i));
}
rc = set_link_state(adapter, IBMVNIC_LOGICAL_LNK_UP);
* L2/L3/L4 packet header descriptors to be sent by send_subcrq_indirect.
*/
-static void build_hdr_descs_arr(struct ibmvnic_tx_buff *txbuff,
+static void build_hdr_descs_arr(struct sk_buff *skb,
+ union sub_crq *indir_arr,
int *num_entries, u8 hdr_field)
{
int hdr_len[3] = {0, 0, 0};
+ u8 hdr_data[140] = {0};
int tot_len;
- u8 *hdr_data = txbuff->hdr_data;
- tot_len = build_hdr_data(hdr_field, txbuff->skb, hdr_len,
- txbuff->hdr_data);
+ tot_len = build_hdr_data(hdr_field, skb, hdr_len,
+ hdr_data);
*num_entries += create_hdr_descs(hdr_field, hdr_data, tot_len, hdr_len,
- txbuff->indir_arr + 1);
+ indir_arr + 1);
}
static int ibmvnic_xmit_workarounds(struct sk_buff *skb,
return 0;
}
+static void ibmvnic_tx_scrq_clean_buffer(struct ibmvnic_adapter *adapter,
+ struct ibmvnic_sub_crq_queue *tx_scrq)
+{
+ struct ibmvnic_ind_xmit_queue *ind_bufp;
+ struct ibmvnic_tx_buff *tx_buff;
+ struct ibmvnic_tx_pool *tx_pool;
+ union sub_crq tx_scrq_entry;
+ int queue_num;
+ int entries;
+ int index;
+ int i;
+
+ ind_bufp = &tx_scrq->ind_buf;
+ entries = (u64)ind_bufp->index;
+ queue_num = tx_scrq->pool_index;
+
+ for (i = entries - 1; i >= 0; --i) {
+ tx_scrq_entry = ind_bufp->indir_arr[i];
+ if (tx_scrq_entry.v1.type != IBMVNIC_TX_DESC)
+ continue;
+ index = be32_to_cpu(tx_scrq_entry.v1.correlator);
+ if (index & IBMVNIC_TSO_POOL_MASK) {
+ tx_pool = &adapter->tso_pool[queue_num];
+ index &= ~IBMVNIC_TSO_POOL_MASK;
+ } else {
+ tx_pool = &adapter->tx_pool[queue_num];
+ }
+ tx_pool->free_map[tx_pool->consumer_index] = index;
+ tx_pool->consumer_index = tx_pool->consumer_index == 0 ?
+ tx_pool->num_buffers - 1 :
+ tx_pool->consumer_index - 1;
+ tx_buff = &tx_pool->tx_buff[index];
+ adapter->netdev->stats.tx_packets--;
+ adapter->netdev->stats.tx_bytes -= tx_buff->skb->len;
+ adapter->tx_stats_buffers[queue_num].packets--;
+ adapter->tx_stats_buffers[queue_num].bytes -=
+ tx_buff->skb->len;
+ dev_kfree_skb_any(tx_buff->skb);
+ tx_buff->skb = NULL;
+ adapter->netdev->stats.tx_dropped++;
+ }
+ ind_bufp->index = 0;
+ if (atomic_sub_return(entries, &tx_scrq->used) <=
+ (adapter->req_tx_entries_per_subcrq / 2) &&
+ __netif_subqueue_stopped(adapter->netdev, queue_num)) {
+ netif_wake_subqueue(adapter->netdev, queue_num);
+ netdev_dbg(adapter->netdev, "Started queue %d\n",
+ queue_num);
+ }
+}
+
+static int ibmvnic_tx_scrq_flush(struct ibmvnic_adapter *adapter,
+ struct ibmvnic_sub_crq_queue *tx_scrq)
+{
+ struct ibmvnic_ind_xmit_queue *ind_bufp;
+ u64 dma_addr;
+ u64 entries;
+ u64 handle;
+ int rc;
+
+ ind_bufp = &tx_scrq->ind_buf;
+ dma_addr = (u64)ind_bufp->indir_dma;
+ entries = (u64)ind_bufp->index;
+ handle = tx_scrq->handle;
+
+ if (!entries)
+ return 0;
+ rc = send_subcrq_indirect(adapter, handle, dma_addr, entries);
+ if (rc)
+ ibmvnic_tx_scrq_clean_buffer(adapter, tx_scrq);
+ else
+ ind_bufp->index = 0;
+ return 0;
+}
+
static netdev_tx_t ibmvnic_xmit(struct sk_buff *skb, struct net_device *netdev)
{
struct ibmvnic_adapter *adapter = netdev_priv(netdev);
int queue_num = skb_get_queue_mapping(skb);
u8 *hdrs = (u8 *)&adapter->tx_rx_desc_req;
struct device *dev = &adapter->vdev->dev;
+ struct ibmvnic_ind_xmit_queue *ind_bufp;
struct ibmvnic_tx_buff *tx_buff = NULL;
struct ibmvnic_sub_crq_queue *tx_scrq;
struct ibmvnic_tx_pool *tx_pool;
unsigned int tx_send_failed = 0;
+ netdev_tx_t ret = NETDEV_TX_OK;
unsigned int tx_map_failed = 0;
+ union sub_crq indir_arr[16];
unsigned int tx_dropped = 0;
unsigned int tx_packets = 0;
unsigned int tx_bytes = 0;
unsigned char *dst;
int index = 0;
u8 proto = 0;
- u64 handle;
- netdev_tx_t ret = NETDEV_TX_OK;
+
+ tx_scrq = adapter->tx_scrq[queue_num];
+ txq = netdev_get_tx_queue(netdev, queue_num);
+ ind_bufp = &tx_scrq->ind_buf;
if (test_bit(0, &adapter->resetting)) {
if (!netif_subqueue_stopped(netdev, skb))
tx_send_failed++;
tx_dropped++;
ret = NETDEV_TX_OK;
+ ibmvnic_tx_scrq_flush(adapter, tx_scrq);
goto out;
}
tx_dropped++;
tx_send_failed++;
ret = NETDEV_TX_OK;
+ ibmvnic_tx_scrq_flush(adapter, tx_scrq);
goto out;
}
if (skb_is_gso(skb))
else
tx_pool = &adapter->tx_pool[queue_num];
- tx_scrq = adapter->tx_scrq[queue_num];
- txq = netdev_get_tx_queue(netdev, skb_get_queue_mapping(skb));
- handle = tx_scrq->handle;
-
index = tx_pool->free_map[tx_pool->consumer_index];
if (index == IBMVNIC_INVALID_MAP) {
tx_send_failed++;
tx_dropped++;
ret = NETDEV_TX_OK;
+ ibmvnic_tx_scrq_flush(adapter, tx_scrq);
goto out;
}
tx_buff = &tx_pool->tx_buff[index];
tx_buff->skb = skb;
- tx_buff->data_dma[0] = data_dma_addr;
- tx_buff->data_len[0] = skb->len;
tx_buff->index = index;
tx_buff->pool_index = queue_num;
- tx_buff->last_frag = true;
memset(&tx_crq, 0, sizeof(tx_crq));
tx_crq.v1.first = IBMVNIC_CRQ_CMD;
tx_crq.v1.mss = cpu_to_be16(skb_shinfo(skb)->gso_size);
hdrs += 2;
}
- /* determine if l2/3/4 headers are sent to firmware */
- if ((*hdrs >> 7) & 1) {
- build_hdr_descs_arr(tx_buff, &num_entries, *hdrs);
- tx_crq.v1.n_crq_elem = num_entries;
- tx_buff->num_entries = num_entries;
- tx_buff->indir_arr[0] = tx_crq;
- tx_buff->indir_dma = dma_map_single(dev, tx_buff->indir_arr,
- sizeof(tx_buff->indir_arr),
- DMA_TO_DEVICE);
- if (dma_mapping_error(dev, tx_buff->indir_dma)) {
- dev_kfree_skb_any(skb);
- tx_buff->skb = NULL;
- if (!firmware_has_feature(FW_FEATURE_CMO))
- dev_err(dev, "tx: unable to map descriptor array\n");
- tx_map_failed++;
- tx_dropped++;
- ret = NETDEV_TX_OK;
- goto tx_err_out;
- }
- lpar_rc = send_subcrq_indirect(adapter, handle,
- (u64)tx_buff->indir_dma,
- (u64)num_entries);
- dma_unmap_single(dev, tx_buff->indir_dma,
- sizeof(tx_buff->indir_arr), DMA_TO_DEVICE);
- } else {
- tx_buff->num_entries = num_entries;
- lpar_rc = send_subcrq(adapter, handle,
- &tx_crq);
- }
- if (lpar_rc != H_SUCCESS) {
- if (lpar_rc != H_CLOSED && lpar_rc != H_PARAMETER)
- dev_err_ratelimited(dev, "tx: send failed\n");
- dev_kfree_skb_any(skb);
- tx_buff->skb = NULL;
- if (lpar_rc == H_CLOSED || adapter->failover_pending) {
- /* Disable TX and report carrier off if queue is closed
- * or pending failover.
- * Firmware guarantees that a signal will be sent to the
- * driver, triggering a reset or some other action.
- */
- netif_tx_stop_all_queues(netdev);
- netif_carrier_off(netdev);
- }
+ if ((*hdrs >> 7) & 1)
+ build_hdr_descs_arr(skb, indir_arr, &num_entries, *hdrs);
- tx_send_failed++;
- tx_dropped++;
- ret = NETDEV_TX_OK;
- goto tx_err_out;
+ tx_crq.v1.n_crq_elem = num_entries;
+ tx_buff->num_entries = num_entries;
+ /* flush buffer if current entry can not fit */
+ if (num_entries + ind_bufp->index > IBMVNIC_MAX_IND_DESCS) {
+ lpar_rc = ibmvnic_tx_scrq_flush(adapter, tx_scrq);
+ if (lpar_rc != H_SUCCESS)
+ goto tx_flush_err;
+ }
+
+ indir_arr[0] = tx_crq;
+ memcpy(&ind_bufp->indir_arr[ind_bufp->index], &indir_arr[0],
+ num_entries * sizeof(struct ibmvnic_generic_scrq));
+ ind_bufp->index += num_entries;
+ if (__netdev_tx_sent_queue(txq, skb->len,
+ netdev_xmit_more() &&
+ ind_bufp->index < IBMVNIC_MAX_IND_DESCS)) {
+ lpar_rc = ibmvnic_tx_scrq_flush(adapter, tx_scrq);
+ if (lpar_rc != H_SUCCESS)
+ goto tx_err;
}
if (atomic_add_return(num_entries, &tx_scrq->used)
ret = NETDEV_TX_OK;
goto out;
-tx_err_out:
- /* roll back consumer index and map array*/
- if (tx_pool->consumer_index == 0)
- tx_pool->consumer_index =
- tx_pool->num_buffers - 1;
- else
- tx_pool->consumer_index--;
- tx_pool->free_map[tx_pool->consumer_index] = index;
+tx_flush_err:
+ dev_kfree_skb_any(skb);
+ tx_buff->skb = NULL;
+ tx_pool->consumer_index = tx_pool->consumer_index == 0 ?
+ tx_pool->num_buffers - 1 :
+ tx_pool->consumer_index - 1;
+ tx_dropped++;
+tx_err:
+ if (lpar_rc != H_CLOSED && lpar_rc != H_PARAMETER)
+ dev_err_ratelimited(dev, "tx: send failed\n");
+
+ if (lpar_rc == H_CLOSED || adapter->failover_pending) {
+ /* Disable TX and report carrier off if queue is closed
+ * or pending failover.
+ * Firmware guarantees that a signal will be sent to the
+ * driver, triggering a reset or some other action.
+ */
+ netif_tx_stop_all_queues(netdev);
+ netif_carrier_off(netdev);
+ }
out:
netdev->stats.tx_dropped += tx_dropped;
netdev->stats.tx_bytes += tx_bytes;
if (reset_state == VNIC_OPEN) {
rc = __ibmvnic_close(netdev);
if (rc)
- return rc;
+ goto out;
}
release_resources(adapter);
}
rc = ibmvnic_reset_init(adapter, true);
- if (rc)
- return IBMVNIC_INIT_FAILED;
+ if (rc) {
+ rc = IBMVNIC_INIT_FAILED;
+ goto out;
+ }
/* If the adapter was in PROBE state prior to the reset,
* exit here.
*/
if (reset_state == VNIC_PROBED)
- return 0;
+ goto out;
rc = ibmvnic_login(netdev);
if (rc) {
- adapter->state = reset_state;
- return rc;
+ goto out;
}
rc = init_resources(adapter);
if (rc)
- return rc;
+ goto out;
ibmvnic_disable_irqs(adapter);
return 0;
rc = __ibmvnic_open(netdev);
- if (rc)
- return IBMVNIC_OPEN_FAILED;
+ if (rc) {
+ rc = IBMVNIC_OPEN_FAILED;
+ goto out;
+ }
/* refresh device's multicast list */
ibmvnic_set_multi(netdev);
for (i = 0; i < adapter->req_rx_queues; i++)
napi_schedule(&adapter->napi[i]);
- return 0;
+out:
+ if (rc)
+ adapter->state = reset_state;
+ return rc;
}
/**
rc = ibmvnic_login(netdev);
if (rc) {
- adapter->state = reset_state;
goto out;
}
for (i = 0; i < adapter->req_rx_queues; i++)
napi_schedule(&adapter->napi[i]);
- if (adapter->reset_reason != VNIC_RESET_FAILOVER)
+ if (adapter->reset_reason == VNIC_RESET_FAILOVER ||
+ adapter->reset_reason == VNIC_RESET_MOBILITY) {
call_netdevice_notifiers(NETDEV_NOTIFY_PEERS, netdev);
+ call_netdevice_notifiers(NETDEV_RESEND_IGMP, netdev);
+ }
rc = 0;
out:
+ /* restore the adapter state if reset failed */
+ if (rc)
+ adapter->state = reset_state;
rtnl_unlock();
return rc;
if (rc) {
netdev_err(adapter->netdev,
"Couldn't initialize crq. rc=%d\n", rc);
- return rc;
+ goto out;
}
rc = ibmvnic_reset_init(adapter, false);
if (rc)
- return rc;
+ goto out;
/* If the adapter was in PROBE state prior to the reset,
* exit here.
*/
if (reset_state == VNIC_PROBED)
- return 0;
+ goto out;
rc = ibmvnic_login(netdev);
- if (rc) {
- adapter->state = VNIC_PROBED;
- return 0;
- }
+ if (rc)
+ goto out;
rc = init_resources(adapter);
if (rc)
- return rc;
+ goto out;
ibmvnic_disable_irqs(adapter);
adapter->state = VNIC_CLOSED;
if (reset_state == VNIC_CLOSED)
- return 0;
+ goto out;
rc = __ibmvnic_open(netdev);
- if (rc)
- return IBMVNIC_OPEN_FAILED;
+ if (rc) {
+ rc = IBMVNIC_OPEN_FAILED;
+ goto out;
+ }
- return 0;
+ call_netdevice_notifiers(NETDEV_NOTIFY_PEERS, netdev);
+ call_netdevice_notifiers(NETDEV_RESEND_IGMP, netdev);
+out:
+ /* restore adapter state if reset failed */
+ if (rc)
+ adapter->state = reset_state;
+ return rc;
}
static struct ibmvnic_rwi *get_next_rwi(struct ibmvnic_adapter *adapter)
return rwi;
}
-static void free_all_rwi(struct ibmvnic_adapter *adapter)
-{
- struct ibmvnic_rwi *rwi;
-
- rwi = get_next_rwi(adapter);
- while (rwi) {
- kfree(rwi);
- rwi = get_next_rwi(adapter);
- }
-}
-
static void __ibmvnic_reset(struct work_struct *work)
{
struct ibmvnic_rwi *rwi;
if (!saved_state) {
reset_state = adapter->state;
- adapter->state = VNIC_RESETTING;
saved_state = true;
}
spin_unlock_irqrestore(&adapter->state_lock, flags);
rc = do_hard_reset(adapter, rwi, reset_state);
rtnl_unlock();
}
+ if (rc) {
+ /* give backing device time to settle down */
+ netdev_dbg(adapter->netdev,
+ "[S:%d] Hard reset failed, waiting 60 secs\n",
+ adapter->state);
+ set_current_state(TASK_UNINTERRUPTIBLE);
+ schedule_timeout(60 * HZ);
+ }
} else if (!(rwi->reset_reason == VNIC_RESET_FATAL &&
adapter->from_passive_init)) {
rc = do_reset(adapter, rwi, reset_state);
}
kfree(rwi);
- if (rc == IBMVNIC_OPEN_FAILED) {
- if (list_empty(&adapter->rwi_list))
- adapter->state = VNIC_CLOSED;
- else
- adapter->state = reset_state;
- rc = 0;
- } else if (rc && rc != IBMVNIC_INIT_FAILED &&
- !adapter->force_reset_recovery)
- break;
+ adapter->last_reset_time = jiffies;
+
+ if (rc)
+ netdev_dbg(adapter->netdev, "Reset failed, rc=%d\n", rc);
rwi = get_next_rwi(adapter);
complete(&adapter->reset_done);
}
- if (rc) {
- netdev_dbg(adapter->netdev, "Reset failed\n");
- free_all_rwi(adapter);
- }
-
clear_bit_unlock(0, &adapter->resetting);
}
{
struct ibmvnic_adapter *adapter = netdev_priv(dev);
+ if (test_bit(0, &adapter->resetting)) {
+ netdev_err(adapter->netdev,
+ "Adapter is resetting, skip timeout reset\n");
+ return;
+ }
+ /* No queuing up reset until at least 5 seconds (default watchdog val)
+ * after last reset
+ */
+ if (time_before(jiffies, (adapter->last_reset_time + dev->watchdog_timeo))) {
+ netdev_dbg(dev, "Not yet time to tx timeout.\n");
+ return;
+ }
ibmvnic_reset(adapter, VNIC_RESET_TIMEOUT);
}
static int ibmvnic_poll(struct napi_struct *napi, int budget)
{
- struct net_device *netdev = napi->dev;
- struct ibmvnic_adapter *adapter = netdev_priv(netdev);
- int scrq_num = (int)(napi - adapter->napi);
- int frames_processed = 0;
+ struct ibmvnic_sub_crq_queue *rx_scrq;
+ struct ibmvnic_adapter *adapter;
+ struct net_device *netdev;
+ int frames_processed;
+ int scrq_num;
+
+ netdev = napi->dev;
+ adapter = netdev_priv(netdev);
+ scrq_num = (int)(napi - adapter->napi);
+ frames_processed = 0;
+ rx_scrq = adapter->rx_scrq[scrq_num];
restart_poll:
while (frames_processed < budget) {
if (unlikely(test_bit(0, &adapter->resetting) &&
adapter->reset_reason != VNIC_RESET_NON_FATAL)) {
- enable_scrq_irq(adapter, adapter->rx_scrq[scrq_num]);
+ enable_scrq_irq(adapter, rx_scrq);
napi_complete_done(napi, frames_processed);
return frames_processed;
}
- if (!pending_scrq(adapter, adapter->rx_scrq[scrq_num]))
+ if (!pending_scrq(adapter, rx_scrq))
break;
- next = ibmvnic_next_scrq(adapter, adapter->rx_scrq[scrq_num]);
+ /* The queue entry at the current index is peeked at above
+ * to determine that there is a valid descriptor awaiting
+ * processing. We want to be sure that the current slot
+ * holds a valid descriptor before reading its contents.
+ */
+ dma_rmb();
+ next = ibmvnic_next_scrq(adapter, rx_scrq);
rx_buff =
(struct ibmvnic_rx_buff *)be64_to_cpu(next->
rx_comp.correlator);
frames_processed++;
}
- if (adapter->state != VNIC_CLOSING)
+ if (adapter->state != VNIC_CLOSING &&
+ ((atomic_read(&adapter->rx_pool[scrq_num].available) <
+ adapter->req_rx_add_entries_per_subcrq / 2) ||
+ frames_processed < budget))
replenish_rx_pool(adapter, &adapter->rx_pool[scrq_num]);
-
if (frames_processed < budget) {
- enable_scrq_irq(adapter, adapter->rx_scrq[scrq_num]);
- napi_complete_done(napi, frames_processed);
- if (pending_scrq(adapter, adapter->rx_scrq[scrq_num]) &&
- napi_reschedule(napi)) {
- disable_scrq_irq(adapter, adapter->rx_scrq[scrq_num]);
- goto restart_poll;
+ if (napi_complete_done(napi, frames_processed)) {
+ enable_scrq_irq(adapter, rx_scrq);
+ if (pending_scrq(adapter, rx_scrq)) {
+ rmb();
+ if (napi_reschedule(napi)) {
+ disable_scrq_irq(adapter, rx_scrq);
+ goto restart_poll;
+ }
+ }
}
}
return frames_processed;
{
int rc;
+ if (!scrq) {
+ netdev_dbg(adapter->netdev,
+ "Invalid scrq reset. irq (%d) or msgs (%p).\n",
+ scrq->irq, scrq->msgs);
+ return -EINVAL;
+ }
+
if (scrq->irq) {
free_irq(scrq->irq, scrq);
irq_dispose_mapping(scrq->irq);
scrq->irq = 0;
}
- memset(scrq->msgs, 0, 4 * PAGE_SIZE);
- atomic_set(&scrq->used, 0);
- scrq->cur = 0;
+ if (scrq->msgs) {
+ memset(scrq->msgs, 0, 4 * PAGE_SIZE);
+ atomic_set(&scrq->used, 0);
+ scrq->cur = 0;
+ scrq->ind_buf.index = 0;
+ } else {
+ netdev_dbg(adapter->netdev, "Invalid scrq reset\n");
+ return -EINVAL;
+ }
rc = h_reg_sub_crq(adapter->vdev->unit_address, scrq->msg_token,
4 * PAGE_SIZE, &scrq->crq_num, &scrq->hw_irq);
{
int i, rc;
+ if (!adapter->tx_scrq || !adapter->rx_scrq)
+ return -EINVAL;
+
for (i = 0; i < adapter->req_tx_queues; i++) {
netdev_dbg(adapter->netdev, "Re-setting tx_scrq[%d]\n", i);
rc = reset_one_sub_crq_queue(adapter, adapter->tx_scrq[i]);
}
}
+ dma_free_coherent(dev,
+ IBMVNIC_IND_ARR_SZ,
+ scrq->ind_buf.indir_arr,
+ scrq->ind_buf.indir_dma);
+
dma_unmap_single(dev, scrq->msg_token, 4 * PAGE_SIZE,
DMA_BIDIRECTIONAL);
free_pages((unsigned long)scrq->msgs, 2);
scrq->adapter = adapter;
scrq->size = 4 * PAGE_SIZE / sizeof(*scrq->msgs);
+ scrq->ind_buf.index = 0;
+
+ scrq->ind_buf.indir_arr =
+ dma_alloc_coherent(dev,
+ IBMVNIC_IND_ARR_SZ,
+ &scrq->ind_buf.indir_dma,
+ GFP_KERNEL);
+
+ if (!scrq->ind_buf.indir_arr)
+ goto indir_failed;
+
spin_lock_init(&scrq->lock);
netdev_dbg(adapter->netdev,
return scrq;
+indir_failed:
+ do {
+ rc = plpar_hcall_norets(H_FREE_SUB_CRQ,
+ adapter->vdev->unit_address,
+ scrq->crq_num);
+ } while (rc == H_BUSY || rc == H_IS_LONG_BUSY(rc));
reg_failed:
dma_unmap_single(dev, scrq->msg_token, 4 * PAGE_SIZE,
DMA_BIDIRECTIONAL);
struct device *dev = &adapter->vdev->dev;
struct ibmvnic_tx_pool *tx_pool;
struct ibmvnic_tx_buff *txbuff;
+ struct netdev_queue *txq;
union sub_crq *next;
int index;
- int i, j;
+ int i;
restart_loop:
while (pending_scrq(adapter, scrq)) {
unsigned int pool = scrq->pool_index;
int num_entries = 0;
+ int total_bytes = 0;
+ int num_packets = 0;
+
+ /* The queue entry at the current index is peeked at above
+ * to determine that there is a valid descriptor awaiting
+ * processing. We want to be sure that the current slot
+ * holds a valid descriptor before reading its contents.
+ */
+ dma_rmb();
next = ibmvnic_next_scrq(adapter, scrq);
for (i = 0; i < next->tx_comp.num_comps; i++) {
- if (next->tx_comp.rcs[i]) {
+ if (next->tx_comp.rcs[i])
dev_err(dev, "tx error %x\n",
next->tx_comp.rcs[i]);
- continue;
- }
index = be32_to_cpu(next->tx_comp.correlators[i]);
if (index & IBMVNIC_TSO_POOL_MASK) {
tx_pool = &adapter->tso_pool[pool];
}
txbuff = &tx_pool->tx_buff[index];
-
- for (j = 0; j < IBMVNIC_MAX_FRAGS_PER_CRQ; j++) {
- if (!txbuff->data_dma[j])
- continue;
-
- txbuff->data_dma[j] = 0;
- }
-
- if (txbuff->last_frag) {
- dev_kfree_skb_any(txbuff->skb);
+ num_packets++;
+ num_entries += txbuff->num_entries;
+ if (txbuff->skb) {
+ total_bytes += txbuff->skb->len;
+ dev_consume_skb_irq(txbuff->skb);
txbuff->skb = NULL;
+ } else {
+ netdev_warn(adapter->netdev,
+ "TX completion received with NULL socket buffer\n");
}
-
- num_entries += txbuff->num_entries;
-
tx_pool->free_map[tx_pool->producer_index] = index;
tx_pool->producer_index =
(tx_pool->producer_index + 1) %
/* remove tx_comp scrq*/
next->tx_comp.first = 0;
+ txq = netdev_get_tx_queue(adapter->netdev, scrq->pool_index);
+ netdev_tx_completed_queue(txq, num_packets, total_bytes);
+
if (atomic_sub_return(num_entries, &scrq->used) <=
(adapter->req_tx_entries_per_subcrq / 2) &&
__netif_subqueue_stopped(adapter->netdev,
}
spin_unlock_irqrestore(&scrq->lock, flags);
+ /* Ensure that the entire buffer descriptor has been
+ * loaded before reading its contents
+ */
+ dma_rmb();
+
return entry;
}
}
}
-static int send_subcrq(struct ibmvnic_adapter *adapter, u64 remote_handle,
- union sub_crq *sub_crq)
-{
- unsigned int ua = adapter->vdev->unit_address;
- struct device *dev = &adapter->vdev->dev;
- u64 *u64_crq = (u64 *)sub_crq;
- int rc;
-
- netdev_dbg(adapter->netdev,
- "Sending sCRQ %016lx: %016lx %016lx %016lx %016lx\n",
- (unsigned long int)cpu_to_be64(remote_handle),
- (unsigned long int)cpu_to_be64(u64_crq[0]),
- (unsigned long int)cpu_to_be64(u64_crq[1]),
- (unsigned long int)cpu_to_be64(u64_crq[2]),
- (unsigned long int)cpu_to_be64(u64_crq[3]));
-
- /* Make sure the hypervisor sees the complete request */
- mb();
-
- rc = plpar_hcall_norets(H_SEND_SUB_CRQ, ua,
- cpu_to_be64(remote_handle),
- cpu_to_be64(u64_crq[0]),
- cpu_to_be64(u64_crq[1]),
- cpu_to_be64(u64_crq[2]),
- cpu_to_be64(u64_crq[3]));
-
- if (rc)
- print_subcrq_error(dev, rc, __func__);
-
- return rc;
-}
-
static int send_subcrq_indirect(struct ibmvnic_adapter *adapter,
u64 remote_handle, u64 ioba, u64 num_entries)
{
struct ibmvnic_login_rsp_buffer *login_rsp_buffer;
struct ibmvnic_login_buffer *login_buffer;
struct device *dev = &adapter->vdev->dev;
+ struct vnic_login_client_data *vlcd;
dma_addr_t rsp_buffer_token;
dma_addr_t buffer_token;
size_t rsp_buffer_size;
union ibmvnic_crq crq;
+ int client_data_len;
size_t buffer_size;
__be64 *tx_list_p;
__be64 *rx_list_p;
- int client_data_len;
- struct vnic_login_client_data *vlcd;
+ int rc;
int i;
if (!adapter->tx_scrq || !adapter->rx_scrq) {
crq.login.cmd = LOGIN;
crq.login.ioba = cpu_to_be32(buffer_token);
crq.login.len = cpu_to_be32(buffer_size);
- ibmvnic_send_crq(adapter, &crq);
+
+ adapter->login_pending = true;
+ rc = ibmvnic_send_crq(adapter, &crq);
+ if (rc) {
+ adapter->login_pending = false;
+ netdev_err(adapter->netdev, "Failed to send login, rc=%d\n", rc);
+ goto buf_rsp_map_failed;
+ }
return 0;
buf_rsp_map_failed:
kfree(login_rsp_buffer);
+ adapter->login_rsp_buf = NULL;
buf_rsp_alloc_failed:
dma_unmap_single(dev, buffer_token, buffer_size, DMA_TO_DEVICE);
buf_map_failed:
kfree(login_buffer);
+ adapter->login_buf = NULL;
buf_alloc_failed:
return -1;
}
u64 *size_array;
int i;
+ /* CHECK: Test/set of login_pending does not need to be atomic
+ * because only ibmvnic_tasklet tests/clears this.
+ */
+ if (!adapter->login_pending) {
+ netdev_warn(netdev, "Ignoring unexpected login response\n");
+ return 0;
+ }
+ adapter->login_pending = false;
+
dma_unmap_single(dev, adapter->login_buf_token, adapter->login_buf_sz,
DMA_TO_DEVICE);
dma_unmap_single(dev, adapter->login_rsp_buf_token,
adapter->req_rx_add_queues !=
be32_to_cpu(login_rsp->num_rxadd_subcrqs))) {
dev_err(dev, "FATAL: Inconsistent login and login rsp\n");
- ibmvnic_remove(adapter->vdev);
+ ibmvnic_reset(adapter, VNIC_RESET_FATAL);
return -EIO;
}
size_array = (u64 *)((u8 *)(adapter->login_rsp_buf) +
case IBMVNIC_CRQ_INIT:
dev_info(dev, "Partner initialized\n");
adapter->from_passive_init = true;
+ /* Discard any stale login responses from prev reset.
+ * CHECK: should we clear even on INIT_COMPLETE?
+ */
+ adapter->login_pending = false;
+
if (!completion_done(&adapter->init_done)) {
complete(&adapter->init_done);
adapter->init_done_rc = -EIO;
} while (rc == H_BUSY || H_IS_LONG_BUSY(rc));
/* Clean out the queue */
+ if (!crq->msgs)
+ return -EINVAL;
+
memset(crq->msgs, 0, PAGE_SIZE);
crq->cur = 0;
crq->active = false;
static int ibmvnic_reset_init(struct ibmvnic_adapter *adapter, bool reset)
{
struct device *dev = &adapter->vdev->dev;
- unsigned long timeout = msecs_to_jiffies(30000);
+ unsigned long timeout = msecs_to_jiffies(20000);
u64 old_num_rx_queues, old_num_tx_queues;
int rc;
dev_set_drvdata(&dev->dev, netdev);
adapter->vdev = dev;
adapter->netdev = netdev;
+ adapter->login_pending = false;
ether_addr_copy(adapter->mac_addr, mac_addr_p);
ether_addr_copy(netdev->dev_addr, adapter->mac_addr);
adapter->state = VNIC_PROBED;
adapter->wait_for_reset = false;
-
+ adapter->last_reset_time = jiffies;
return 0;
ibmvnic_register_fail:
unsigned long flags;
spin_lock_irqsave(&adapter->state_lock, flags);
- if (adapter->state == VNIC_RESETTING) {
+ if (test_bit(0, &adapter->resetting)) {
spin_unlock_irqrestore(&adapter->state_lock, flags);
return -EBUSY;
}
#define IBMVNIC_BUFFS_PER_POOL 100
#define IBMVNIC_MAX_QUEUES 16
#define IBMVNIC_MAX_QUEUE_SZ 4096
+#define IBMVNIC_MAX_IND_DESCS 128
+#define IBMVNIC_IND_ARR_SZ (IBMVNIC_MAX_IND_DESCS * 32)
#define IBMVNIC_TSO_BUF_SZ 65536
#define IBMVNIC_TSO_BUFS 64
#define IBMVNIC_TCP_CHKSUM 0x20
#define IBMVNIC_UDP_CHKSUM 0x08
-#define IBMVNIC_MAX_FRAGS_PER_CRQ 3
-
struct ibmvnic_tx_desc {
u8 first;
u8 type;
struct ibmvnic_rx_buff_add_desc rx_add;
};
+struct ibmvnic_ind_xmit_queue {
+ union sub_crq *indir_arr;
+ dma_addr_t indir_dma;
+ int index;
+};
+
struct ibmvnic_sub_crq_queue {
union sub_crq *msgs;
int size, cur;
spinlock_t lock;
struct sk_buff *rx_skb_top;
struct ibmvnic_adapter *adapter;
+ struct ibmvnic_ind_xmit_queue ind_buf;
atomic_t used;
char name[32];
u64 handle;
-};
+} ____cacheline_aligned;
struct ibmvnic_long_term_buff {
unsigned char *buff;
struct ibmvnic_tx_buff {
struct sk_buff *skb;
- dma_addr_t data_dma[IBMVNIC_MAX_FRAGS_PER_CRQ];
- unsigned int data_len[IBMVNIC_MAX_FRAGS_PER_CRQ];
int index;
int pool_index;
- bool last_frag;
- union sub_crq indir_arr[6];
- u8 hdr_data[140];
- dma_addr_t indir_dma;
int num_entries;
};
struct ibmvnic_long_term_buff long_term_buff;
int num_buffers;
int buf_size;
-};
+} ____cacheline_aligned;
struct ibmvnic_rx_buff {
struct sk_buff *skb;
int next_alloc;
int active;
struct ibmvnic_long_term_buff long_term_buff;
-};
+} ____cacheline_aligned;
struct ibmvnic_vpd {
unsigned char *buff;
VNIC_CLOSING,
VNIC_CLOSED,
VNIC_REMOVING,
- VNIC_REMOVED,
- VNIC_RESETTING};
+ VNIC_REMOVED};
enum ibmvnic_reset_reason {VNIC_RESET_FAILOVER = 1,
VNIC_RESET_MOBILITY,
atomic_t running_cap_crqs;
bool wait_capability;
- struct ibmvnic_sub_crq_queue **tx_scrq;
- struct ibmvnic_sub_crq_queue **rx_scrq;
+ struct ibmvnic_sub_crq_queue **tx_scrq ____cacheline_aligned;
+ struct ibmvnic_sub_crq_queue **rx_scrq ____cacheline_aligned;
/* rx structs */
struct napi_struct *napi;
struct delayed_work ibmvnic_delayed_reset;
unsigned long resetting;
bool napi_enabled, from_passive_init;
+ bool login_pending;
+ /* last device reset time */
+ unsigned long last_reset_time;
bool failover_pending;
bool force_reset_recovery;
// SPDX-License-Identifier: GPL-2.0
/* Copyright(c) 2013 - 2019 Intel Corporation. */
+#include <linux/ethtool.h>
#include <linux/vmalloc.h>
#include "fm10k.h"
__I40E_CLIENT_RESET,
__I40E_VIRTCHNL_OP_PENDING,
__I40E_RECOVERY_MODE,
+ __I40E_VF_RESETS_DISABLED, /* disable resets during i40e_remove */
/* This must be last as it determines the size of the BITMAP */
__I40E_STATE_SIZE__,
};
}
if (icr0 & I40E_PFINT_ICR0_VFLR_MASK) {
- ena_mask &= ~I40E_PFINT_ICR0_ENA_VFLR_MASK;
- set_bit(__I40E_VFLR_EVENT_PENDING, pf->state);
+ /* disable any further VFLR event notifications */
+ if (test_bit(__I40E_VF_RESETS_DISABLED, pf->state)) {
+ u32 reg = rd32(hw, I40E_PFINT_ICR0_ENA);
+
+ reg &= ~I40E_PFINT_ICR0_VFLR_MASK;
+ wr32(hw, I40E_PFINT_ICR0_ENA, reg);
+ } else {
+ ena_mask &= ~I40E_PFINT_ICR0_ENA_VFLR_MASK;
+ set_bit(__I40E_VFLR_EVENT_PENDING, pf->state);
+ }
}
if (icr0 & I40E_PFINT_ICR0_GRST_MASK) {
while (test_bit(__I40E_RESET_RECOVERY_PENDING, pf->state))
usleep_range(1000, 2000);
+ if (pf->flags & I40E_FLAG_SRIOV_ENABLED) {
+ set_bit(__I40E_VF_RESETS_DISABLED, pf->state);
+ i40e_free_vfs(pf);
+ pf->flags &= ~I40E_FLAG_SRIOV_ENABLED;
+ }
/* no more scheduling of any task */
set_bit(__I40E_SUSPENDED, pf->state);
set_bit(__I40E_DOWN, pf->state);
*/
i40e_notify_client_of_netdev_close(pf->vsi[pf->lan_vsi], false);
- if (pf->flags & I40E_FLAG_SRIOV_ENABLED) {
- i40e_free_vfs(pf);
- pf->flags &= ~I40E_FLAG_SRIOV_ENABLED;
- }
-
i40e_fdir_teardown(pf);
/* If there is a switch structure or any orphans, remove them.
} else if (vf->link_forced) {
pfe.event_data.link_event.link_status = vf->link_up;
pfe.event_data.link_event.link_speed =
- (vf->link_up ? VIRTCHNL_LINK_SPEED_40GB : 0);
+ (vf->link_up ? i40e_virtchnl_link_speed(ls->link_speed) : 0);
} else {
pfe.event_data.link_event.link_status =
ls->link_info & I40E_AQ_LINK_UP;
* @vf: pointer to the VF structure
* @flr: VFLR was issued or not
*
- * Returns true if the VF is reset, false otherwise.
+ * Returns true if the VF is in reset, resets successfully, or resets
+ * are disabled and false otherwise.
**/
bool i40e_reset_vf(struct i40e_vf *vf, bool flr)
{
u32 reg;
int i;
+ if (test_bit(__I40E_VF_RESETS_DISABLED, pf->state))
+ return true;
+
/* If the VFs have been disabled, this means something else is
* resetting the VF, so we shouldn't continue.
*/
if (test_and_set_bit(__I40E_VF_DISABLE, pf->state))
- return false;
+ return true;
i40e_trigger_vf_reset(vf, flr);
i40e_notify_client_of_vf_enable(pf, 0);
+ /* Disable IOV before freeing resources. This lets any VF drivers
+ * running in the host get themselves cleaned up before we yank
+ * the carpet out from underneath their feet.
+ */
+ if (!pci_vfs_assigned(pf->pdev))
+ pci_disable_sriov(pf->pdev);
+ else
+ dev_warn(&pf->pdev->dev, "VFs are assigned - not disabling SR-IOV\n");
+
/* Amortize wait time by stopping all VFs at the same time */
for (i = 0; i < pf->num_alloc_vfs; i++) {
if (test_bit(I40E_VF_STATE_INIT, &pf->vf[i].vf_states))
i40e_vsi_wait_queues_disabled(pf->vsi[pf->vf[i].lan_vsi_idx]);
}
- /* Disable IOV before freeing resources. This lets any VF drivers
- * running in the host get themselves cleaned up before we yank
- * the carpet out from underneath their feet.
- */
- if (!pci_vfs_assigned(pf->pdev))
- pci_disable_sriov(pf->pdev);
- else
- dev_warn(&pf->pdev->dev, "VFs are assigned - not disabling SR-IOV\n");
-
/* free up VF resources */
tmp = pf->num_alloc_vfs;
pf->num_alloc_vfs = 0;
{
struct i40e_netdev_priv *np = netdev_priv(netdev);
struct i40e_pf *pf = np->vsi->back;
+ struct i40e_link_status *ls = &pf->hw.phy.link_info;
struct virtchnl_pf_event pfe;
struct i40e_hw *hw = &pf->hw;
struct i40e_vf *vf;
vf->link_forced = true;
vf->link_up = true;
pfe.event_data.link_event.link_status = true;
- pfe.event_data.link_event.link_speed = VIRTCHNL_LINK_SPEED_40GB;
+ pfe.event_data.link_event.link_speed = i40e_virtchnl_link_speed(ls->link_speed);
break;
case IFLA_VF_LINK_STATE_DISABLE:
vf->link_forced = true;
continue;
}
- bi = i40e_rx_bi(rx_ring, rx_ring->next_to_clean);
size = (qword & I40E_RXD_QW1_LENGTH_PBUF_MASK) >>
I40E_RXD_QW1_LENGTH_PBUF_SHIFT;
if (!size)
struct netlink_ext_ack *extack)
{
struct ice_pf *pf = devlink_priv(devlink);
- struct device *dev = &pf->pdev->dev;
struct ice_hw *hw = &pf->hw;
- const struct firmware *fw;
u8 preservation;
int err;
if (err)
return err;
- err = request_firmware(&fw, params->file_name, dev);
- if (err) {
- NL_SET_ERR_MSG_MOD(extack, "Unable to read file from disk");
- return err;
- }
-
- dev_dbg(dev, "Beginning flash update with file '%s'\n", params->file_name);
-
- devlink_flash_update_begin_notify(devlink);
devlink_flash_update_status_notify(devlink, "Preparing to flash", NULL, 0, 0);
- err = ice_flash_pldm_image(pf, fw, preservation, extack);
- devlink_flash_update_end_notify(devlink);
-
- release_firmware(fw);
- return err;
+ return ice_flash_pldm_image(pf, params->fw, preservation, extack);
}
static const struct devlink_ops ice_devlink_ops = {
spin_lock_bh(&hw->mbx_lock);
if (hw->mac.ops.set_vfta(hw, vid, true)) {
- dev_err(&adapter->pdev->dev, "Failed to add vlan id %d\n", vid);
+ dev_warn(&adapter->pdev->dev, "Vlan id %d\n is not added", vid);
spin_unlock_bh(&hw->mbx_lock);
return -EINVAL;
}
/* Allow time for pending master requests to run */
if (mac->ops.reset_hw(hw))
- dev_err(&adapter->pdev->dev, "PF still resetting\n");
+ dev_warn(&adapter->pdev->dev, "PF still resetting\n");
mac->ops.init_hw(hw);
static void
mvneta_xdp_put_buff(struct mvneta_port *pp, struct mvneta_rx_queue *rxq,
- struct xdp_buff *xdp, int sync_len, bool napi)
+ struct xdp_buff *xdp, struct skb_shared_info *sinfo,
+ int sync_len)
{
- struct skb_shared_info *sinfo = xdp_get_shared_info_from_buff(xdp);
int i;
for (i = 0; i < sinfo->nr_frags; i++)
page_pool_put_full_page(rxq->page_pool,
- skb_frag_page(&sinfo->frags[i]), napi);
+ skb_frag_page(&sinfo->frags[i]), true);
page_pool_put_page(rxq->page_pool, virt_to_head_page(xdp->data),
- sync_len, napi);
+ sync_len, true);
}
static int
struct bpf_prog *prog, struct xdp_buff *xdp,
u32 frame_sz, struct mvneta_stats *stats)
{
+ struct skb_shared_info *sinfo = xdp_get_shared_info_from_buff(xdp);
unsigned int len, data_len, sync;
u32 ret, act;
err = xdp_do_redirect(pp->dev, xdp, prog);
if (unlikely(err)) {
- mvneta_xdp_put_buff(pp, rxq, xdp, sync, true);
+ mvneta_xdp_put_buff(pp, rxq, xdp, sinfo, sync);
ret = MVNETA_XDP_DROPPED;
} else {
ret = MVNETA_XDP_REDIR;
case XDP_TX:
ret = mvneta_xdp_xmit_back(pp, xdp);
if (ret != MVNETA_XDP_TX)
- mvneta_xdp_put_buff(pp, rxq, xdp, sync, true);
+ mvneta_xdp_put_buff(pp, rxq, xdp, sinfo, sync);
break;
default:
bpf_warn_invalid_xdp_action(act);
trace_xdp_exception(pp->dev, prog, act);
fallthrough;
case XDP_DROP:
- mvneta_xdp_put_buff(pp, rxq, xdp, sync, true);
+ mvneta_xdp_put_buff(pp, rxq, xdp, sinfo, sync);
ret = MVNETA_XDP_DROPPED;
stats->xdp_drop++;
break;
struct mvneta_rx_desc *rx_desc,
struct mvneta_rx_queue *rxq,
struct xdp_buff *xdp, int *size,
+ struct skb_shared_info *xdp_sinfo,
struct page *page)
{
- struct skb_shared_info *sinfo = xdp_get_shared_info_from_buff(xdp);
struct net_device *dev = pp->dev;
enum dma_data_direction dma_dir;
int data_len, len;
dma_sync_single_for_cpu(dev->dev.parent,
rx_desc->buf_phys_addr,
len, dma_dir);
+ rx_desc->buf_phys_addr = 0;
- if (data_len > 0 && sinfo->nr_frags < MAX_SKB_FRAGS) {
- skb_frag_t *frag = &sinfo->frags[sinfo->nr_frags];
+ if (data_len > 0 && xdp_sinfo->nr_frags < MAX_SKB_FRAGS) {
+ skb_frag_t *frag = &xdp_sinfo->frags[xdp_sinfo->nr_frags++];
skb_frag_off_set(frag, pp->rx_offset_correction);
skb_frag_size_set(frag, data_len);
__skb_frag_set_page(frag, page);
- sinfo->nr_frags++;
- rx_desc->buf_phys_addr = 0;
+ /* last fragment */
+ if (len == *size) {
+ struct skb_shared_info *sinfo;
+
+ sinfo = xdp_get_shared_info_from_buff(xdp);
+ sinfo->nr_frags = xdp_sinfo->nr_frags;
+ memcpy(sinfo->frags, xdp_sinfo->frags,
+ sinfo->nr_frags * sizeof(skb_frag_t));
+ }
+ } else {
+ page_pool_put_full_page(rxq->page_pool, page, true);
}
*size -= len;
}
{
int rx_proc = 0, rx_todo, refill, size = 0;
struct net_device *dev = pp->dev;
- struct xdp_buff xdp_buf = {
- .frame_sz = PAGE_SIZE,
- .rxq = &rxq->xdp_rxq,
- };
+ struct skb_shared_info sinfo;
struct mvneta_stats ps = {};
struct bpf_prog *xdp_prog;
u32 desc_status, frame_sz;
+ struct xdp_buff xdp_buf;
+
+ xdp_buf.data_hard_start = NULL;
+ xdp_buf.frame_sz = PAGE_SIZE;
+ xdp_buf.rxq = &rxq->xdp_rxq;
+
+ sinfo.nr_frags = 0;
/* Get number of received packets */
rx_todo = mvneta_rxq_busy_desc_num_get(pp, rxq);
rx_desc->buf_phys_addr = 0;
page_pool_put_full_page(rxq->page_pool, page,
true);
- continue;
+ goto next;
}
mvneta_swbm_add_rx_fragment(pp, rx_desc, rxq, &xdp_buf,
- &size, page);
+ &size, &sinfo, page);
} /* Middle or Last descriptor */
if (!(rx_status & MVNETA_RXD_LAST_DESC))
continue;
if (size) {
- mvneta_xdp_put_buff(pp, rxq, &xdp_buf, -1, true);
+ mvneta_xdp_put_buff(pp, rxq, &xdp_buf, &sinfo, -1);
goto next;
}
if (IS_ERR(skb)) {
struct mvneta_pcpu_stats *stats = this_cpu_ptr(pp->stats);
- mvneta_xdp_put_buff(pp, rxq, &xdp_buf, -1, true);
+ mvneta_xdp_put_buff(pp, rxq, &xdp_buf, &sinfo, -1);
u64_stats_update_begin(&stats->syncp);
stats->es.skb_alloc_error++;
napi_gro_receive(napi, skb);
next:
xdp_buf.data_hard_start = NULL;
+ sinfo.nr_frags = 0;
}
rcu_read_unlock();
if (xdp_buf.data_hard_start)
- mvneta_xdp_put_buff(pp, rxq, &xdp_buf, -1, true);
+ mvneta_xdp_put_buff(pp, rxq, &xdp_buf, &sinfo, -1);
if (ps.xdp_redirect)
xdp_do_flush_map();
/* Maximum number of supported ports */
#define MVPP2_MAX_PORTS 4
+/* Loopback port index */
+#define MVPP2_LOOPBACK_PORT_INDEX 3
+
/* Maximum number of TXQs used by single port */
#define MVPP2_MAX_TXQ 8
#define MVPP2_TX_DESC_ALIGN (MVPP2_DESC_ALIGNED_SIZE - 1)
/* RX FIFO constants */
+#define MVPP2_RX_FIFO_PORT_DATA_SIZE_44KB 0xb000
#define MVPP2_RX_FIFO_PORT_DATA_SIZE_32KB 0x8000
#define MVPP2_RX_FIFO_PORT_DATA_SIZE_8KB 0x2000
#define MVPP2_RX_FIFO_PORT_DATA_SIZE_4KB 0x1000
-#define MVPP2_RX_FIFO_PORT_ATTR_SIZE_32KB 0x200
-#define MVPP2_RX_FIFO_PORT_ATTR_SIZE_8KB 0x80
+#define MVPP2_RX_FIFO_PORT_ATTR_SIZE(data_size) ((data_size) >> 6)
#define MVPP2_RX_FIFO_PORT_ATTR_SIZE_4KB 0x40
#define MVPP2_RX_FIFO_PORT_MIN_PKT 0x80
/* TX FIFO constants */
-#define MVPP22_TX_FIFO_DATA_SIZE_10KB 0xa
-#define MVPP22_TX_FIFO_DATA_SIZE_3KB 0x3
-#define MVPP2_TX_FIFO_THRESHOLD_MIN 256
-#define MVPP2_TX_FIFO_THRESHOLD_10KB \
- (MVPP22_TX_FIFO_DATA_SIZE_10KB * 1024 - MVPP2_TX_FIFO_THRESHOLD_MIN)
-#define MVPP2_TX_FIFO_THRESHOLD_3KB \
- (MVPP22_TX_FIFO_DATA_SIZE_3KB * 1024 - MVPP2_TX_FIFO_THRESHOLD_MIN)
+#define MVPP22_TX_FIFO_DATA_SIZE_16KB 16
+#define MVPP22_TX_FIFO_DATA_SIZE_10KB 10
+#define MVPP22_TX_FIFO_DATA_SIZE_3KB 3
+#define MVPP2_TX_FIFO_THRESHOLD_MIN 256 /* Bytes */
+#define MVPP2_TX_FIFO_THRESHOLD(kb) \
+ ((kb) * 1024 - MVPP2_TX_FIFO_THRESHOLD_MIN)
/* RX buffer constants */
#define MVPP2_SKB_SHINFO_SIZE \
/* List of pointers to port structures */
int port_count;
struct mvpp2_port *port_list[MVPP2_MAX_PORTS];
+ /* Map of enabled ports */
+ unsigned long port_map;
+
struct mvpp2_tai *tai;
/* Number of Tx threads used */
if (!valid) {
netdev_err(port->dev,
"invalid configuration: no dt or link IRQ");
+ err = -ENOENT;
goto err_free_irq;
}
mvpp2_write(priv, MVPP2_RX_FIFO_INIT_REG, 0x1);
}
-static void mvpp22_rx_fifo_init(struct mvpp2 *priv)
+static void mvpp22_rx_fifo_set_hw(struct mvpp2 *priv, int port, int data_size)
{
- int port;
+ int attr_size = MVPP2_RX_FIFO_PORT_ATTR_SIZE(data_size);
- /* The FIFO size parameters are set depending on the maximum speed a
- * given port can handle:
- * - Port 0: 10Gbps
- * - Port 1: 2.5Gbps
- * - Ports 2 and 3: 1Gbps
- */
+ mvpp2_write(priv, MVPP2_RX_DATA_FIFO_SIZE_REG(port), data_size);
+ mvpp2_write(priv, MVPP2_RX_ATTR_FIFO_SIZE_REG(port), attr_size);
+}
- mvpp2_write(priv, MVPP2_RX_DATA_FIFO_SIZE_REG(0),
- MVPP2_RX_FIFO_PORT_DATA_SIZE_32KB);
- mvpp2_write(priv, MVPP2_RX_ATTR_FIFO_SIZE_REG(0),
- MVPP2_RX_FIFO_PORT_ATTR_SIZE_32KB);
+/* Initialize TX FIFO's: the total FIFO size is 48kB on PPv2.2.
+ * 4kB fixed space must be assigned for the loopback port.
+ * Redistribute remaining avialable 44kB space among all active ports.
+ * Guarantee minimum 32kB for 10G port and 8kB for port 1, capable of 2.5G
+ * SGMII link.
+ */
+static void mvpp22_rx_fifo_init(struct mvpp2 *priv)
+{
+ int remaining_ports_count;
+ unsigned long port_map;
+ int size_remainder;
+ int port, size;
+
+ /* The loopback requires fixed 4kB of the FIFO space assignment. */
+ mvpp22_rx_fifo_set_hw(priv, MVPP2_LOOPBACK_PORT_INDEX,
+ MVPP2_RX_FIFO_PORT_DATA_SIZE_4KB);
+ port_map = priv->port_map & ~BIT(MVPP2_LOOPBACK_PORT_INDEX);
+
+ /* Set RX FIFO size to 0 for inactive ports. */
+ for_each_clear_bit(port, &port_map, MVPP2_LOOPBACK_PORT_INDEX)
+ mvpp22_rx_fifo_set_hw(priv, port, 0);
+
+ /* Assign remaining RX FIFO space among all active ports. */
+ size_remainder = MVPP2_RX_FIFO_PORT_DATA_SIZE_44KB;
+ remaining_ports_count = hweight_long(port_map);
+
+ for_each_set_bit(port, &port_map, MVPP2_LOOPBACK_PORT_INDEX) {
+ if (remaining_ports_count == 1)
+ size = size_remainder;
+ else if (port == 0)
+ size = max(size_remainder / remaining_ports_count,
+ MVPP2_RX_FIFO_PORT_DATA_SIZE_32KB);
+ else if (port == 1)
+ size = max(size_remainder / remaining_ports_count,
+ MVPP2_RX_FIFO_PORT_DATA_SIZE_8KB);
+ else
+ size = size_remainder / remaining_ports_count;
- mvpp2_write(priv, MVPP2_RX_DATA_FIFO_SIZE_REG(1),
- MVPP2_RX_FIFO_PORT_DATA_SIZE_8KB);
- mvpp2_write(priv, MVPP2_RX_ATTR_FIFO_SIZE_REG(1),
- MVPP2_RX_FIFO_PORT_ATTR_SIZE_8KB);
+ size_remainder -= size;
+ remaining_ports_count--;
- for (port = 2; port < MVPP2_MAX_PORTS; port++) {
- mvpp2_write(priv, MVPP2_RX_DATA_FIFO_SIZE_REG(port),
- MVPP2_RX_FIFO_PORT_DATA_SIZE_4KB);
- mvpp2_write(priv, MVPP2_RX_ATTR_FIFO_SIZE_REG(port),
- MVPP2_RX_FIFO_PORT_ATTR_SIZE_4KB);
+ mvpp22_rx_fifo_set_hw(priv, port, size);
}
mvpp2_write(priv, MVPP2_RX_MIN_PKT_SIZE_REG,
mvpp2_write(priv, MVPP2_RX_FIFO_INIT_REG, 0x1);
}
-/* Initialize Tx FIFO's: the total FIFO size is 19kB on PPv2.2 and 10G
- * interfaces must have a Tx FIFO size of 10kB. As only port 0 can do 10G,
- * configure its Tx FIFO size to 10kB and the others ports Tx FIFO size to 3kB.
+static void mvpp22_tx_fifo_set_hw(struct mvpp2 *priv, int port, int size)
+{
+ int threshold = MVPP2_TX_FIFO_THRESHOLD(size);
+
+ mvpp2_write(priv, MVPP22_TX_FIFO_SIZE_REG(port), size);
+ mvpp2_write(priv, MVPP22_TX_FIFO_THRESH_REG(port), threshold);
+}
+
+/* Initialize TX FIFO's: the total FIFO size is 19kB on PPv2.2.
+ * 3kB fixed space must be assigned for the loopback port.
+ * Redistribute remaining avialable 16kB space among all active ports.
+ * The 10G interface should use 10kB (which is maximum possible size
+ * per single port).
*/
static void mvpp22_tx_fifo_init(struct mvpp2 *priv)
{
- int port, size, thrs;
-
- for (port = 0; port < MVPP2_MAX_PORTS; port++) {
- if (port == 0) {
+ int remaining_ports_count;
+ unsigned long port_map;
+ int size_remainder;
+ int port, size;
+
+ /* The loopback requires fixed 3kB of the FIFO space assignment. */
+ mvpp22_tx_fifo_set_hw(priv, MVPP2_LOOPBACK_PORT_INDEX,
+ MVPP22_TX_FIFO_DATA_SIZE_3KB);
+ port_map = priv->port_map & ~BIT(MVPP2_LOOPBACK_PORT_INDEX);
+
+ /* Set TX FIFO size to 0 for inactive ports. */
+ for_each_clear_bit(port, &port_map, MVPP2_LOOPBACK_PORT_INDEX)
+ mvpp22_tx_fifo_set_hw(priv, port, 0);
+
+ /* Assign remaining TX FIFO space among all active ports. */
+ size_remainder = MVPP22_TX_FIFO_DATA_SIZE_16KB;
+ remaining_ports_count = hweight_long(port_map);
+
+ for_each_set_bit(port, &port_map, MVPP2_LOOPBACK_PORT_INDEX) {
+ if (remaining_ports_count == 1)
+ size = min(size_remainder,
+ MVPP22_TX_FIFO_DATA_SIZE_10KB);
+ else if (port == 0)
size = MVPP22_TX_FIFO_DATA_SIZE_10KB;
- thrs = MVPP2_TX_FIFO_THRESHOLD_10KB;
- } else {
- size = MVPP22_TX_FIFO_DATA_SIZE_3KB;
- thrs = MVPP2_TX_FIFO_THRESHOLD_3KB;
- }
- mvpp2_write(priv, MVPP22_TX_FIFO_SIZE_REG(port), size);
- mvpp2_write(priv, MVPP22_TX_FIFO_THRESH_REG(port), thrs);
+ else
+ size = size_remainder / remaining_ports_count;
+
+ size_remainder -= size;
+ remaining_ports_count--;
+
+ mvpp22_tx_fifo_set_hw(priv, port, size);
}
}
goto err_axi_clk;
}
+ /* Map DTS-active ports. Should be done before FIFO mvpp2_init */
+ fwnode_for_each_available_child_node(fwnode, port_fwnode) {
+ if (!fwnode_property_read_u32(port_fwnode, "port-id", &i))
+ priv->port_map |= BIT(i);
+ }
+
/* Initialize network controller */
err = mvpp2_init(pdev, priv);
if (err < 0) {
octeontx2_mbox-y := mbox.o rvu_trace.o
octeontx2_af-y := cgx.o rvu.o rvu_cgx.o rvu_npa.o rvu_nix.o \
- rvu_reg.o rvu_npc.o rvu_debugfs.o ptp.o
+ rvu_reg.o rvu_npc.o rvu_debugfs.o ptp.o rvu_npc_fs.o \
+ rvu_cpt.o
#define NIX_RX_ACTIONOP_UCAST_IPSEC (0x2ull)
#define NIX_RX_ACTIONOP_MCAST (0x3ull)
#define NIX_RX_ACTIONOP_RSS (0x4ull)
+/* Use the RX action set in the default unicast entry */
+#define NIX_RX_ACTION_DEFAULT (0xfull)
/* NIX TX action operation*/
#define NIX_TX_ACTIONOP_DROP (0x0ull)
/* SSO/SSOW mbox IDs (range 0x600 - 0x7FF) */ \
/* TIM mbox IDs (range 0x800 - 0x9FF) */ \
/* CPT mbox IDs (range 0xA00 - 0xBFF) */ \
+M(CPT_LF_ALLOC, 0xA00, cpt_lf_alloc, cpt_lf_alloc_req_msg, \
+ msg_rsp) \
+M(CPT_LF_FREE, 0xA01, cpt_lf_free, msg_req, msg_rsp) \
+M(CPT_RD_WR_REGISTER, 0xA02, cpt_rd_wr_register, cpt_rd_wr_reg_msg, \
+ cpt_rd_wr_reg_msg) \
/* NPC mbox IDs (range 0x6000 - 0x7FFF) */ \
M(NPC_MCAM_ALLOC_ENTRY, 0x6000, npc_mcam_alloc_entry, npc_mcam_alloc_entry_req,\
npc_mcam_alloc_entry_rsp) \
npc_mcam_alloc_and_write_entry_rsp) \
M(NPC_GET_KEX_CFG, 0x600c, npc_get_kex_cfg, \
msg_req, npc_get_kex_cfg_rsp) \
+M(NPC_INSTALL_FLOW, 0x600d, npc_install_flow, \
+ npc_install_flow_req, npc_install_flow_rsp) \
+M(NPC_DELETE_FLOW, 0x600e, npc_delete_flow, \
+ npc_delete_flow_req, msg_rsp) \
+M(NPC_MCAM_READ_ENTRY, 0x600f, npc_mcam_read_entry, \
+ npc_mcam_read_entry_req, \
+ npc_mcam_read_entry_rsp) \
+M(NPC_MCAM_READ_BASE_RULE, 0x6011, npc_read_base_steer_rule, \
+ msg_req, npc_mcam_read_base_rule_rsp) \
/* NIX mbox IDs (range 0x8000 - 0xFFFF) */ \
M(NIX_LF_ALLOC, 0x8000, nix_lf_alloc, \
nix_lf_alloc_req, nix_lf_alloc_rsp) \
-M(NIX_LF_FREE, 0x8001, nix_lf_free, msg_req, msg_rsp) \
+M(NIX_LF_FREE, 0x8001, nix_lf_free, nix_lf_free_req, msg_rsp) \
M(NIX_AQ_ENQ, 0x8002, nix_aq_enq, nix_aq_enq_req, nix_aq_enq_rsp) \
M(NIX_HWCTX_DISABLE, 0x8003, nix_hwctx_disable, \
hwctx_disable_req, msg_rsp) \
M(NIX_TXSCH_FREE, 0x8005, nix_txsch_free, nix_txsch_free_req, msg_rsp) \
M(NIX_TXSCHQ_CFG, 0x8006, nix_txschq_cfg, nix_txschq_config, msg_rsp) \
M(NIX_STATS_RST, 0x8007, nix_stats_rst, msg_req, msg_rsp) \
-M(NIX_VTAG_CFG, 0x8008, nix_vtag_cfg, nix_vtag_config, msg_rsp) \
+M(NIX_VTAG_CFG, 0x8008, nix_vtag_cfg, nix_vtag_config, \
+ nix_vtag_config_rsp) \
M(NIX_RSS_FLOWKEY_CFG, 0x8009, nix_rss_flowkey_cfg, \
nix_rss_flowkey_cfg, \
nix_rss_flowkey_cfg_rsp) \
M(NIX_LSO_FORMAT_CFG, 0x8011, nix_lso_format_cfg, \
nix_lso_format_cfg, \
nix_lso_format_cfg_rsp) \
-M(NIX_RXVLAN_ALLOC, 0x8012, nix_rxvlan_alloc, msg_req, msg_rsp) \
M(NIX_LF_PTP_TX_ENABLE, 0x8013, nix_lf_ptp_tx_enable, msg_req, msg_rsp) \
M(NIX_LF_PTP_TX_DISABLE, 0x8014, nix_lf_ptp_tx_disable, msg_req, msg_rsp) \
M(NIX_BP_ENABLE, 0x8016, nix_bp_enable, nix_bp_cfg_req, \
NIX_AF_ERR_LSO_CFG_FAIL = -418,
NIX_AF_INVAL_NPA_PF_FUNC = -419,
NIX_AF_INVAL_SSO_PF_FUNC = -420,
+ NIX_AF_ERR_TX_VTAG_NOSPC = -421,
+ NIX_AF_ERR_RX_VTAG_INUSE = -422,
+};
+
+/* For NIX RX vtag action */
+enum nix_rx_vtag0_type {
+ NIX_AF_LFX_RX_VTAG_TYPE0, /* reserved for rx vlan offload */
+ NIX_AF_LFX_RX_VTAG_TYPE1,
+ NIX_AF_LFX_RX_VTAG_TYPE2,
+ NIX_AF_LFX_RX_VTAG_TYPE3,
+ NIX_AF_LFX_RX_VTAG_TYPE4,
+ NIX_AF_LFX_RX_VTAG_TYPE5,
+ NIX_AF_LFX_RX_VTAG_TYPE6,
+ NIX_AF_LFX_RX_VTAG_TYPE7,
};
/* For NIX LF context alloc and init */
u8 sdp_links; /* No. of SDP links present in HW */
};
+struct nix_lf_free_req {
+ struct mbox_msghdr hdr;
+#define NIX_LF_DISABLE_FLOWS BIT_ULL(0)
+#define NIX_LF_DONT_FREE_TX_VTAG BIT_ULL(1)
+ u64 flags;
+};
+
/* NIX AQ enqueue msg */
struct nix_aq_enq_req {
struct mbox_msghdr hdr;
union {
/* valid when cfg_type is '0' */
struct {
- /* tx vlan0 tag(C-VLAN) */
- u64 vlan0;
- /* tx vlan1 tag(S-VLAN) */
- u64 vlan1;
- /* insert tx vlan tag */
- u8 insert_vlan :1;
- /* insert tx double vlan tag */
- u8 double_vlan :1;
+ u64 vtag0;
+ u64 vtag1;
+
+ /* cfg_vtag0 & cfg_vtag1 fields are valid
+ * when free_vtag0 & free_vtag1 are '0's.
+ */
+ /* cfg_vtag0 = 1 to configure vtag0 */
+ u8 cfg_vtag0 :1;
+ /* cfg_vtag1 = 1 to configure vtag1 */
+ u8 cfg_vtag1 :1;
+
+ /* vtag0_idx & vtag1_idx are only valid when
+ * both cfg_vtag0 & cfg_vtag1 are '0's,
+ * these fields are used along with free_vtag0
+ * & free_vtag1 to free the nix lf's tx_vlan
+ * configuration.
+ *
+ * Denotes the indices of tx_vtag def registers
+ * that needs to be cleared and freed.
+ */
+ int vtag0_idx;
+ int vtag1_idx;
+
+ /* free_vtag0 & free_vtag1 fields are valid
+ * when cfg_vtag0 & cfg_vtag1 are '0's.
+ */
+ /* free_vtag0 = 1 clears vtag0 configuration
+ * vtag0_idx denotes the index to be cleared.
+ */
+ u8 free_vtag0 :1;
+ /* free_vtag1 = 1 clears vtag1 configuration
+ * vtag1_idx denotes the index to be cleared.
+ */
+ u8 free_vtag1 :1;
} tx;
/* valid when cfg_type is '1' */
};
};
+struct nix_vtag_config_rsp {
+ struct mbox_msghdr hdr;
+ int vtag0_idx;
+ int vtag1_idx;
+ /* Indices of tx_vtag def registers used to configure
+ * tx vtag0 & vtag1 headers, these indices are valid
+ * when nix_vtag_config mbox requested for vtag0 and/
+ * or vtag1 configuration.
+ */
+};
+
struct nix_rss_flowkey_cfg {
struct mbox_msghdr hdr;
int mcam_index; /* MCAM entry index to modify */
#define NIX_FLOW_KEY_TYPE_INNR_SCTP BIT(16)
#define NIX_FLOW_KEY_TYPE_INNR_ETH_DMAC BIT(17)
#define NIX_FLOW_KEY_TYPE_VLAN BIT(20)
+#define NIX_FLOW_KEY_TYPE_IPV4_PROTO BIT(21)
u32 flowkey_cfg; /* Flowkey types selected */
u8 group; /* RSS context or group */
};
u8 mkex_pfl_name[MKEX_NAME_LEN];
};
+struct flow_msg {
+ unsigned char dmac[6];
+ unsigned char smac[6];
+ __be16 etype;
+ __be16 vlan_etype;
+ __be16 vlan_tci;
+ union {
+ __be32 ip4src;
+ __be32 ip6src[4];
+ };
+ union {
+ __be32 ip4dst;
+ __be32 ip6dst[4];
+ };
+ u8 tos;
+ u8 ip_ver;
+ u8 ip_proto;
+ u8 tc;
+ __be16 sport;
+ __be16 dport;
+};
+
+struct npc_install_flow_req {
+ struct mbox_msghdr hdr;
+ struct flow_msg packet;
+ struct flow_msg mask;
+ u64 features;
+ u16 entry;
+ u16 channel;
+ u8 intf;
+ u8 set_cntr; /* If counter is available set counter for this entry ? */
+ u8 default_rule;
+ u8 append; /* overwrite(0) or append(1) flow to default rule? */
+ u16 vf;
+ /* action */
+ u32 index;
+ u16 match_id;
+ u8 flow_key_alg;
+ u8 op;
+ /* vtag rx action */
+ u8 vtag0_type;
+ u8 vtag0_valid;
+ u8 vtag1_type;
+ u8 vtag1_valid;
+ /* vtag tx action */
+ u16 vtag0_def;
+ u8 vtag0_op;
+ u16 vtag1_def;
+ u8 vtag1_op;
+};
+
+struct npc_install_flow_rsp {
+ struct mbox_msghdr hdr;
+ int counter; /* negative if no counter else counter number */
+};
+
+struct npc_delete_flow_req {
+ struct mbox_msghdr hdr;
+ u16 entry;
+ u16 start;/*Disable range of entries */
+ u16 end;
+ u8 all; /* PF + VFs */
+};
+
+struct npc_mcam_read_entry_req {
+ struct mbox_msghdr hdr;
+ u16 entry; /* MCAM entry to read */
+};
+
+struct npc_mcam_read_entry_rsp {
+ struct mbox_msghdr hdr;
+ struct mcam_entry entry_data;
+ u8 intf;
+ u8 enable;
+};
+
+struct npc_mcam_read_base_rule_rsp {
+ struct mbox_msghdr hdr;
+ struct mcam_entry entry;
+};
+
enum ptp_op {
PTP_OP_ADJFINE = 0,
PTP_OP_GET_CLOCK = 1,
u64 clk;
};
+/* CPT mailbox error codes
+ * Range 901 - 1000.
+ */
+enum cpt_af_status {
+ CPT_AF_ERR_PARAM = -901,
+ CPT_AF_ERR_GRP_INVALID = -902,
+ CPT_AF_ERR_LF_INVALID = -903,
+ CPT_AF_ERR_ACCESS_DENIED = -904,
+ CPT_AF_ERR_SSO_PF_FUNC_INVALID = -905,
+ CPT_AF_ERR_NIX_PF_FUNC_INVALID = -906
+};
+
+/* CPT mbox message formats */
+struct cpt_rd_wr_reg_msg {
+ struct mbox_msghdr hdr;
+ u64 reg_offset;
+ u64 *ret_val;
+ u64 val;
+ u8 is_write;
+};
+
+struct cpt_lf_alloc_req_msg {
+ struct mbox_msghdr hdr;
+ u16 nix_pf_func;
+ u16 sso_pf_func;
+ u16 eng_grpmsk;
+};
+
#endif /* MBOX_H */
NPC_LT_LH_CUSTOM1 = 0xF,
};
+/* NPC port kind defines how the incoming or outgoing packets
+ * are processed. NPC accepts packets from up to 64 pkinds.
+ * Software assigns pkind for each incoming port such as CGX
+ * Ethernet interfaces, LBK interfaces, etc.
+ */
+enum npc_pkind_type {
+ NPC_TX_DEF_PKIND = 63ULL, /* NIX-TX PKIND */
+};
+
+/* list of known and supported fields in packet header and
+ * fields present in key structure.
+ */
+enum key_fields {
+ NPC_DMAC,
+ NPC_SMAC,
+ NPC_ETYPE,
+ NPC_OUTER_VID,
+ NPC_TOS,
+ NPC_SIP_IPV4,
+ NPC_DIP_IPV4,
+ NPC_SIP_IPV6,
+ NPC_DIP_IPV6,
+ NPC_SPORT_TCP,
+ NPC_DPORT_TCP,
+ NPC_SPORT_UDP,
+ NPC_DPORT_UDP,
+ NPC_SPORT_SCTP,
+ NPC_DPORT_SCTP,
+ NPC_HEADER_FIELDS_MAX,
+ NPC_CHAN = NPC_HEADER_FIELDS_MAX, /* Valid when Rx */
+ NPC_PF_FUNC, /* Valid when Tx */
+ NPC_ERRLEV,
+ NPC_ERRCODE,
+ NPC_LXMB,
+ NPC_LA,
+ NPC_LB,
+ NPC_LC,
+ NPC_LD,
+ NPC_LE,
+ NPC_LF,
+ NPC_LG,
+ NPC_LH,
+ /* Ethertype for untagged frame */
+ NPC_ETYPE_ETHER,
+ /* Ethertype for single tagged frame */
+ NPC_ETYPE_TAG1,
+ /* Ethertype for double tagged frame */
+ NPC_ETYPE_TAG2,
+ /* outer vlan tci for single tagged frame */
+ NPC_VLAN_TAG1,
+ /* outer vlan tci for double tagged frame */
+ NPC_VLAN_TAG2,
+ /* other header fields programmed to extract but not of our interest */
+ NPC_UNKNOWN,
+ NPC_KEY_FIELDS_MAX,
+};
+
struct npc_kpu_profile_cam {
u8 state;
u8 state_mask;
/* NPC_AF_INTFX_KEX_CFG field masks */
#define NPC_PARSE_NIBBLE GENMASK_ULL(30, 0)
+/* NPC_PARSE_KEX_S nibble definitions for each field */
+#define NPC_PARSE_NIBBLE_CHAN GENMASK_ULL(2, 0)
+#define NPC_PARSE_NIBBLE_ERRLEV BIT_ULL(3)
+#define NPC_PARSE_NIBBLE_ERRCODE GENMASK_ULL(5, 4)
+#define NPC_PARSE_NIBBLE_L2L3_BCAST BIT_ULL(6)
+#define NPC_PARSE_NIBBLE_LA_FLAGS GENMASK_ULL(8, 7)
+#define NPC_PARSE_NIBBLE_LA_LTYPE BIT_ULL(9)
+#define NPC_PARSE_NIBBLE_LB_FLAGS GENMASK_ULL(11, 10)
+#define NPC_PARSE_NIBBLE_LB_LTYPE BIT_ULL(12)
+#define NPC_PARSE_NIBBLE_LC_FLAGS GENMASK_ULL(14, 13)
+#define NPC_PARSE_NIBBLE_LC_LTYPE BIT_ULL(15)
+#define NPC_PARSE_NIBBLE_LD_FLAGS GENMASK_ULL(17, 16)
+#define NPC_PARSE_NIBBLE_LD_LTYPE BIT_ULL(18)
+#define NPC_PARSE_NIBBLE_LE_FLAGS GENMASK_ULL(20, 19)
+#define NPC_PARSE_NIBBLE_LE_LTYPE BIT_ULL(21)
+#define NPC_PARSE_NIBBLE_LF_FLAGS GENMASK_ULL(23, 22)
+#define NPC_PARSE_NIBBLE_LF_LTYPE BIT_ULL(24)
+#define NPC_PARSE_NIBBLE_LG_FLAGS GENMASK_ULL(26, 25)
+#define NPC_PARSE_NIBBLE_LG_LTYPE BIT_ULL(27)
+#define NPC_PARSE_NIBBLE_LH_FLAGS GENMASK_ULL(29, 28)
+#define NPC_PARSE_NIBBLE_LH_LTYPE BIT_ULL(30)
+
+struct nix_tx_action {
+#if defined(__BIG_ENDIAN_BITFIELD)
+ u64 rsvd_63_48 :16;
+ u64 match_id :16;
+ u64 index :20;
+ u64 rsvd_11_8 :8;
+ u64 op :4;
+#else
+ u64 op :4;
+ u64 rsvd_11_8 :8;
+ u64 index :20;
+ u64 match_id :16;
+ u64 rsvd_63_48 :16;
+#endif
+};
+
/* NIX Receive Vtag Action Structure */
-#define VTAG0_VALID_BIT BIT_ULL(15)
-#define VTAG0_TYPE_MASK GENMASK_ULL(14, 12)
-#define VTAG0_LID_MASK GENMASK_ULL(10, 8)
-#define VTAG0_RELPTR_MASK GENMASK_ULL(7, 0)
+#define RX_VTAG0_VALID_BIT BIT_ULL(15)
+#define RX_VTAG0_TYPE_MASK GENMASK_ULL(14, 12)
+#define RX_VTAG0_LID_MASK GENMASK_ULL(10, 8)
+#define RX_VTAG0_RELPTR_MASK GENMASK_ULL(7, 0)
+#define RX_VTAG1_VALID_BIT BIT_ULL(47)
+#define RX_VTAG1_TYPE_MASK GENMASK_ULL(46, 44)
+#define RX_VTAG1_LID_MASK GENMASK_ULL(42, 40)
+#define RX_VTAG1_RELPTR_MASK GENMASK_ULL(39, 32)
+
+/* NIX Transmit Vtag Action Structure */
+#define TX_VTAG0_DEF_MASK GENMASK_ULL(25, 16)
+#define TX_VTAG0_OP_MASK GENMASK_ULL(13, 12)
+#define TX_VTAG0_LID_MASK GENMASK_ULL(10, 8)
+#define TX_VTAG0_RELPTR_MASK GENMASK_ULL(7, 0)
+#define TX_VTAG1_DEF_MASK GENMASK_ULL(57, 48)
+#define TX_VTAG1_OP_MASK GENMASK_ULL(45, 44)
+#define TX_VTAG1_LID_MASK GENMASK_ULL(42, 40)
+#define TX_VTAG1_RELPTR_MASK GENMASK_ULL(39, 32)
struct npc_mcam_kex {
/* MKEX Profle Header */
struct npc_lt_def pck_iip4;
};
+struct rvu_npc_mcam_rule {
+ struct flow_msg packet;
+ struct flow_msg mask;
+ u8 intf;
+ union {
+ struct nix_tx_action tx_action;
+ struct nix_rx_action rx_action;
+ };
+ u64 vtag_action;
+ struct list_head list;
+ u64 features;
+ u16 owner;
+ u16 entry;
+ u16 cntr;
+ bool has_cntr;
+ u8 default_rule;
+ bool enable;
+ bool vfvlan_cfg;
+};
+
#endif /* NPC_H */
(((bytesm1) << 16) | ((hdr_ofs) << 8) | ((ena) << 7) | \
((flags_ena) << 6) | ((key_ofs) & 0x3F))
+/* Rx parse key extract nibble enable */
+#define NPC_PARSE_NIBBLE_INTF_RX (NPC_PARSE_NIBBLE_CHAN | \
+ NPC_PARSE_NIBBLE_LA_LTYPE | \
+ NPC_PARSE_NIBBLE_LB_LTYPE | \
+ NPC_PARSE_NIBBLE_LC_LTYPE | \
+ NPC_PARSE_NIBBLE_LD_LTYPE | \
+ NPC_PARSE_NIBBLE_LE_LTYPE)
+/* Tx parse key extract nibble enable */
+#define NPC_PARSE_NIBBLE_INTF_TX (NPC_PARSE_NIBBLE_LA_LTYPE | \
+ NPC_PARSE_NIBBLE_LB_LTYPE | \
+ NPC_PARSE_NIBBLE_LC_LTYPE | \
+ NPC_PARSE_NIBBLE_LD_LTYPE | \
+ NPC_PARSE_NIBBLE_LE_LTYPE)
+
enum npc_kpu_parser_state {
NPC_S_NA = 0,
NPC_S_KPU1_ETHER,
.name = "default",
.kpu_version = NPC_KPU_PROFILE_VER,
.keyx_cfg = {
- /* nibble: LA..LE (ltype only) + Channel */
- [NIX_INTF_RX] = ((u64)NPC_MCAM_KEY_X2 << 32) | 0x49247,
- [NIX_INTF_TX] = ((u64)NPC_MCAM_KEY_X2 << 32) | ((1ULL << 19) - 1),
+ /* nibble: LA..LE (ltype only) + channel */
+ [NIX_INTF_RX] = ((u64)NPC_MCAM_KEY_X2 << 32) | NPC_PARSE_NIBBLE_INTF_RX,
+ /* nibble: LA..LE (ltype only) */
+ [NIX_INTF_TX] = ((u64)NPC_MCAM_KEY_X2 << 32) | NPC_PARSE_NIBBLE_INTF_TX,
},
.intf_lid_lt_ld = {
/* Default RX MCAM KEX profile */
/* Layer B: Single VLAN (CTAG) */
/* CTAG VLAN[2..3] + Ethertype, 4 bytes, KW0[63:32] */
[NPC_LT_LB_CTAG] = {
- KEX_LD_CFG(0x03, 0x0, 0x1, 0x0, 0x4),
+ KEX_LD_CFG(0x03, 0x2, 0x1, 0x0, 0x4),
},
/* Layer B: Stacked VLAN (STAG|QinQ) */
[NPC_LT_LB_STAG_QINQ] = {
- /* CTAG VLAN[2..3] + Ethertype, 4 bytes, KW0[63:32] */
- KEX_LD_CFG(0x03, 0x4, 0x1, 0x0, 0x4),
+ /* Outer VLAN: 2 bytes, KW0[63:48] */
+ KEX_LD_CFG(0x01, 0x2, 0x1, 0x0, 0x6),
+ /* Ethertype: 2 bytes, KW0[47:32] */
+ KEX_LD_CFG(0x01, 0x8, 0x1, 0x0, 0x4),
},
[NPC_LT_LB_FDSA] = {
/* SWITCH PORT: 1 byte, KW0[63:48] */
[NPC_LID_LD] = {
/* Layer D:UDP */
[NPC_LT_LD_UDP] = {
- /* SPORT: 2 bytes, KW3[15:0] */
- KEX_LD_CFG(0x1, 0x0, 0x1, 0x0, 0x18),
- /* DPORT: 2 bytes, KW3[31:16] */
- KEX_LD_CFG(0x1, 0x2, 0x1, 0x0, 0x1a),
+ /* SPORT+DPORT: 4 bytes, KW3[31:0] */
+ KEX_LD_CFG(0x3, 0x0, 0x1, 0x0, 0x18),
+ },
+ /* Layer D:TCP */
+ [NPC_LT_LD_TCP] = {
+ /* SPORT+DPORT: 4 bytes, KW3[31:0] */
+ KEX_LD_CFG(0x3, 0x0, 0x1, 0x0, 0x18),
+ },
+ },
+ },
+
+ /* Default TX MCAM KEX profile */
+ [NIX_INTF_TX] = {
+ [NPC_LID_LA] = {
+ /* Layer A: NIX_INST_HDR_S + Ethernet */
+ /* NIX appends 8 bytes of NIX_INST_HDR_S at the
+ * start of each TX packet supplied to NPC.
+ */
+ [NPC_LT_LA_IH_NIX_ETHER] = {
+ /* PF_FUNC: 2B , KW0 [47:32] */
+ KEX_LD_CFG(0x01, 0x0, 0x1, 0x0, 0x4),
+ /* DMAC: 6 bytes, KW1[63:16] */
+ KEX_LD_CFG(0x05, 0x8, 0x1, 0x0, 0xa),
+ },
+ },
+ [NPC_LID_LB] = {
+ /* Layer B: Single VLAN (CTAG) */
+ [NPC_LT_LB_CTAG] = {
+ /* CTAG VLAN[2..3] KW0[63:48] */
+ KEX_LD_CFG(0x01, 0x2, 0x1, 0x0, 0x6),
+ /* CTAG VLAN[2..3] KW1[15:0] */
+ KEX_LD_CFG(0x01, 0x4, 0x1, 0x0, 0x8),
+ },
+ /* Layer B: Stacked VLAN (STAG|QinQ) */
+ [NPC_LT_LB_STAG_QINQ] = {
+ /* Outer VLAN: 2 bytes, KW0[63:48] */
+ KEX_LD_CFG(0x01, 0x2, 0x1, 0x0, 0x6),
+ /* Outer VLAN: 2 Bytes, KW1[15:0] */
+ KEX_LD_CFG(0x01, 0x8, 0x1, 0x0, 0x8),
+ },
+ },
+ [NPC_LID_LC] = {
+ /* Layer C: IPv4 */
+ [NPC_LT_LC_IP] = {
+ /* SIP+DIP: 8 bytes, KW2[63:0] */
+ KEX_LD_CFG(0x07, 0xc, 0x1, 0x0, 0x10),
+ /* TOS: 1 byte, KW1[63:56] */
+ KEX_LD_CFG(0x0, 0x1, 0x1, 0x0, 0xf),
+ },
+ /* Layer C: IPv6 */
+ [NPC_LT_LC_IP6] = {
+ /* Everything up to SADDR: 8 bytes, KW2[63:0] */
+ KEX_LD_CFG(0x07, 0x0, 0x1, 0x0, 0x10),
+ },
+ },
+ [NPC_LID_LD] = {
+ /* Layer D:UDP */
+ [NPC_LT_LD_UDP] = {
+ /* SPORT+DPORT: 4 bytes, KW3[31:0] */
+ KEX_LD_CFG(0x3, 0x0, 0x1, 0x0, 0x18),
},
/* Layer D:TCP */
[NPC_LT_LD_TCP] = {
- /* SPORT: 2 bytes, KW3[15:0] */
- KEX_LD_CFG(0x1, 0x0, 0x1, 0x0, 0x18),
- /* DPORT: 2 bytes, KW3[31:16] */
- KEX_LD_CFG(0x1, 0x2, 0x1, 0x0, 0x1a),
+ /* SPORT+DPORT: 4 bytes, KW3[31:0] */
+ KEX_LD_CFG(0x3, 0x0, 0x1, 0x0, 0x18),
},
},
},
u64 *mac;
for (pf = 0; pf < hw->total_pfs; pf++) {
+ /* For PF0(AF), Assign MAC address to only VFs (LBKVFs) */
+ if (!pf)
+ goto lbkvf;
+
if (!is_pf_cgxmapped(rvu, pf))
continue;
/* Assign MAC address to PF */
} else {
eth_random_addr(pfvf->mac_addr);
}
+ ether_addr_copy(pfvf->default_mac, pfvf->mac_addr);
- /* Assign MAC address to VFs */
+lbkvf:
+ /* Assign MAC address to VFs*/
rvu_get_pf_numvfs(rvu, pf, &numvfs, &hwvf);
for (vf = 0; vf < numvfs; vf++, hwvf++) {
pfvf = &rvu->hwvf[hwvf];
} else {
eth_random_addr(pfvf->mac_addr);
}
+ ether_addr_copy(pfvf->default_mac, pfvf->mac_addr);
}
}
}
if (blkaddr < 0)
return;
+ if (blktype == BLKTYPE_NIX)
+ rvu_nix_reset_mac(pfvf, pcifunc);
+
block = &hw->block[blkaddr];
num_lfs = rvu_get_rsrc_mapcount(pfvf, block->addr);
#define PCI_DEVID_OCTEONTX2_LBK 0xA061
-static int lbk_get_num_chans(void)
+int rvu_get_num_lbk_chans(void)
{
struct pci_dev *pdev;
void __iomem *base;
return 0;
}
- chans = lbk_get_num_chans();
+ chans = rvu_get_num_lbk_chans();
if (chans < 0)
return chans;
#include "rvu_struct.h"
#include "common.h"
#include "mbox.h"
+#include "npc.h"
/* PCI device IDs */
#define PCI_DEVID_OCTEONTX2_RVU_AF 0xA065
struct dentry *npa;
struct dentry *nix;
struct dentry *npc;
+ struct dentry *cpt;
struct dump_ctx npa_aura_ctx;
struct dump_ctx npa_pool_ctx;
struct dump_ctx nix_cq_ctx;
int max;
};
+/* layer metadata to uniquely identify a packet header field */
+struct npc_layer_mdata {
+ u8 lid;
+ u8 ltype;
+ u8 hdr;
+ u8 key;
+ u8 len;
+};
+
+/* Structure to represent a field present in the
+ * generated key. A key field may present anywhere and can
+ * be of any size in the generated key. Once this structure
+ * is populated for fields of interest then field's presence
+ * and location (if present) can be known.
+ */
+struct npc_key_field {
+ /* Masks where all set bits indicate position
+ * of a field in the key
+ */
+ u64 kw_mask[NPC_MAX_KWS_IN_KEY];
+ /* Number of words in the key a field spans. If a field is
+ * of 16 bytes and key offset is 4 then the field will use
+ * 4 bytes in KW0, 8 bytes in KW1 and 4 bytes in KW2 and
+ * nr_kws will be 3(KW0, KW1 and KW2).
+ */
+ int nr_kws;
+ /* used by packet header fields */
+ struct npc_layer_mdata layer_mdata;
+};
+
struct npc_mcam {
struct rsrc_bmap counters;
struct mutex lock; /* MCAM entries and counters update lock */
u16 *entry2cntr_map;
u16 *cntr2pfvf_map;
u16 *cntr_refcnt;
+ u16 *entry2target_pffunc;
u8 keysize; /* MCAM keysize 112/224/448 bits */
u8 banks; /* Number of MCAM banks */
u8 banks_per_entry;/* Number of keywords in key */
u16 hprio_count;
u16 hprio_end;
u16 rx_miss_act_cntr; /* Counter for RX MISS action */
+ /* fields present in the generated key */
+ struct npc_key_field tx_key_fields[NPC_KEY_FIELDS_MAX];
+ struct npc_key_field rx_key_fields[NPC_KEY_FIELDS_MAX];
+ u64 tx_features;
+ u64 rx_features;
+ struct list_head mcam_rules;
};
/* Structure for per RVU func info ie PF/VF */
u16 maxlen;
u16 minlen;
+ u8 pf_set_vf_cfg;
u8 mac_addr[ETH_ALEN]; /* MAC address of this PF/VF */
+ u8 default_mac[ETH_ALEN]; /* MAC address from FWdata */
/* Broadcast pkt replication info */
u16 bcast_mce_idx;
struct nix_mce_list bcast_mce_list;
- /* VLAN offload */
- struct mcam_entry entry;
- int rxvlan_index;
- bool rxvlan;
+ struct rvu_npc_mcam_rule *def_ucast_rule;
bool cgx_in_use; /* this PF/VF using CGX? */
int cgx_users; /* number of cgx users - used only by PFs */
u8 in_use;
};
+struct nix_txvlan {
+#define NIX_TX_VTAG_DEF_MAX 0x400
+ struct rsrc_bmap rsrc;
+ u16 *entry2pfvf_map;
+ struct mutex rsrc_lock; /* Serialize resource alloc/free */
+};
+
struct nix_hw {
int blkaddr;
struct rvu *rvu;
struct nix_flowkey flowkey;
struct nix_mark_format mark_format;
struct nix_lso lso;
+ struct nix_txvlan txvlan;
};
/* RVU block's capabilities or functionality,
int rvu_lf_reset(struct rvu *rvu, struct rvu_block *block, int lf);
int rvu_get_blkaddr(struct rvu *rvu, int blktype, u16 pcifunc);
int rvu_poll_reg(struct rvu *rvu, u64 block, u64 offset, u64 mask, bool zero);
+int rvu_get_num_lbk_chans(void);
/* RVU HW reg validation */
enum regmap_block {
int nix_update_bcast_mce_list(struct rvu *rvu, u16 pcifunc, bool add);
struct nix_hw *get_nix_hw(struct rvu_hwinfo *hw, int blkaddr);
int rvu_get_next_nix_blkaddr(struct rvu *rvu, int blkaddr);
+void rvu_nix_reset_mac(struct rvu_pfvf *pfvf, int pcifunc);
/* NPC APIs */
int rvu_npc_init(struct rvu *rvu);
void rvu_npc_install_bcast_match_entry(struct rvu *rvu, u16 pcifunc,
int nixlf, u64 chan);
void rvu_npc_enable_bcast_entry(struct rvu *rvu, u16 pcifunc, bool enable);
-int rvu_npc_update_rxvlan(struct rvu *rvu, u16 pcifunc, int nixlf);
void rvu_npc_disable_mcam_entries(struct rvu *rvu, u16 pcifunc, int nixlf);
+void rvu_npc_free_mcam_entries(struct rvu *rvu, u16 pcifunc, int nixlf);
void rvu_npc_disable_default_entries(struct rvu *rvu, u16 pcifunc, int nixlf);
void rvu_npc_enable_default_entries(struct rvu *rvu, u16 pcifunc, int nixlf);
void rvu_npc_update_flowkey_alg_idx(struct rvu *rvu, u16 pcifunc, int nixlf,
bool is_npc_intf_rx(u8 intf);
bool is_npc_interface_valid(struct rvu *rvu, u8 intf);
int rvu_npc_get_tx_nibble_cfg(struct rvu *rvu, u64 nibble_ena);
+int npc_mcam_verify_channel(struct rvu *rvu, u16 pcifunc, u8 intf, u16 channel);
+int npc_flow_steering_init(struct rvu *rvu, int blkaddr);
+const char *npc_get_field_name(u8 hdr);
+bool rvu_npc_write_default_rule(struct rvu *rvu, int blkaddr, int nixlf,
+ u16 pcifunc, u8 intf, struct mcam_entry *entry,
+ int *entry_index);
+int npc_get_bank(struct npc_mcam *mcam, int index);
+void npc_mcam_enable_flows(struct rvu *rvu, u16 target);
+void npc_mcam_disable_flows(struct rvu *rvu, u16 target);
+void npc_enable_mcam_entry(struct rvu *rvu, struct npc_mcam *mcam,
+ int blkaddr, int index, bool enable);
+void npc_read_mcam_entry(struct rvu *rvu, struct npc_mcam *mcam,
+ int blkaddr, u16 src, struct mcam_entry *entry,
+ u8 *intf, u8 *ena);
#ifdef CONFIG_DEBUG_FS
void rvu_dbg_init(struct rvu *rvu);
--- /dev/null
+// SPDX-License-Identifier: GPL-2.0-only
+/* Copyright (C) 2020 Marvell. */
+
+#include <linux/pci.h>
+#include "rvu_struct.h"
+#include "rvu_reg.h"
+#include "mbox.h"
+#include "rvu.h"
+
+/* CPT PF device id */
+#define PCI_DEVID_OTX2_CPT_PF 0xA0FD
+
+static int get_cpt_pf_num(struct rvu *rvu)
+{
+ int i, domain_nr, cpt_pf_num = -1;
+ struct pci_dev *pdev;
+
+ domain_nr = pci_domain_nr(rvu->pdev->bus);
+ for (i = 0; i < rvu->hw->total_pfs; i++) {
+ pdev = pci_get_domain_bus_and_slot(domain_nr, i + 1, 0);
+ if (!pdev)
+ continue;
+
+ if (pdev->device == PCI_DEVID_OTX2_CPT_PF) {
+ cpt_pf_num = i;
+ put_device(&pdev->dev);
+ break;
+ }
+ put_device(&pdev->dev);
+ }
+ return cpt_pf_num;
+}
+
+static bool is_cpt_pf(struct rvu *rvu, u16 pcifunc)
+{
+ int cpt_pf_num = get_cpt_pf_num(rvu);
+
+ if (rvu_get_pf(pcifunc) != cpt_pf_num)
+ return false;
+ if (pcifunc & RVU_PFVF_FUNC_MASK)
+ return false;
+
+ return true;
+}
+
+static bool is_cpt_vf(struct rvu *rvu, u16 pcifunc)
+{
+ int cpt_pf_num = get_cpt_pf_num(rvu);
+
+ if (rvu_get_pf(pcifunc) != cpt_pf_num)
+ return false;
+ if (!(pcifunc & RVU_PFVF_FUNC_MASK))
+ return false;
+
+ return true;
+}
+
+int rvu_mbox_handler_cpt_lf_alloc(struct rvu *rvu,
+ struct cpt_lf_alloc_req_msg *req,
+ struct msg_rsp *rsp)
+{
+ u16 pcifunc = req->hdr.pcifunc;
+ struct rvu_block *block;
+ int cptlf, blkaddr;
+ int num_lfs, slot;
+ u64 val;
+
+ if (req->eng_grpmsk == 0x0)
+ return CPT_AF_ERR_GRP_INVALID;
+
+ blkaddr = rvu_get_blkaddr(rvu, BLKTYPE_CPT, 0);
+ if (blkaddr < 0)
+ return blkaddr;
+
+ block = &rvu->hw->block[blkaddr];
+ num_lfs = rvu_get_rsrc_mapcount(rvu_get_pfvf(rvu, pcifunc),
+ block->addr);
+ if (!num_lfs)
+ return CPT_AF_ERR_LF_INVALID;
+
+ /* Check if requested 'CPTLF <=> NIXLF' mapping is valid */
+ if (req->nix_pf_func) {
+ /* If default, use 'this' CPTLF's PFFUNC */
+ if (req->nix_pf_func == RVU_DEFAULT_PF_FUNC)
+ req->nix_pf_func = pcifunc;
+ if (!is_pffunc_map_valid(rvu, req->nix_pf_func, BLKTYPE_NIX))
+ return CPT_AF_ERR_NIX_PF_FUNC_INVALID;
+ }
+
+ /* Check if requested 'CPTLF <=> SSOLF' mapping is valid */
+ if (req->sso_pf_func) {
+ /* If default, use 'this' CPTLF's PFFUNC */
+ if (req->sso_pf_func == RVU_DEFAULT_PF_FUNC)
+ req->sso_pf_func = pcifunc;
+ if (!is_pffunc_map_valid(rvu, req->sso_pf_func, BLKTYPE_SSO))
+ return CPT_AF_ERR_SSO_PF_FUNC_INVALID;
+ }
+
+ for (slot = 0; slot < num_lfs; slot++) {
+ cptlf = rvu_get_lf(rvu, block, pcifunc, slot);
+ if (cptlf < 0)
+ return CPT_AF_ERR_LF_INVALID;
+
+ /* Set CPT LF group and priority */
+ val = (u64)req->eng_grpmsk << 48 | 1;
+ rvu_write64(rvu, blkaddr, CPT_AF_LFX_CTL(cptlf), val);
+
+ /* Set CPT LF NIX_PF_FUNC and SSO_PF_FUNC */
+ val = (u64)req->nix_pf_func << 48 |
+ (u64)req->sso_pf_func << 32;
+ rvu_write64(rvu, blkaddr, CPT_AF_LFX_CTL2(cptlf), val);
+ }
+
+ return 0;
+}
+
+int rvu_mbox_handler_cpt_lf_free(struct rvu *rvu, struct msg_req *req,
+ struct msg_rsp *rsp)
+{
+ u16 pcifunc = req->hdr.pcifunc;
+ struct rvu_block *block;
+ int cptlf, blkaddr;
+ int num_lfs, slot;
+
+ blkaddr = rvu_get_blkaddr(rvu, BLKTYPE_CPT, 0);
+ if (blkaddr < 0)
+ return blkaddr;
+
+ block = &rvu->hw->block[blkaddr];
+ num_lfs = rvu_get_rsrc_mapcount(rvu_get_pfvf(rvu, pcifunc),
+ block->addr);
+ if (!num_lfs)
+ return CPT_AF_ERR_LF_INVALID;
+
+ for (slot = 0; slot < num_lfs; slot++) {
+ cptlf = rvu_get_lf(rvu, block, pcifunc, slot);
+ if (cptlf < 0)
+ return CPT_AF_ERR_LF_INVALID;
+
+ /* Reset CPT LF group and priority */
+ rvu_write64(rvu, blkaddr, CPT_AF_LFX_CTL(cptlf), 0x0);
+ /* Reset CPT LF NIX_PF_FUNC and SSO_PF_FUNC */
+ rvu_write64(rvu, blkaddr, CPT_AF_LFX_CTL2(cptlf), 0x0);
+ }
+
+ return 0;
+}
+
+static bool is_valid_offset(struct rvu *rvu, struct cpt_rd_wr_reg_msg *req)
+{
+ u64 offset = req->reg_offset;
+ int blkaddr, num_lfs, lf;
+ struct rvu_block *block;
+ struct rvu_pfvf *pfvf;
+
+ blkaddr = rvu_get_blkaddr(rvu, BLKTYPE_CPT, 0);
+
+ /* Registers that can be accessed from PF/VF */
+ if ((offset & 0xFF000) == CPT_AF_LFX_CTL(0) ||
+ (offset & 0xFF000) == CPT_AF_LFX_CTL2(0)) {
+ if (offset & 7)
+ return false;
+
+ lf = (offset & 0xFFF) >> 3;
+ block = &rvu->hw->block[blkaddr];
+ pfvf = rvu_get_pfvf(rvu, req->hdr.pcifunc);
+ num_lfs = rvu_get_rsrc_mapcount(pfvf, block->addr);
+ if (lf >= num_lfs)
+ /* Slot is not valid for that PF/VF */
+ return false;
+
+ /* Translate local LF used by VFs to global CPT LF */
+ lf = rvu_get_lf(rvu, &rvu->hw->block[blkaddr],
+ req->hdr.pcifunc, lf);
+ if (lf < 0)
+ return false;
+
+ return true;
+ } else if (!(req->hdr.pcifunc & RVU_PFVF_FUNC_MASK)) {
+ /* Registers that can be accessed from PF */
+ switch (offset) {
+ case CPT_AF_CTL:
+ case CPT_AF_PF_FUNC:
+ case CPT_AF_BLK_RST:
+ case CPT_AF_CONSTANTS1:
+ return true;
+ }
+
+ switch (offset & 0xFF000) {
+ case CPT_AF_EXEX_STS(0):
+ case CPT_AF_EXEX_CTL(0):
+ case CPT_AF_EXEX_CTL2(0):
+ case CPT_AF_EXEX_UCODE_BASE(0):
+ if (offset & 7)
+ return false;
+ break;
+ default:
+ return false;
+ }
+ return true;
+ }
+ return false;
+}
+
+int rvu_mbox_handler_cpt_rd_wr_register(struct rvu *rvu,
+ struct cpt_rd_wr_reg_msg *req,
+ struct cpt_rd_wr_reg_msg *rsp)
+{
+ int blkaddr;
+
+ blkaddr = rvu_get_blkaddr(rvu, BLKTYPE_CPT, 0);
+ if (blkaddr < 0)
+ return blkaddr;
+
+ /* This message is accepted only if sent from CPT PF/VF */
+ if (!is_cpt_pf(rvu, req->hdr.pcifunc) &&
+ !is_cpt_vf(rvu, req->hdr.pcifunc))
+ return CPT_AF_ERR_ACCESS_DENIED;
+
+ rsp->reg_offset = req->reg_offset;
+ rsp->ret_val = req->ret_val;
+ rsp->is_write = req->is_write;
+
+ if (!is_valid_offset(rvu, req))
+ return CPT_AF_ERR_ACCESS_DENIED;
+
+ if (req->is_write)
+ rvu_write64(rvu, blkaddr, req->reg_offset, req->val);
+ else
+ rsp->val = rvu_read64(rvu, blkaddr, req->reg_offset);
+
+ return 0;
+}
[CGX_STAT17] = "Control/PAUSE packets sent",
};
+enum cpt_eng_type {
+ CPT_AE_TYPE = 1,
+ CPT_SE_TYPE = 2,
+ CPT_IE_TYPE = 3,
+};
+
#define NDC_MAX_BANK(rvu, blk_addr) (rvu_read64(rvu, \
blk_addr, NDC_AF_CONST) & 0xFF)
static void rvu_dbg_nix_init(struct rvu *rvu, int blkaddr)
{
- const struct device *dev = &rvu->pdev->dev;
struct nix_hw *nix_hw;
- struct dentry *pfile;
if (!is_block_implemented(rvu->hw, blkaddr))
return;
if (blkaddr == BLKADDR_NIX0) {
rvu->rvu_dbg.nix = debugfs_create_dir("nix", rvu->rvu_dbg.root);
- if (!rvu->rvu_dbg.nix) {
- dev_err(rvu->dev, "create debugfs dir failed for nix\n");
- return;
- }
nix_hw = &rvu->hw->nix[0];
} else {
rvu->rvu_dbg.nix = debugfs_create_dir("nix1",
rvu->rvu_dbg.root);
- if (!rvu->rvu_dbg.nix) {
- dev_err(rvu->dev,
- "create debugfs dir failed for nix1\n");
- return;
- }
nix_hw = &rvu->hw->nix[1];
}
- pfile = debugfs_create_file("sq_ctx", 0600, rvu->rvu_dbg.nix, nix_hw,
- &rvu_dbg_nix_sq_ctx_fops);
- if (!pfile)
- goto create_failed;
-
- pfile = debugfs_create_file("rq_ctx", 0600, rvu->rvu_dbg.nix, nix_hw,
- &rvu_dbg_nix_rq_ctx_fops);
- if (!pfile)
- goto create_failed;
-
- pfile = debugfs_create_file("cq_ctx", 0600, rvu->rvu_dbg.nix, nix_hw,
- &rvu_dbg_nix_cq_ctx_fops);
- if (!pfile)
- goto create_failed;
-
- pfile = debugfs_create_file("ndc_tx_cache", 0600, rvu->rvu_dbg.nix,
- nix_hw, &rvu_dbg_nix_ndc_tx_cache_fops);
- if (!pfile)
- goto create_failed;
-
- pfile = debugfs_create_file("ndc_rx_cache", 0600, rvu->rvu_dbg.nix,
- nix_hw, &rvu_dbg_nix_ndc_rx_cache_fops);
- if (!pfile)
- goto create_failed;
-
- pfile = debugfs_create_file("ndc_tx_hits_miss", 0600, rvu->rvu_dbg.nix,
- nix_hw,
- &rvu_dbg_nix_ndc_tx_hits_miss_fops);
- if (!pfile)
- goto create_failed;
-
- pfile = debugfs_create_file("ndc_rx_hits_miss", 0600, rvu->rvu_dbg.nix,
- nix_hw,
- &rvu_dbg_nix_ndc_rx_hits_miss_fops);
- if (!pfile)
- goto create_failed;
-
- pfile = debugfs_create_file("qsize", 0600, rvu->rvu_dbg.nix, rvu,
- &rvu_dbg_nix_qsize_fops);
- if (!pfile)
- goto create_failed;
-
- return;
-create_failed:
- dev_err(dev,
- "Failed to create debugfs dir/file for NIX blk\n");
- debugfs_remove_recursive(rvu->rvu_dbg.nix);
+ debugfs_create_file("sq_ctx", 0600, rvu->rvu_dbg.nix, nix_hw,
+ &rvu_dbg_nix_sq_ctx_fops);
+ debugfs_create_file("rq_ctx", 0600, rvu->rvu_dbg.nix, nix_hw,
+ &rvu_dbg_nix_rq_ctx_fops);
+ debugfs_create_file("cq_ctx", 0600, rvu->rvu_dbg.nix, nix_hw,
+ &rvu_dbg_nix_cq_ctx_fops);
+ debugfs_create_file("ndc_tx_cache", 0600, rvu->rvu_dbg.nix, nix_hw,
+ &rvu_dbg_nix_ndc_tx_cache_fops);
+ debugfs_create_file("ndc_rx_cache", 0600, rvu->rvu_dbg.nix, nix_hw,
+ &rvu_dbg_nix_ndc_rx_cache_fops);
+ debugfs_create_file("ndc_tx_hits_miss", 0600, rvu->rvu_dbg.nix, nix_hw,
+ &rvu_dbg_nix_ndc_tx_hits_miss_fops);
+ debugfs_create_file("ndc_rx_hits_miss", 0600, rvu->rvu_dbg.nix, nix_hw,
+ &rvu_dbg_nix_ndc_rx_hits_miss_fops);
+ debugfs_create_file("qsize", 0600, rvu->rvu_dbg.nix, rvu,
+ &rvu_dbg_nix_qsize_fops);
}
static void rvu_dbg_npa_init(struct rvu *rvu)
{
- const struct device *dev = &rvu->pdev->dev;
- struct dentry *pfile;
-
rvu->rvu_dbg.npa = debugfs_create_dir("npa", rvu->rvu_dbg.root);
- if (!rvu->rvu_dbg.npa)
- return;
-
- pfile = debugfs_create_file("qsize", 0600, rvu->rvu_dbg.npa, rvu,
- &rvu_dbg_npa_qsize_fops);
- if (!pfile)
- goto create_failed;
-
- pfile = debugfs_create_file("aura_ctx", 0600, rvu->rvu_dbg.npa, rvu,
- &rvu_dbg_npa_aura_ctx_fops);
- if (!pfile)
- goto create_failed;
-
- pfile = debugfs_create_file("pool_ctx", 0600, rvu->rvu_dbg.npa, rvu,
- &rvu_dbg_npa_pool_ctx_fops);
- if (!pfile)
- goto create_failed;
-
- pfile = debugfs_create_file("ndc_cache", 0600, rvu->rvu_dbg.npa, rvu,
- &rvu_dbg_npa_ndc_cache_fops);
- if (!pfile)
- goto create_failed;
- pfile = debugfs_create_file("ndc_hits_miss", 0600, rvu->rvu_dbg.npa,
- rvu, &rvu_dbg_npa_ndc_hits_miss_fops);
- if (!pfile)
- goto create_failed;
-
- return;
-
-create_failed:
- dev_err(dev, "Failed to create debugfs dir/file for NPA\n");
- debugfs_remove_recursive(rvu->rvu_dbg.npa);
+ debugfs_create_file("qsize", 0600, rvu->rvu_dbg.npa, rvu,
+ &rvu_dbg_npa_qsize_fops);
+ debugfs_create_file("aura_ctx", 0600, rvu->rvu_dbg.npa, rvu,
+ &rvu_dbg_npa_aura_ctx_fops);
+ debugfs_create_file("pool_ctx", 0600, rvu->rvu_dbg.npa, rvu,
+ &rvu_dbg_npa_pool_ctx_fops);
+ debugfs_create_file("ndc_cache", 0600, rvu->rvu_dbg.npa, rvu,
+ &rvu_dbg_npa_ndc_cache_fops);
+ debugfs_create_file("ndc_hits_miss", 0600, rvu->rvu_dbg.npa, rvu,
+ &rvu_dbg_npa_ndc_hits_miss_fops);
}
#define PRINT_CGX_CUML_NIXRX_STATUS(idx, name) \
static void rvu_dbg_cgx_init(struct rvu *rvu)
{
- const struct device *dev = &rvu->pdev->dev;
- struct dentry *pfile;
int i, lmac_id;
char dname[20];
void *cgx;
rvu->rvu_dbg.lmac =
debugfs_create_dir(dname, rvu->rvu_dbg.cgx);
- pfile = debugfs_create_file("stats", 0600,
- rvu->rvu_dbg.lmac, cgx,
- &rvu_dbg_cgx_stat_fops);
- if (!pfile)
- goto create_failed;
+ debugfs_create_file("stats", 0600, rvu->rvu_dbg.lmac,
+ cgx, &rvu_dbg_cgx_stat_fops);
}
}
- return;
-
-create_failed:
- dev_err(dev, "Failed to create debugfs dir/file for CGX\n");
- debugfs_remove_recursive(rvu->rvu_dbg.cgx_root);
}
/* NPC debugfs APIs */
RVU_DEBUG_SEQ_FOPS(npc_rx_miss_act, npc_rx_miss_stats_display, NULL);
-static void rvu_dbg_npc_init(struct rvu *rvu)
+static void rvu_dbg_npc_mcam_show_flows(struct seq_file *s,
+ struct rvu_npc_mcam_rule *rule)
{
- const struct device *dev = &rvu->pdev->dev;
- struct dentry *pfile;
+ u8 bit;
+
+ for_each_set_bit(bit, (unsigned long *)&rule->features, 64) {
+ seq_printf(s, "\t%s ", npc_get_field_name(bit));
+ switch (bit) {
+ case NPC_DMAC:
+ seq_printf(s, "%pM ", rule->packet.dmac);
+ seq_printf(s, "mask %pM\n", rule->mask.dmac);
+ break;
+ case NPC_SMAC:
+ seq_printf(s, "%pM ", rule->packet.smac);
+ seq_printf(s, "mask %pM\n", rule->mask.smac);
+ break;
+ case NPC_ETYPE:
+ seq_printf(s, "0x%x ", ntohs(rule->packet.etype));
+ seq_printf(s, "mask 0x%x\n", ntohs(rule->mask.etype));
+ break;
+ case NPC_OUTER_VID:
+ seq_printf(s, "%d ", ntohs(rule->packet.vlan_tci));
+ seq_printf(s, "mask 0x%x\n",
+ ntohs(rule->mask.vlan_tci));
+ break;
+ case NPC_TOS:
+ seq_printf(s, "%d ", rule->packet.tos);
+ seq_printf(s, "mask 0x%x\n", rule->mask.tos);
+ break;
+ case NPC_SIP_IPV4:
+ seq_printf(s, "%pI4 ", &rule->packet.ip4src);
+ seq_printf(s, "mask %pI4\n", &rule->mask.ip4src);
+ break;
+ case NPC_DIP_IPV4:
+ seq_printf(s, "%pI4 ", &rule->packet.ip4dst);
+ seq_printf(s, "mask %pI4\n", &rule->mask.ip4dst);
+ break;
+ case NPC_SIP_IPV6:
+ seq_printf(s, "%pI6 ", rule->packet.ip6src);
+ seq_printf(s, "mask %pI6\n", rule->mask.ip6src);
+ break;
+ case NPC_DIP_IPV6:
+ seq_printf(s, "%pI6 ", rule->packet.ip6dst);
+ seq_printf(s, "mask %pI6\n", rule->mask.ip6dst);
+ break;
+ case NPC_SPORT_TCP:
+ case NPC_SPORT_UDP:
+ case NPC_SPORT_SCTP:
+ seq_printf(s, "%d ", ntohs(rule->packet.sport));
+ seq_printf(s, "mask 0x%x\n", ntohs(rule->mask.sport));
+ break;
+ case NPC_DPORT_TCP:
+ case NPC_DPORT_UDP:
+ case NPC_DPORT_SCTP:
+ seq_printf(s, "%d ", ntohs(rule->packet.dport));
+ seq_printf(s, "mask 0x%x\n", ntohs(rule->mask.dport));
+ break;
+ default:
+ break;
+ }
+ }
+}
+static void rvu_dbg_npc_mcam_show_action(struct seq_file *s,
+ struct rvu_npc_mcam_rule *rule)
+{
+ if (rule->intf == NIX_INTF_TX) {
+ switch (rule->tx_action.op) {
+ case NIX_TX_ACTIONOP_DROP:
+ seq_puts(s, "\taction: Drop\n");
+ break;
+ case NIX_TX_ACTIONOP_UCAST_DEFAULT:
+ seq_puts(s, "\taction: Unicast to default channel\n");
+ break;
+ case NIX_TX_ACTIONOP_UCAST_CHAN:
+ seq_printf(s, "\taction: Unicast to channel %d\n",
+ rule->tx_action.index);
+ break;
+ case NIX_TX_ACTIONOP_MCAST:
+ seq_puts(s, "\taction: Multicast\n");
+ break;
+ case NIX_TX_ACTIONOP_DROP_VIOL:
+ seq_puts(s, "\taction: Lockdown Violation Drop\n");
+ break;
+ default:
+ break;
+ };
+ } else {
+ switch (rule->rx_action.op) {
+ case NIX_RX_ACTIONOP_DROP:
+ seq_puts(s, "\taction: Drop\n");
+ break;
+ case NIX_RX_ACTIONOP_UCAST:
+ seq_printf(s, "\taction: Direct to queue %d\n",
+ rule->rx_action.index);
+ break;
+ case NIX_RX_ACTIONOP_RSS:
+ seq_puts(s, "\taction: RSS\n");
+ break;
+ case NIX_RX_ACTIONOP_UCAST_IPSEC:
+ seq_puts(s, "\taction: Unicast ipsec\n");
+ break;
+ case NIX_RX_ACTIONOP_MCAST:
+ seq_puts(s, "\taction: Multicast\n");
+ break;
+ default:
+ break;
+ };
+ }
+}
+
+static const char *rvu_dbg_get_intf_name(int intf)
+{
+ switch (intf) {
+ case NIX_INTFX_RX(0):
+ return "NIX0_RX";
+ case NIX_INTFX_RX(1):
+ return "NIX1_RX";
+ case NIX_INTFX_TX(0):
+ return "NIX0_TX";
+ case NIX_INTFX_TX(1):
+ return "NIX1_TX";
+ default:
+ break;
+ }
+
+ return "unknown";
+}
+
+static int rvu_dbg_npc_mcam_show_rules(struct seq_file *s, void *unused)
+{
+ struct rvu_npc_mcam_rule *iter;
+ struct rvu *rvu = s->private;
+ struct npc_mcam *mcam;
+ int pf, vf = -1;
+ int blkaddr;
+ u16 target;
+ u64 hits;
+
+ blkaddr = rvu_get_blkaddr(rvu, BLKTYPE_NPC, 0);
+ if (blkaddr < 0)
+ return 0;
+
+ mcam = &rvu->hw->mcam;
+
+ mutex_lock(&mcam->lock);
+ list_for_each_entry(iter, &mcam->mcam_rules, list) {
+ pf = (iter->owner >> RVU_PFVF_PF_SHIFT) & RVU_PFVF_PF_MASK;
+ seq_printf(s, "\n\tInstalled by: PF%d ", pf);
+
+ if (iter->owner & RVU_PFVF_FUNC_MASK) {
+ vf = (iter->owner & RVU_PFVF_FUNC_MASK) - 1;
+ seq_printf(s, "VF%d", vf);
+ }
+ seq_puts(s, "\n");
+
+ seq_printf(s, "\tdirection: %s\n", is_npc_intf_rx(iter->intf) ?
+ "RX" : "TX");
+ seq_printf(s, "\tinterface: %s\n",
+ rvu_dbg_get_intf_name(iter->intf));
+ seq_printf(s, "\tmcam entry: %d\n", iter->entry);
+
+ rvu_dbg_npc_mcam_show_flows(s, iter);
+ if (iter->intf == NIX_INTF_RX) {
+ target = iter->rx_action.pf_func;
+ pf = (target >> RVU_PFVF_PF_SHIFT) & RVU_PFVF_PF_MASK;
+ seq_printf(s, "\tForward to: PF%d ", pf);
+
+ if (target & RVU_PFVF_FUNC_MASK) {
+ vf = (target & RVU_PFVF_FUNC_MASK) - 1;
+ seq_printf(s, "VF%d", vf);
+ }
+ seq_puts(s, "\n");
+ }
+
+ rvu_dbg_npc_mcam_show_action(s, iter);
+ seq_printf(s, "\tenabled: %s\n", iter->enable ? "yes" : "no");
+
+ if (!iter->has_cntr)
+ continue;
+ seq_printf(s, "\tcounter: %d\n", iter->cntr);
+
+ hits = rvu_read64(rvu, blkaddr, NPC_AF_MATCH_STATX(iter->cntr));
+ seq_printf(s, "\thits: %lld\n", hits);
+ }
+ mutex_unlock(&mcam->lock);
+
+ return 0;
+}
+
+RVU_DEBUG_SEQ_FOPS(npc_mcam_rules, npc_mcam_show_rules, NULL);
+
+static void rvu_dbg_npc_init(struct rvu *rvu)
+{
rvu->rvu_dbg.npc = debugfs_create_dir("npc", rvu->rvu_dbg.root);
- if (!rvu->rvu_dbg.npc)
- return;
- pfile = debugfs_create_file("mcam_info", 0444, rvu->rvu_dbg.npc,
- rvu, &rvu_dbg_npc_mcam_info_fops);
- if (!pfile)
- goto create_failed;
+ debugfs_create_file("mcam_info", 0444, rvu->rvu_dbg.npc, rvu,
+ &rvu_dbg_npc_mcam_info_fops);
+ debugfs_create_file("mcam_rules", 0444, rvu->rvu_dbg.npc, rvu,
+ &rvu_dbg_npc_mcam_rules_fops);
+ debugfs_create_file("rx_miss_act_stats", 0444, rvu->rvu_dbg.npc, rvu,
+ &rvu_dbg_npc_rx_miss_act_fops);
+}
+
+/* CPT debugfs APIs */
+static int cpt_eng_sts_display(struct seq_file *filp, u8 eng_type)
+{
+ struct rvu *rvu = filp->private;
+ u64 busy_sts = 0, free_sts = 0;
+ u32 e_min = 0, e_max = 0, e, i;
+ u16 max_ses, max_ies, max_aes;
+ int blkaddr;
+ u64 reg;
- pfile = debugfs_create_file("rx_miss_act_stats", 0444, rvu->rvu_dbg.npc,
- rvu, &rvu_dbg_npc_rx_miss_act_fops);
- if (!pfile)
- goto create_failed;
+ blkaddr = rvu_get_blkaddr(rvu, BLKTYPE_CPT, 0);
+ if (blkaddr < 0)
+ return -ENODEV;
- return;
+ reg = rvu_read64(rvu, blkaddr, CPT_AF_CONSTANTS1);
+ max_ses = reg & 0xffff;
+ max_ies = (reg >> 16) & 0xffff;
+ max_aes = (reg >> 32) & 0xffff;
-create_failed:
- dev_err(dev, "Failed to create debugfs dir/file for NPC\n");
- debugfs_remove_recursive(rvu->rvu_dbg.npc);
+ switch (eng_type) {
+ case CPT_AE_TYPE:
+ e_min = max_ses + max_ies;
+ e_max = max_ses + max_ies + max_aes;
+ break;
+ case CPT_SE_TYPE:
+ e_min = 0;
+ e_max = max_ses;
+ break;
+ case CPT_IE_TYPE:
+ e_min = max_ses;
+ e_max = max_ses + max_ies;
+ break;
+ default:
+ return -EINVAL;
+ }
+
+ for (e = e_min, i = 0; e < e_max; e++, i++) {
+ reg = rvu_read64(rvu, blkaddr, CPT_AF_EXEX_STS(e));
+ if (reg & 0x1)
+ busy_sts |= 1ULL << i;
+
+ if (reg & 0x2)
+ free_sts |= 1ULL << i;
+ }
+ seq_printf(filp, "FREE STS : 0x%016llx\n", free_sts);
+ seq_printf(filp, "BUSY STS : 0x%016llx\n", busy_sts);
+
+ return 0;
}
-void rvu_dbg_init(struct rvu *rvu)
+static int rvu_dbg_cpt_ae_sts_display(struct seq_file *filp, void *unused)
{
- struct device *dev = &rvu->pdev->dev;
- struct dentry *pfile;
+ return cpt_eng_sts_display(filp, CPT_AE_TYPE);
+}
- rvu->rvu_dbg.root = debugfs_create_dir(DEBUGFS_DIR_NAME, NULL);
- if (!rvu->rvu_dbg.root) {
- dev_err(rvu->dev, "%s failed\n", __func__);
- return;
+RVU_DEBUG_SEQ_FOPS(cpt_ae_sts, cpt_ae_sts_display, NULL);
+
+static int rvu_dbg_cpt_se_sts_display(struct seq_file *filp, void *unused)
+{
+ return cpt_eng_sts_display(filp, CPT_SE_TYPE);
+}
+
+RVU_DEBUG_SEQ_FOPS(cpt_se_sts, cpt_se_sts_display, NULL);
+
+static int rvu_dbg_cpt_ie_sts_display(struct seq_file *filp, void *unused)
+{
+ return cpt_eng_sts_display(filp, CPT_IE_TYPE);
+}
+
+RVU_DEBUG_SEQ_FOPS(cpt_ie_sts, cpt_ie_sts_display, NULL);
+
+static int rvu_dbg_cpt_engines_info_display(struct seq_file *filp, void *unused)
+{
+ struct rvu *rvu = filp->private;
+ u16 max_ses, max_ies, max_aes;
+ u32 e_max, e;
+ int blkaddr;
+ u64 reg;
+
+ blkaddr = rvu_get_blkaddr(rvu, BLKTYPE_CPT, 0);
+ if (blkaddr < 0)
+ return -ENODEV;
+
+ reg = rvu_read64(rvu, blkaddr, CPT_AF_CONSTANTS1);
+ max_ses = reg & 0xffff;
+ max_ies = (reg >> 16) & 0xffff;
+ max_aes = (reg >> 32) & 0xffff;
+
+ e_max = max_ses + max_ies + max_aes;
+
+ seq_puts(filp, "===========================================\n");
+ for (e = 0; e < e_max; e++) {
+ reg = rvu_read64(rvu, blkaddr, CPT_AF_EXEX_CTL2(e));
+ seq_printf(filp, "CPT Engine[%u] Group Enable 0x%02llx\n", e,
+ reg & 0xff);
+ reg = rvu_read64(rvu, blkaddr, CPT_AF_EXEX_ACTIVE(e));
+ seq_printf(filp, "CPT Engine[%u] Active Info 0x%llx\n", e,
+ reg);
+ reg = rvu_read64(rvu, blkaddr, CPT_AF_EXEX_CTL(e));
+ seq_printf(filp, "CPT Engine[%u] Control 0x%llx\n", e,
+ reg);
+ seq_puts(filp, "===========================================\n");
}
- pfile = debugfs_create_file("rsrc_alloc", 0444, rvu->rvu_dbg.root, rvu,
- &rvu_dbg_rsrc_status_fops);
- if (!pfile)
- goto create_failed;
+ return 0;
+}
+
+RVU_DEBUG_SEQ_FOPS(cpt_engines_info, cpt_engines_info_display, NULL);
+
+static int rvu_dbg_cpt_lfs_info_display(struct seq_file *filp, void *unused)
+{
+ struct rvu *rvu = filp->private;
+ struct rvu_hwinfo *hw = rvu->hw;
+ struct rvu_block *block;
+ int blkaddr;
+ u64 reg;
+ u32 lf;
+
+ blkaddr = rvu_get_blkaddr(rvu, BLKTYPE_CPT, 0);
+ if (blkaddr < 0)
+ return -ENODEV;
+
+ block = &hw->block[blkaddr];
+ if (!block->lf.bmap)
+ return -ENODEV;
+
+ seq_puts(filp, "===========================================\n");
+ for (lf = 0; lf < block->lf.max; lf++) {
+ reg = rvu_read64(rvu, blkaddr, CPT_AF_LFX_CTL(lf));
+ seq_printf(filp, "CPT Lf[%u] CTL 0x%llx\n", lf, reg);
+ reg = rvu_read64(rvu, blkaddr, CPT_AF_LFX_CTL2(lf));
+ seq_printf(filp, "CPT Lf[%u] CTL2 0x%llx\n", lf, reg);
+ reg = rvu_read64(rvu, blkaddr, CPT_AF_LFX_PTR_CTL(lf));
+ seq_printf(filp, "CPT Lf[%u] PTR_CTL 0x%llx\n", lf, reg);
+ reg = rvu_read64(rvu, blkaddr, block->lfcfg_reg |
+ (lf << block->lfshift));
+ seq_printf(filp, "CPT Lf[%u] CFG 0x%llx\n", lf, reg);
+ seq_puts(filp, "===========================================\n");
+ }
+ return 0;
+}
- pfile = debugfs_create_file("rvu_pf_cgx_map", 0444, rvu->rvu_dbg.root,
- rvu, &rvu_dbg_rvu_pf_cgx_map_fops);
- if (!pfile)
- goto create_failed;
+RVU_DEBUG_SEQ_FOPS(cpt_lfs_info, cpt_lfs_info_display, NULL);
+
+static int rvu_dbg_cpt_err_info_display(struct seq_file *filp, void *unused)
+{
+ struct rvu *rvu = filp->private;
+ u64 reg0, reg1;
+ int blkaddr;
+
+ blkaddr = rvu_get_blkaddr(rvu, BLKTYPE_CPT, 0);
+ if (blkaddr < 0)
+ return -ENODEV;
+
+ reg0 = rvu_read64(rvu, blkaddr, CPT_AF_FLTX_INT(0));
+ reg1 = rvu_read64(rvu, blkaddr, CPT_AF_FLTX_INT(1));
+ seq_printf(filp, "CPT_AF_FLTX_INT: 0x%llx 0x%llx\n", reg0, reg1);
+ reg0 = rvu_read64(rvu, blkaddr, CPT_AF_PSNX_EXE(0));
+ reg1 = rvu_read64(rvu, blkaddr, CPT_AF_PSNX_EXE(1));
+ seq_printf(filp, "CPT_AF_PSNX_EXE: 0x%llx 0x%llx\n", reg0, reg1);
+ reg0 = rvu_read64(rvu, blkaddr, CPT_AF_PSNX_LF(0));
+ seq_printf(filp, "CPT_AF_PSNX_LF: 0x%llx\n", reg0);
+ reg0 = rvu_read64(rvu, blkaddr, CPT_AF_RVU_INT);
+ seq_printf(filp, "CPT_AF_RVU_INT: 0x%llx\n", reg0);
+ reg0 = rvu_read64(rvu, blkaddr, CPT_AF_RAS_INT);
+ seq_printf(filp, "CPT_AF_RAS_INT: 0x%llx\n", reg0);
+ reg0 = rvu_read64(rvu, blkaddr, CPT_AF_EXE_ERR_INFO);
+ seq_printf(filp, "CPT_AF_EXE_ERR_INFO: 0x%llx\n", reg0);
+
+ return 0;
+}
+
+RVU_DEBUG_SEQ_FOPS(cpt_err_info, cpt_err_info_display, NULL);
+
+static int rvu_dbg_cpt_pc_display(struct seq_file *filp, void *unused)
+{
+ struct rvu *rvu;
+ int blkaddr;
+ u64 reg;
+
+ rvu = filp->private;
+ blkaddr = rvu_get_blkaddr(rvu, BLKTYPE_CPT, 0);
+ if (blkaddr < 0)
+ return -ENODEV;
+
+ reg = rvu_read64(rvu, blkaddr, CPT_AF_INST_REQ_PC);
+ seq_printf(filp, "CPT instruction requests %llu\n", reg);
+ reg = rvu_read64(rvu, blkaddr, CPT_AF_INST_LATENCY_PC);
+ seq_printf(filp, "CPT instruction latency %llu\n", reg);
+ reg = rvu_read64(rvu, blkaddr, CPT_AF_RD_REQ_PC);
+ seq_printf(filp, "CPT NCB read requests %llu\n", reg);
+ reg = rvu_read64(rvu, blkaddr, CPT_AF_RD_LATENCY_PC);
+ seq_printf(filp, "CPT NCB read latency %llu\n", reg);
+ reg = rvu_read64(rvu, blkaddr, CPT_AF_RD_UC_PC);
+ seq_printf(filp, "CPT read requests caused by UC fills %llu\n", reg);
+ reg = rvu_read64(rvu, blkaddr, CPT_AF_ACTIVE_CYCLES_PC);
+ seq_printf(filp, "CPT active cycles pc %llu\n", reg);
+ reg = rvu_read64(rvu, blkaddr, CPT_AF_CPTCLK_CNT);
+ seq_printf(filp, "CPT clock count pc %llu\n", reg);
+
+ return 0;
+}
+
+RVU_DEBUG_SEQ_FOPS(cpt_pc, cpt_pc_display, NULL);
+
+static void rvu_dbg_cpt_init(struct rvu *rvu)
+{
+ if (!is_block_implemented(rvu->hw, BLKADDR_CPT0))
+ return;
+
+ rvu->rvu_dbg.cpt = debugfs_create_dir("cpt", rvu->rvu_dbg.root);
+
+ debugfs_create_file("cpt_pc", 0600, rvu->rvu_dbg.cpt, rvu,
+ &rvu_dbg_cpt_pc_fops);
+ debugfs_create_file("cpt_ae_sts", 0600, rvu->rvu_dbg.cpt, rvu,
+ &rvu_dbg_cpt_ae_sts_fops);
+ debugfs_create_file("cpt_se_sts", 0600, rvu->rvu_dbg.cpt, rvu,
+ &rvu_dbg_cpt_se_sts_fops);
+ debugfs_create_file("cpt_ie_sts", 0600, rvu->rvu_dbg.cpt, rvu,
+ &rvu_dbg_cpt_ie_sts_fops);
+ debugfs_create_file("cpt_engines_info", 0600, rvu->rvu_dbg.cpt, rvu,
+ &rvu_dbg_cpt_engines_info_fops);
+ debugfs_create_file("cpt_lfs_info", 0600, rvu->rvu_dbg.cpt, rvu,
+ &rvu_dbg_cpt_lfs_info_fops);
+ debugfs_create_file("cpt_err_info", 0600, rvu->rvu_dbg.cpt, rvu,
+ &rvu_dbg_cpt_err_info_fops);
+}
+
+void rvu_dbg_init(struct rvu *rvu)
+{
+ rvu->rvu_dbg.root = debugfs_create_dir(DEBUGFS_DIR_NAME, NULL);
+
+ debugfs_create_file("rsrc_alloc", 0444, rvu->rvu_dbg.root, rvu,
+ &rvu_dbg_rsrc_status_fops);
+ debugfs_create_file("rvu_pf_cgx_map", 0444, rvu->rvu_dbg.root, rvu,
+ &rvu_dbg_rvu_pf_cgx_map_fops);
rvu_dbg_npa_init(rvu);
rvu_dbg_nix_init(rvu, BLKADDR_NIX0);
rvu_dbg_nix_init(rvu, BLKADDR_NIX1);
rvu_dbg_cgx_init(rvu);
rvu_dbg_npc_init(rvu);
-
- return;
-
-create_failed:
- dev_err(dev, "Failed to create debugfs dir\n");
- debugfs_remove_recursive(rvu->rvu_dbg.root);
+ rvu_dbg_cpt_init(rvu);
}
void rvu_dbg_exit(struct rvu *rvu)
#include "npc.h"
#include "cgx.h"
+static void nix_free_tx_vtag_entries(struct rvu *rvu, u16 pcifunc);
static int rvu_nix_get_bpid(struct rvu *rvu, struct nix_bp_cfg_req *req,
int type, int chan_id);
pfvf->maxlen = 0;
pfvf->minlen = 0;
- pfvf->rxvlan = false;
/* Remove this PF_FUNC from bcast pkt replication list */
err = nix_update_bcast_mce_list(rvu, pcifunc, false);
/* Config Rx pkt length, csum checks and apad enable / disable */
rvu_write64(rvu, blkaddr, NIX_AF_LFX_RX_CFG(nixlf), req->rx_cfg);
+ /* Configure pkind for TX parse config */
+ cfg = NPC_TX_DEF_PKIND;
+ rvu_write64(rvu, blkaddr, NIX_AF_LFX_TX_PARSE_CFG(nixlf), cfg);
+
intf = is_afvf(pcifunc) ? NIX_INTF_TYPE_LBK : NIX_INTF_TYPE_CGX;
err = nix_interface_init(rvu, pcifunc, intf, nixlf);
if (err)
/* Disable NPC entries as NIXLF's contexts are not initialized yet */
rvu_npc_disable_default_entries(rvu, pcifunc, nixlf);
+ /* Configure RX VTAG Type 7 (strip) for vf vlan */
+ rvu_write64(rvu, blkaddr,
+ NIX_AF_LFX_RX_VTAG_TYPEX(nixlf, NIX_AF_LFX_RX_VTAG_TYPE7),
+ VTAGSIZE_T4 | VTAG_STRIP);
+
goto exit;
free_mem:
return rc;
}
-int rvu_mbox_handler_nix_lf_free(struct rvu *rvu, struct msg_req *req,
+int rvu_mbox_handler_nix_lf_free(struct rvu *rvu, struct nix_lf_free_req *req,
struct msg_rsp *rsp)
{
struct rvu_hwinfo *hw = rvu->hw;
if (nixlf < 0)
return NIX_AF_ERR_AF_LF_INVALID;
+ if (req->flags & NIX_LF_DISABLE_FLOWS)
+ rvu_npc_disable_mcam_entries(rvu, pcifunc, nixlf);
+ else
+ rvu_npc_free_mcam_entries(rvu, pcifunc, nixlf);
+
+ /* Free any tx vtag def entries used by this NIX LF */
+ if (!(req->flags & NIX_LF_DONT_FREE_TX_VTAG))
+ nix_free_tx_vtag_entries(rvu, pcifunc);
+
nix_interface_deinit(rvu, pcifunc, nixlf);
/* Reset this NIX LF */
{
u64 regval = req->vtag_size;
- if (req->rx.vtag_type > 7 || req->vtag_size > VTAGSIZE_T8)
+ if (req->rx.vtag_type > NIX_AF_LFX_RX_VTAG_TYPE7 ||
+ req->vtag_size > VTAGSIZE_T8)
return -EINVAL;
+ /* RX VTAG Type 7 reserved for vf vlan */
+ if (req->rx.vtag_type == NIX_AF_LFX_RX_VTAG_TYPE7)
+ return NIX_AF_ERR_RX_VTAG_INUSE;
+
if (req->rx.capture_vtag)
regval |= BIT_ULL(5);
if (req->rx.strip_vtag)
return 0;
}
+static int nix_tx_vtag_free(struct rvu *rvu, int blkaddr,
+ u16 pcifunc, int index)
+{
+ struct nix_hw *nix_hw = get_nix_hw(rvu->hw, blkaddr);
+ struct nix_txvlan *vlan = &nix_hw->txvlan;
+
+ if (vlan->entry2pfvf_map[index] != pcifunc)
+ return NIX_AF_ERR_PARAM;
+
+ rvu_write64(rvu, blkaddr,
+ NIX_AF_TX_VTAG_DEFX_DATA(index), 0x0ull);
+ rvu_write64(rvu, blkaddr,
+ NIX_AF_TX_VTAG_DEFX_CTL(index), 0x0ull);
+
+ vlan->entry2pfvf_map[index] = 0;
+ rvu_free_rsrc(&vlan->rsrc, index);
+
+ return 0;
+}
+
+static void nix_free_tx_vtag_entries(struct rvu *rvu, u16 pcifunc)
+{
+ struct nix_txvlan *vlan;
+ struct nix_hw *nix_hw;
+ int index, blkaddr;
+
+ blkaddr = rvu_get_blkaddr(rvu, BLKTYPE_NIX, pcifunc);
+ if (blkaddr < 0)
+ return;
+
+ nix_hw = get_nix_hw(rvu->hw, blkaddr);
+ vlan = &nix_hw->txvlan;
+
+ mutex_lock(&vlan->rsrc_lock);
+ /* Scan all the entries and free the ones mapped to 'pcifunc' */
+ for (index = 0; index < vlan->rsrc.max; index++) {
+ if (vlan->entry2pfvf_map[index] == pcifunc)
+ nix_tx_vtag_free(rvu, blkaddr, pcifunc, index);
+ }
+ mutex_unlock(&vlan->rsrc_lock);
+}
+
+static int nix_tx_vtag_alloc(struct rvu *rvu, int blkaddr,
+ u64 vtag, u8 size)
+{
+ struct nix_hw *nix_hw = get_nix_hw(rvu->hw, blkaddr);
+ struct nix_txvlan *vlan = &nix_hw->txvlan;
+ u64 regval;
+ int index;
+
+ mutex_lock(&vlan->rsrc_lock);
+
+ index = rvu_alloc_rsrc(&vlan->rsrc);
+ if (index < 0) {
+ mutex_unlock(&vlan->rsrc_lock);
+ return index;
+ }
+
+ mutex_unlock(&vlan->rsrc_lock);
+
+ regval = size ? vtag : vtag << 32;
+
+ rvu_write64(rvu, blkaddr,
+ NIX_AF_TX_VTAG_DEFX_DATA(index), regval);
+ rvu_write64(rvu, blkaddr,
+ NIX_AF_TX_VTAG_DEFX_CTL(index), size);
+
+ return index;
+}
+
+static int nix_tx_vtag_decfg(struct rvu *rvu, int blkaddr,
+ struct nix_vtag_config *req)
+{
+ struct nix_hw *nix_hw = get_nix_hw(rvu->hw, blkaddr);
+ struct nix_txvlan *vlan = &nix_hw->txvlan;
+ u16 pcifunc = req->hdr.pcifunc;
+ int idx0 = req->tx.vtag0_idx;
+ int idx1 = req->tx.vtag1_idx;
+ int err = 0;
+
+ if (req->tx.free_vtag0 && req->tx.free_vtag1)
+ if (vlan->entry2pfvf_map[idx0] != pcifunc ||
+ vlan->entry2pfvf_map[idx1] != pcifunc)
+ return NIX_AF_ERR_PARAM;
+
+ mutex_lock(&vlan->rsrc_lock);
+
+ if (req->tx.free_vtag0) {
+ err = nix_tx_vtag_free(rvu, blkaddr, pcifunc, idx0);
+ if (err)
+ goto exit;
+ }
+
+ if (req->tx.free_vtag1)
+ err = nix_tx_vtag_free(rvu, blkaddr, pcifunc, idx1);
+
+exit:
+ mutex_unlock(&vlan->rsrc_lock);
+ return err;
+}
+
+static int nix_tx_vtag_cfg(struct rvu *rvu, int blkaddr,
+ struct nix_vtag_config *req,
+ struct nix_vtag_config_rsp *rsp)
+{
+ struct nix_hw *nix_hw = get_nix_hw(rvu->hw, blkaddr);
+ struct nix_txvlan *vlan = &nix_hw->txvlan;
+ u16 pcifunc = req->hdr.pcifunc;
+
+ if (req->tx.cfg_vtag0) {
+ rsp->vtag0_idx =
+ nix_tx_vtag_alloc(rvu, blkaddr,
+ req->tx.vtag0, req->vtag_size);
+
+ if (rsp->vtag0_idx < 0)
+ return NIX_AF_ERR_TX_VTAG_NOSPC;
+
+ vlan->entry2pfvf_map[rsp->vtag0_idx] = pcifunc;
+ }
+
+ if (req->tx.cfg_vtag1) {
+ rsp->vtag1_idx =
+ nix_tx_vtag_alloc(rvu, blkaddr,
+ req->tx.vtag1, req->vtag_size);
+
+ if (rsp->vtag1_idx < 0)
+ goto err_free;
+
+ vlan->entry2pfvf_map[rsp->vtag1_idx] = pcifunc;
+ }
+
+ return 0;
+
+err_free:
+ if (req->tx.cfg_vtag0)
+ nix_tx_vtag_free(rvu, blkaddr, pcifunc, rsp->vtag0_idx);
+
+ return NIX_AF_ERR_TX_VTAG_NOSPC;
+}
+
int rvu_mbox_handler_nix_vtag_cfg(struct rvu *rvu,
struct nix_vtag_config *req,
- struct msg_rsp *rsp)
+ struct nix_vtag_config_rsp *rsp)
{
u16 pcifunc = req->hdr.pcifunc;
int blkaddr, nixlf, err;
return err;
if (req->cfg_type) {
+ /* rx vtag configuration */
err = nix_rx_vtag_cfg(rvu, nixlf, blkaddr, req);
if (err)
return NIX_AF_ERR_PARAM;
} else {
- /* TODO: handle tx vtag configuration */
- return 0;
+ /* tx vtag configuration */
+ if ((req->tx.cfg_vtag0 || req->tx.cfg_vtag1) &&
+ (req->tx.free_vtag0 || req->tx.free_vtag1))
+ return NIX_AF_ERR_PARAM;
+
+ if (req->tx.cfg_vtag0 || req->tx.cfg_vtag1)
+ return nix_tx_vtag_cfg(rvu, blkaddr, req, rsp);
+
+ if (req->tx.free_vtag0 || req->tx.free_vtag1)
+ return nix_tx_vtag_decfg(rvu, blkaddr, req);
}
return 0;
return nix_setup_bcast_tables(rvu, nix_hw);
}
+static int nix_setup_txvlan(struct rvu *rvu, struct nix_hw *nix_hw)
+{
+ struct nix_txvlan *vlan = &nix_hw->txvlan;
+ int err;
+
+ /* Allocate resource bimap for tx vtag def registers*/
+ vlan->rsrc.max = NIX_TX_VTAG_DEF_MAX;
+ err = rvu_alloc_bitmap(&vlan->rsrc);
+ if (err)
+ return -ENOMEM;
+
+ /* Alloc memory for saving entry to RVU PFFUNC allocation mapping */
+ vlan->entry2pfvf_map = devm_kcalloc(rvu->dev, vlan->rsrc.max,
+ sizeof(u16), GFP_KERNEL);
+ if (!vlan->entry2pfvf_map)
+ goto free_mem;
+
+ mutex_init(&vlan->rsrc_lock);
+ return 0;
+
+free_mem:
+ kfree(vlan->rsrc.bmap);
+ return -ENOMEM;
+}
+
static int nix_setup_txschq(struct rvu *rvu, struct nix_hw *nix_hw, int blkaddr)
{
struct nix_txsch *txsch;
/* This should be set to 1, when SEL_CHAN is set */
field->bytesm1 = 1;
break;
+ case NIX_FLOW_KEY_TYPE_IPV4_PROTO:
+ field->lid = NPC_LID_LC;
+ field->hdr_offset = 9; /* offset */
+ field->bytesm1 = 0; /* 1 byte */
+ field->ltype_match = NPC_LT_LC_IP;
+ field->ltype_mask = 0xF;
+ break;
case NIX_FLOW_KEY_TYPE_IPV4:
case NIX_FLOW_KEY_TYPE_INNR_IPV4:
field->lid = NPC_LID_LC;
struct nix_set_mac_addr *req,
struct msg_rsp *rsp)
{
+ bool from_vf = req->hdr.pcifunc & RVU_PFVF_FUNC_MASK;
u16 pcifunc = req->hdr.pcifunc;
int blkaddr, nixlf, err;
struct rvu_pfvf *pfvf;
pfvf = rvu_get_pfvf(rvu, pcifunc);
+ /* VF can't overwrite admin(PF) changes */
+ if (from_vf && pfvf->pf_set_vf_cfg)
+ return -EPERM;
+
ether_addr_copy(pfvf->mac_addr, req->mac_addr);
rvu_npc_install_ucast_entry(rvu, pcifunc, nixlf,
pfvf->rx_chan_base, req->mac_addr);
- rvu_npc_update_rxvlan(rvu, pcifunc, nixlf);
-
return 0;
}
else
rvu_npc_install_promisc_entry(rvu, pcifunc, nixlf,
pfvf->rx_chan_base, allmulti);
-
- rvu_npc_update_rxvlan(rvu, pcifunc, nixlf);
-
return 0;
}
return 0;
}
-int rvu_mbox_handler_nix_rxvlan_alloc(struct rvu *rvu, struct msg_req *req,
- struct msg_rsp *rsp)
-{
- struct npc_mcam_alloc_entry_req alloc_req = { };
- struct npc_mcam_alloc_entry_rsp alloc_rsp = { };
- struct npc_mcam_free_entry_req free_req = { };
- u16 pcifunc = req->hdr.pcifunc;
- int blkaddr, nixlf, err;
- struct rvu_pfvf *pfvf;
-
- /* LBK VFs do not have separate MCAM UCAST entry hence
- * skip allocating rxvlan for them
- */
- if (is_afvf(pcifunc))
- return 0;
-
- pfvf = rvu_get_pfvf(rvu, pcifunc);
- if (pfvf->rxvlan)
- return 0;
-
- /* alloc new mcam entry */
- alloc_req.hdr.pcifunc = pcifunc;
- alloc_req.count = 1;
-
- err = rvu_mbox_handler_npc_mcam_alloc_entry(rvu, &alloc_req,
- &alloc_rsp);
- if (err)
- return err;
-
- /* update entry to enable rxvlan offload */
- blkaddr = rvu_get_blkaddr(rvu, BLKTYPE_NIX, pcifunc);
- if (blkaddr < 0) {
- err = NIX_AF_ERR_AF_LF_INVALID;
- goto free_entry;
- }
-
- nixlf = rvu_get_lf(rvu, &rvu->hw->block[blkaddr], pcifunc, 0);
- if (nixlf < 0) {
- err = NIX_AF_ERR_AF_LF_INVALID;
- goto free_entry;
- }
-
- pfvf->rxvlan_index = alloc_rsp.entry_list[0];
- /* all it means is that rxvlan_index is valid */
- pfvf->rxvlan = true;
-
- err = rvu_npc_update_rxvlan(rvu, pcifunc, nixlf);
- if (err)
- goto free_entry;
-
- return 0;
-free_entry:
- free_req.hdr.pcifunc = pcifunc;
- free_req.entry = alloc_rsp.entry_list[0];
- rvu_mbox_handler_npc_mcam_free_entry(rvu, &free_req, rsp);
- pfvf->rxvlan = false;
- return err;
-}
-
int rvu_mbox_handler_nix_set_rx_cfg(struct rvu *rvu, struct nix_rx_cfg *req,
struct msg_rsp *rsp)
{
if (err)
return err;
+ err = nix_setup_txvlan(rvu, nix_hw);
+ if (err)
+ return err;
+
/* Configure segmentation offload formats */
nix_setup_lso(rvu, nix_hw, blkaddr);
{
struct nix_txsch *txsch;
struct nix_mcast *mcast;
+ struct nix_txvlan *vlan;
struct nix_hw *nix_hw;
int lvl;
kfree(txsch->schq.bmap);
}
+ vlan = &nix_hw->txvlan;
+ kfree(vlan->rsrc.bmap);
+ mutex_destroy(&vlan->rsrc_lock);
+ devm_kfree(rvu->dev, vlan->entry2pfvf_map);
+
mcast = &nix_hw->mcast;
qmem_free(rvu->dev, mcast->mce_ctx);
qmem_free(rvu->dev, mcast->mcast_buf);
rvu_npc_enable_default_entries(rvu, pcifunc, nixlf);
+ npc_mcam_enable_flows(rvu, pcifunc);
+
return rvu_cgx_start_stop_io(rvu, pcifunc, true);
}
rvu_npc_disable_default_entries(rvu, pcifunc, nixlf);
+ npc_mcam_disable_flows(rvu, pcifunc);
+
return rvu_cgx_start_stop_io(rvu, pcifunc, false);
}
ctx_req.hdr.pcifunc = pcifunc;
/* Cleanup NPC MCAM entries, free Tx scheduler queues being used */
+ rvu_npc_disable_mcam_entries(rvu, pcifunc, nixlf);
+ rvu_npc_free_mcam_entries(rvu, pcifunc, nixlf);
nix_interface_deinit(rvu, pcifunc, nixlf);
nix_rx_sync(rvu, blkaddr);
nix_txschq_free(rvu, pcifunc);
return 0;
}
+
+void rvu_nix_reset_mac(struct rvu_pfvf *pfvf, int pcifunc)
+{
+ bool from_vf = !!(pcifunc & RVU_PFVF_FUNC_MASK);
+
+ /* overwrite vf mac address with default_mac */
+ if (from_vf)
+ ether_addr_copy(pfvf->mac_addr, pfvf->default_mac);
+}
#define NPC_PARSE_RESULT_DMAC_OFFSET 8
#define NPC_HW_TSTAMP_OFFSET 8
+#define NPC_KEX_CHAN_MASK 0xFFFULL
+#define NPC_KEX_PF_FUNC_MASK 0xFFFFULL
static const char def_pfl_name[] = "default";
return 0;
}
+static int npc_mcam_verify_pf_func(struct rvu *rvu,
+ struct mcam_entry *entry_data, u8 intf,
+ u16 pcifunc)
+{
+ u16 pf_func, pf_func_mask;
+
+ if (is_npc_intf_rx(intf))
+ return 0;
+
+ pf_func_mask = (entry_data->kw_mask[0] >> 32) &
+ NPC_KEX_PF_FUNC_MASK;
+ pf_func = (entry_data->kw[0] >> 32) & NPC_KEX_PF_FUNC_MASK;
+
+ pf_func = be16_to_cpu((__force __be16)pf_func);
+ if (pf_func_mask != NPC_KEX_PF_FUNC_MASK ||
+ ((pf_func & ~RVU_PFVF_FUNC_MASK) !=
+ (pcifunc & ~RVU_PFVF_FUNC_MASK)))
+ return -EINVAL;
+
+ return 0;
+}
+
+int npc_mcam_verify_channel(struct rvu *rvu, u16 pcifunc, u8 intf, u16 channel)
+{
+ int pf = rvu_get_pf(pcifunc);
+ u8 cgx_id, lmac_id;
+ int base = 0, end;
+
+ if (is_npc_intf_tx(intf))
+ return 0;
+
+ if (is_afvf(pcifunc)) {
+ end = rvu_get_num_lbk_chans();
+ if (end < 0)
+ return -EINVAL;
+ } else {
+ rvu_get_cgx_lmac_id(rvu->pf2cgxlmac_map[pf], &cgx_id, &lmac_id);
+ base = NIX_CHAN_CGX_LMAC_CHX(cgx_id, lmac_id, 0x0);
+ /* CGX mapped functions has maximum of 16 channels */
+ end = NIX_CHAN_CGX_LMAC_CHX(cgx_id, lmac_id, 0xF);
+ }
+
+ if (channel < base || channel > end)
+ return -EINVAL;
+
+ return 0;
+}
+
void rvu_npc_set_pkind(struct rvu *rvu, int pkind, struct rvu_pfvf *pfvf)
{
int blkaddr;
return npc_get_ucast_mcam_index(mcam, pcifunc, nixlf);
}
-static int npc_get_bank(struct npc_mcam *mcam, int index)
+int npc_get_bank(struct npc_mcam *mcam, int index)
{
int bank = index / mcam->banksize;
return (cfg & 1);
}
-static void npc_enable_mcam_entry(struct rvu *rvu, struct npc_mcam *mcam,
- int blkaddr, int index, bool enable)
+void npc_enable_mcam_entry(struct rvu *rvu, struct npc_mcam *mcam,
+ int blkaddr, int index, bool enable)
{
int bank = npc_get_bank(mcam, index);
int actbank = bank;
*cam0 = ~*cam1 & kw_mask;
}
+static void npc_fill_entryword(struct mcam_entry *entry, int idx,
+ u64 cam0, u64 cam1)
+{
+ /* Similar to npc_get_keyword, but fills mcam_entry structure from
+ * CAM registers.
+ */
+ switch (idx) {
+ case 0:
+ entry->kw[0] = cam1;
+ entry->kw_mask[0] = cam1 ^ cam0;
+ break;
+ case 1:
+ entry->kw[1] = cam1;
+ entry->kw_mask[1] = cam1 ^ cam0;
+ break;
+ case 2:
+ entry->kw[1] |= (cam1 & CAM_MASK(16)) << 48;
+ entry->kw[2] = (cam1 >> 16) & CAM_MASK(48);
+ entry->kw_mask[1] |= ((cam1 ^ cam0) & CAM_MASK(16)) << 48;
+ entry->kw_mask[2] = ((cam1 ^ cam0) >> 16) & CAM_MASK(48);
+ break;
+ case 3:
+ entry->kw[2] |= (cam1 & CAM_MASK(16)) << 48;
+ entry->kw[3] = (cam1 >> 16) & CAM_MASK(32);
+ entry->kw_mask[2] |= ((cam1 ^ cam0) & CAM_MASK(16)) << 48;
+ entry->kw_mask[3] = ((cam1 ^ cam0) >> 16) & CAM_MASK(32);
+ break;
+ case 4:
+ entry->kw[3] |= (cam1 & CAM_MASK(32)) << 32;
+ entry->kw[4] = (cam1 >> 32) & CAM_MASK(32);
+ entry->kw_mask[3] |= ((cam1 ^ cam0) & CAM_MASK(32)) << 32;
+ entry->kw_mask[4] = ((cam1 ^ cam0) >> 32) & CAM_MASK(32);
+ break;
+ case 5:
+ entry->kw[4] |= (cam1 & CAM_MASK(32)) << 32;
+ entry->kw[5] = (cam1 >> 32) & CAM_MASK(16);
+ entry->kw_mask[4] |= ((cam1 ^ cam0) & CAM_MASK(32)) << 32;
+ entry->kw_mask[5] = ((cam1 ^ cam0) >> 32) & CAM_MASK(16);
+ break;
+ case 6:
+ entry->kw[5] |= (cam1 & CAM_MASK(48)) << 16;
+ entry->kw[6] = (cam1 >> 48) & CAM_MASK(16);
+ entry->kw_mask[5] |= ((cam1 ^ cam0) & CAM_MASK(48)) << 16;
+ entry->kw_mask[6] = ((cam1 ^ cam0) >> 48) & CAM_MASK(16);
+ break;
+ case 7:
+ entry->kw[6] |= (cam1 & CAM_MASK(48)) << 16;
+ entry->kw_mask[6] |= ((cam1 ^ cam0) & CAM_MASK(48)) << 16;
+ break;
+ }
+}
+
+static void npc_get_default_entry_action(struct rvu *rvu, struct npc_mcam *mcam,
+ int blkaddr, int index,
+ struct mcam_entry *entry)
+{
+ u16 owner, target_func;
+ struct rvu_pfvf *pfvf;
+ int bank, nixlf;
+ u64 rx_action;
+
+ owner = mcam->entry2pfvf_map[index];
+ target_func = (entry->action >> 4) & 0xffff;
+ /* return incase target is PF or LBK or rule owner is not PF */
+ if (is_afvf(target_func) || (owner & RVU_PFVF_FUNC_MASK) ||
+ !(target_func & RVU_PFVF_FUNC_MASK))
+ return;
+
+ pfvf = rvu_get_pfvf(rvu, target_func);
+ mcam->entry2target_pffunc[index] = target_func;
+ /* return if nixlf is not attached or initialized */
+ if (!is_nixlf_attached(rvu, target_func) || !pfvf->def_ucast_rule)
+ return;
+
+ /* get VF ucast entry rule */
+ nix_get_nixlf(rvu, target_func, &nixlf, NULL);
+ index = npc_get_nixlf_mcam_index(mcam, target_func,
+ nixlf, NIXLF_UCAST_ENTRY);
+ bank = npc_get_bank(mcam, index);
+ index &= (mcam->banksize - 1);
+
+ rx_action = rvu_read64(rvu, blkaddr,
+ NPC_AF_MCAMEX_BANKX_ACTION(index, bank));
+ if (rx_action)
+ entry->action = rx_action;
+}
+
static void npc_config_mcam_entry(struct rvu *rvu, struct npc_mcam *mcam,
int blkaddr, int index, u8 intf,
struct mcam_entry *entry, bool enable)
NPC_AF_MCAMEX_BANKX_CAMX_W1(index, bank, 0), cam0);
}
+ /* copy VF default entry action to the VF mcam entry */
+ if (intf == NIX_INTF_RX && actindex < mcam->bmap_entries)
+ npc_get_default_entry_action(rvu, mcam, blkaddr, actindex,
+ entry);
+
/* Set 'action' */
rvu_write64(rvu, blkaddr,
NPC_AF_MCAMEX_BANKX_ACTION(index, actbank), entry->action);
npc_enable_mcam_entry(rvu, mcam, blkaddr, actindex, true);
}
+void npc_read_mcam_entry(struct rvu *rvu, struct npc_mcam *mcam,
+ int blkaddr, u16 src,
+ struct mcam_entry *entry, u8 *intf, u8 *ena)
+{
+ int sbank = npc_get_bank(mcam, src);
+ int bank, kw = 0;
+ u64 cam0, cam1;
+
+ src &= (mcam->banksize - 1);
+ bank = sbank;
+
+ for (; bank < (sbank + mcam->banks_per_entry); bank++, kw = kw + 2) {
+ cam1 = rvu_read64(rvu, blkaddr,
+ NPC_AF_MCAMEX_BANKX_CAMX_W0(src, bank, 1));
+ cam0 = rvu_read64(rvu, blkaddr,
+ NPC_AF_MCAMEX_BANKX_CAMX_W0(src, bank, 0));
+ npc_fill_entryword(entry, kw, cam0, cam1);
+
+ cam1 = rvu_read64(rvu, blkaddr,
+ NPC_AF_MCAMEX_BANKX_CAMX_W1(src, bank, 1));
+ cam0 = rvu_read64(rvu, blkaddr,
+ NPC_AF_MCAMEX_BANKX_CAMX_W1(src, bank, 0));
+ npc_fill_entryword(entry, kw + 1, cam0, cam1);
+ }
+
+ entry->action = rvu_read64(rvu, blkaddr,
+ NPC_AF_MCAMEX_BANKX_ACTION(src, sbank));
+ entry->vtag_action =
+ rvu_read64(rvu, blkaddr,
+ NPC_AF_MCAMEX_BANKX_TAG_ACT(src, sbank));
+ *intf = rvu_read64(rvu, blkaddr,
+ NPC_AF_MCAMEX_BANKX_CAMX_INTF(src, sbank, 1)) & 3;
+ *ena = rvu_read64(rvu, blkaddr,
+ NPC_AF_MCAMEX_BANKX_CFG(src, sbank)) & 1;
+}
+
static void npc_copy_mcam_entry(struct rvu *rvu, struct npc_mcam *mcam,
int blkaddr, u16 src, u16 dest)
{
int nixlf, u64 chan, u8 *mac_addr)
{
struct rvu_pfvf *pfvf = rvu_get_pfvf(rvu, pcifunc);
+ struct npc_install_flow_req req = { 0 };
+ struct npc_install_flow_rsp rsp = { 0 };
struct npc_mcam *mcam = &rvu->hw->mcam;
- struct mcam_entry entry = { {0} };
struct nix_rx_action action;
- int blkaddr, index, kwi;
- u64 mac = 0;
+ int blkaddr, index;
/* AF's VFs work in promiscuous mode */
if (is_afvf(pcifunc))
if (blkaddr < 0)
return;
- for (index = ETH_ALEN - 1; index >= 0; index--)
- mac |= ((u64)*mac_addr++) << (8 * index);
-
index = npc_get_nixlf_mcam_index(mcam, pcifunc,
nixlf, NIXLF_UCAST_ENTRY);
- /* Match ingress channel and DMAC */
- entry.kw[0] = chan;
- entry.kw_mask[0] = 0xFFFULL;
-
- kwi = NPC_PARSE_RESULT_DMAC_OFFSET / sizeof(u64);
- entry.kw[kwi] = mac;
- entry.kw_mask[kwi] = BIT_ULL(48) - 1;
-
/* Don't change the action if entry is already enabled
* Otherwise RSS action may get overwritten.
*/
action.pf_func = pcifunc;
}
- entry.action = *(u64 *)&action;
- npc_config_mcam_entry(rvu, mcam, blkaddr, index,
- pfvf->nix_rx_intf, &entry, true);
-
- /* add VLAN matching, setup action and save entry back for later */
- entry.kw[0] |= (NPC_LT_LB_STAG_QINQ | NPC_LT_LB_CTAG) << 20;
- entry.kw_mask[0] |= (NPC_LT_LB_STAG_QINQ & NPC_LT_LB_CTAG) << 20;
-
- entry.vtag_action = VTAG0_VALID_BIT |
- FIELD_PREP(VTAG0_TYPE_MASK, 0) |
- FIELD_PREP(VTAG0_LID_MASK, NPC_LID_LA) |
- FIELD_PREP(VTAG0_RELPTR_MASK, 12);
+ req.default_rule = 1;
+ ether_addr_copy(req.packet.dmac, mac_addr);
+ eth_broadcast_addr((u8 *)&req.mask.dmac);
+ req.features = BIT_ULL(NPC_DMAC);
+ req.channel = chan;
+ req.intf = pfvf->nix_rx_intf;
+ req.op = action.op;
+ req.hdr.pcifunc = 0; /* AF is requester */
+ req.vf = action.pf_func;
+ req.index = action.index;
+ req.match_id = action.match_id;
+ req.flow_key_alg = action.flow_key_alg;
- memcpy(&pfvf->entry, &entry, sizeof(entry));
+ rvu_mbox_handler_npc_install_flow(rvu, &req, &rsp);
}
void rvu_npc_install_promisc_entry(struct rvu *rvu, u16 pcifunc,
npc_enable_mcam_entry(rvu, mcam, blkaddr, index, enable);
}
+static void npc_update_vf_flow_entry(struct rvu *rvu, struct npc_mcam *mcam,
+ int blkaddr, u16 pcifunc, u64 rx_action)
+{
+ int actindex, index, bank;
+ bool enable;
+
+ if (!(pcifunc & RVU_PFVF_FUNC_MASK))
+ return;
+
+ mutex_lock(&mcam->lock);
+ for (index = 0; index < mcam->bmap_entries; index++) {
+ if (mcam->entry2target_pffunc[index] == pcifunc) {
+ bank = npc_get_bank(mcam, index);
+ actindex = index;
+ index &= (mcam->banksize - 1);
+
+ /* read vf flow entry enable status */
+ enable = is_mcam_entry_enabled(rvu, mcam, blkaddr,
+ actindex);
+ /* disable before mcam entry update */
+ npc_enable_mcam_entry(rvu, mcam, blkaddr, actindex,
+ false);
+ /* update 'action' */
+ rvu_write64(rvu, blkaddr,
+ NPC_AF_MCAMEX_BANKX_ACTION(index, bank),
+ rx_action);
+ if (enable)
+ npc_enable_mcam_entry(rvu, mcam, blkaddr,
+ actindex, true);
+ }
+ }
+ mutex_unlock(&mcam->lock);
+}
+
void rvu_npc_update_flowkey_alg_idx(struct rvu *rvu, u16 pcifunc, int nixlf,
int group, int alg_idx, int mcam_index)
{
struct npc_mcam *mcam = &rvu->hw->mcam;
struct nix_rx_action action;
int blkaddr, index, bank;
+ struct rvu_pfvf *pfvf;
blkaddr = rvu_get_blkaddr(rvu, BLKTYPE_NPC, 0);
if (blkaddr < 0)
rvu_write64(rvu, blkaddr,
NPC_AF_MCAMEX_BANKX_ACTION(index, bank), *(u64 *)&action);
+ /* update the VF flow rule action with the VF default entry action */
+ if (mcam_index < 0)
+ npc_update_vf_flow_entry(rvu, mcam, blkaddr, pcifunc,
+ *(u64 *)&action);
+
+ /* update the action change in default rule */
+ pfvf = rvu_get_pfvf(rvu, pcifunc);
+ if (pfvf->def_ucast_rule)
+ pfvf->def_ucast_rule->rx_action = action;
+
index = npc_get_nixlf_mcam_index(mcam, pcifunc,
nixlf, NIXLF_PROMISC_ENTRY);
NPC_AF_MCAMEX_BANKX_ACTION(index, bank),
*(u64 *)&action);
}
-
- rvu_npc_update_rxvlan(rvu, pcifunc, nixlf);
}
static void npc_enadis_default_entries(struct rvu *rvu, u16 pcifunc,
rvu_npc_enable_promisc_entry(rvu, pcifunc, nixlf);
else
rvu_npc_disable_promisc_entry(rvu, pcifunc, nixlf);
-
- rvu_npc_update_rxvlan(rvu, pcifunc, nixlf);
}
void rvu_npc_disable_default_entries(struct rvu *rvu, u16 pcifunc, int nixlf)
}
void rvu_npc_disable_mcam_entries(struct rvu *rvu, u16 pcifunc, int nixlf)
+{
+ struct rvu_pfvf *pfvf = rvu_get_pfvf(rvu, pcifunc);
+ struct npc_mcam *mcam = &rvu->hw->mcam;
+ struct rvu_npc_mcam_rule *rule;
+ int blkaddr;
+
+ blkaddr = rvu_get_blkaddr(rvu, BLKTYPE_NPC, 0);
+ if (blkaddr < 0)
+ return;
+
+ mutex_lock(&mcam->lock);
+
+ /* Disable MCAM entries directing traffic to this 'pcifunc' */
+ list_for_each_entry(rule, &mcam->mcam_rules, list) {
+ if (is_npc_intf_rx(rule->intf) &&
+ rule->rx_action.pf_func == pcifunc) {
+ npc_enable_mcam_entry(rvu, mcam, blkaddr,
+ rule->entry, false);
+ rule->enable = false;
+ /* Indicate that default rule is disabled */
+ if (rule->default_rule)
+ pfvf->def_ucast_rule = NULL;
+ }
+ }
+
+ mutex_unlock(&mcam->lock);
+
+ npc_mcam_disable_flows(rvu, pcifunc);
+
+ rvu_npc_disable_default_entries(rvu, pcifunc, nixlf);
+}
+
+void rvu_npc_free_mcam_entries(struct rvu *rvu, u16 pcifunc, int nixlf)
{
struct npc_mcam *mcam = &rvu->hw->mcam;
+ struct rvu_npc_mcam_rule *rule, *tmp;
int blkaddr;
blkaddr = rvu_get_blkaddr(rvu, BLKTYPE_NPC, 0);
mutex_lock(&mcam->lock);
- /* Disable and free all MCAM entries mapped to this 'pcifunc' */
+ /* Free all MCAM entries owned by this 'pcifunc' */
npc_mcam_free_all_entries(rvu, mcam, blkaddr, pcifunc);
- /* Free all MCAM counters mapped to this 'pcifunc' */
+ /* Free all MCAM counters owned by this 'pcifunc' */
npc_mcam_free_all_counters(rvu, mcam, pcifunc);
+ /* Delete MCAM entries owned by this 'pcifunc' */
+ list_for_each_entry_safe(rule, tmp, &mcam->mcam_rules, list) {
+ if (rule->owner == pcifunc && !rule->default_rule) {
+ list_del(&rule->list);
+ kfree(rule);
+ }
+ }
+
mutex_unlock(&mcam->lock);
rvu_npc_disable_default_entries(rvu, pcifunc, nixlf);
if (!mcam->cntr_refcnt)
goto free_mem;
+ /* Alloc memory for saving target device of mcam rule */
+ mcam->entry2target_pffunc = devm_kcalloc(rvu->dev, mcam->total_entries,
+ sizeof(u16), GFP_KERNEL);
+ if (!mcam->entry2target_pffunc)
+ goto free_mem;
+
mutex_init(&mcam->lock);
return 0;
rvu_npc_setup_interfaces(rvu, blkaddr);
+ /* Configure MKEX profile */
+ npc_load_mkex_profile(rvu, blkaddr, rvu->mkex_pfl_name);
+
err = npc_mcam_rsrcs_init(rvu, blkaddr);
if (err)
return err;
- /* Configure MKEX profile */
- npc_load_mkex_profile(rvu, blkaddr, rvu->mkex_pfl_name);
+ err = npc_flow_steering_init(rvu, blkaddr);
+ if (err) {
+ dev_err(rvu->dev,
+ "Incorrect mkex profile loaded using default mkex\n");
+ npc_load_mkex_profile(rvu, blkaddr, def_pfl_name);
+ }
return 0;
}
npc_unmap_mcam_entry_and_cntr(rvu, mcam,
blkaddr, index,
cntr);
+ mcam->entry2target_pffunc[index] = 0x0;
}
}
}
goto exit;
mcam->entry2pfvf_map[req->entry] = 0;
+ mcam->entry2target_pffunc[req->entry] = 0x0;
npc_mcam_clear_bit(mcam, req->entry);
npc_enable_mcam_entry(rvu, mcam, blkaddr, req->entry, false);
return rc;
}
+int rvu_mbox_handler_npc_mcam_read_entry(struct rvu *rvu,
+ struct npc_mcam_read_entry_req *req,
+ struct npc_mcam_read_entry_rsp *rsp)
+{
+ struct npc_mcam *mcam = &rvu->hw->mcam;
+ u16 pcifunc = req->hdr.pcifunc;
+ int blkaddr, rc;
+
+ blkaddr = rvu_get_blkaddr(rvu, BLKTYPE_NPC, 0);
+ if (blkaddr < 0)
+ return NPC_MCAM_INVALID_REQ;
+
+ mutex_lock(&mcam->lock);
+ rc = npc_mcam_verify_entry(mcam, pcifunc, req->entry);
+ if (!rc) {
+ npc_read_mcam_entry(rvu, mcam, blkaddr, req->entry,
+ &rsp->entry_data,
+ &rsp->intf, &rsp->enable);
+ }
+
+ mutex_unlock(&mcam->lock);
+ return rc;
+}
+
int rvu_mbox_handler_npc_mcam_write_entry(struct rvu *rvu,
struct npc_mcam_write_entry_req *req,
struct msg_rsp *rsp)
struct rvu_pfvf *pfvf = rvu_get_pfvf(rvu, req->hdr.pcifunc);
struct npc_mcam *mcam = &rvu->hw->mcam;
u16 pcifunc = req->hdr.pcifunc;
+ u16 channel, chan_mask;
int blkaddr, rc;
u8 nix_intf;
if (blkaddr < 0)
return NPC_MCAM_INVALID_REQ;
+ chan_mask = req->entry_data.kw_mask[0] & NPC_KEX_CHAN_MASK;
+ channel = req->entry_data.kw[0] & NPC_KEX_CHAN_MASK;
+ channel &= chan_mask;
+
mutex_lock(&mcam->lock);
rc = npc_mcam_verify_entry(mcam, pcifunc, req->entry);
if (rc)
else
nix_intf = pfvf->nix_rx_intf;
+ if (npc_mcam_verify_channel(rvu, pcifunc, req->intf, channel)) {
+ rc = NPC_MCAM_INVALID_REQ;
+ goto exit;
+ }
+
+ if (npc_mcam_verify_pf_func(rvu, &req->entry_data, req->intf,
+ pcifunc)) {
+ rc = NPC_MCAM_INVALID_REQ;
+ goto exit;
+ }
+
npc_config_mcam_entry(rvu, mcam, blkaddr, req->entry, nix_intf,
&req->entry_data, req->enable_entry);
struct npc_mcam *mcam = &rvu->hw->mcam;
u16 entry = NPC_MCAM_ENTRY_INVALID;
u16 cntr = NPC_MCAM_ENTRY_INVALID;
+ u16 channel, chan_mask;
int blkaddr, rc;
u8 nix_intf;
if (!is_npc_interface_valid(rvu, req->intf))
return NPC_MCAM_INVALID_REQ;
+ chan_mask = req->entry_data.kw_mask[0] & NPC_KEX_CHAN_MASK;
+ channel = req->entry_data.kw[0] & NPC_KEX_CHAN_MASK;
+ channel &= chan_mask;
+
+ if (npc_mcam_verify_channel(rvu, req->hdr.pcifunc, req->intf, channel))
+ return NPC_MCAM_INVALID_REQ;
+
+ if (npc_mcam_verify_pf_func(rvu, &req->entry_data, req->intf,
+ req->hdr.pcifunc))
+ return NPC_MCAM_INVALID_REQ;
+
/* Try to allocate a MCAM entry */
entry_req.hdr.pcifunc = req->hdr.pcifunc;
entry_req.contig = true;
return 0;
}
-int rvu_npc_update_rxvlan(struct rvu *rvu, u16 pcifunc, int nixlf)
+bool rvu_npc_write_default_rule(struct rvu *rvu, int blkaddr, int nixlf,
+ u16 pcifunc, u8 intf, struct mcam_entry *entry,
+ int *index)
{
struct rvu_pfvf *pfvf = rvu_get_pfvf(rvu, pcifunc);
struct npc_mcam *mcam = &rvu->hw->mcam;
- int blkaddr, index;
bool enable;
+ u8 nix_intf;
+
+ if (is_npc_intf_tx(intf))
+ nix_intf = pfvf->nix_tx_intf;
+ else
+ nix_intf = pfvf->nix_rx_intf;
+
+ *index = npc_get_nixlf_mcam_index(mcam, pcifunc,
+ nixlf, NIXLF_UCAST_ENTRY);
+ /* dont force enable unicast entry */
+ enable = is_mcam_entry_enabled(rvu, mcam, blkaddr, *index);
+ npc_config_mcam_entry(rvu, mcam, blkaddr, *index, nix_intf,
+ entry, enable);
+
+ return enable;
+}
+
+int rvu_mbox_handler_npc_read_base_steer_rule(struct rvu *rvu,
+ struct msg_req *req,
+ struct npc_mcam_read_base_rule_rsp *rsp)
+{
+ struct npc_mcam *mcam = &rvu->hw->mcam;
+ int index, blkaddr, nixlf, rc = 0;
+ u16 pcifunc = req->hdr.pcifunc;
+ struct rvu_pfvf *pfvf;
+ u8 intf, enable;
blkaddr = rvu_get_blkaddr(rvu, BLKTYPE_NPC, 0);
if (blkaddr < 0)
- return NIX_AF_ERR_AF_LF_INVALID;
+ return NPC_MCAM_INVALID_REQ;
- if (!pfvf->rxvlan)
- return 0;
+ /* Return the channel number in case of PF */
+ if (!(pcifunc & RVU_PFVF_FUNC_MASK)) {
+ pfvf = rvu_get_pfvf(rvu, pcifunc);
+ rsp->entry.kw[0] = pfvf->rx_chan_base;
+ rsp->entry.kw_mask[0] = 0xFFFULL;
+ goto out;
+ }
+
+ /* Find the pkt steering rule installed by PF to this VF */
+ mutex_lock(&mcam->lock);
+ for (index = 0; index < mcam->bmap_entries; index++) {
+ if (mcam->entry2target_pffunc[index] == pcifunc)
+ goto read_entry;
+ }
+ rc = nix_get_nixlf(rvu, pcifunc, &nixlf, NULL);
+ if (rc < 0) {
+ mutex_unlock(&mcam->lock);
+ goto out;
+ }
+ /* Read the default ucast entry if there is no pkt steering rule */
index = npc_get_nixlf_mcam_index(mcam, pcifunc, nixlf,
NIXLF_UCAST_ENTRY);
- pfvf->entry.action = npc_get_mcam_action(rvu, mcam, blkaddr, index);
- enable = is_mcam_entry_enabled(rvu, mcam, blkaddr, index);
- npc_config_mcam_entry(rvu, mcam, blkaddr, pfvf->rxvlan_index,
- pfvf->nix_rx_intf, &pfvf->entry, enable);
-
- return 0;
+read_entry:
+ /* Read the mcam entry */
+ npc_read_mcam_entry(rvu, mcam, blkaddr, index, &rsp->entry, &intf,
+ &enable);
+ mutex_unlock(&mcam->lock);
+out:
+ return rc;
}
--- /dev/null
+// SPDX-License-Identifier: GPL-2.0
+/* Marvell OcteonTx2 RVU Admin Function driver
+ *
+ * Copyright (C) 2020 Marvell.
+ */
+
+#include <linux/bitfield.h>
+
+#include "rvu_struct.h"
+#include "rvu_reg.h"
+#include "rvu.h"
+#include "npc.h"
+
+#define NPC_BYTESM GENMASK_ULL(19, 16)
+#define NPC_HDR_OFFSET GENMASK_ULL(15, 8)
+#define NPC_KEY_OFFSET GENMASK_ULL(5, 0)
+#define NPC_LDATA_EN BIT_ULL(7)
+
+static const char * const npc_flow_names[] = {
+ [NPC_DMAC] = "dmac",
+ [NPC_SMAC] = "smac",
+ [NPC_ETYPE] = "ether type",
+ [NPC_OUTER_VID] = "outer vlan id",
+ [NPC_TOS] = "tos",
+ [NPC_SIP_IPV4] = "ipv4 source ip",
+ [NPC_DIP_IPV4] = "ipv4 destination ip",
+ [NPC_SIP_IPV6] = "ipv6 source ip",
+ [NPC_DIP_IPV6] = "ipv6 destination ip",
+ [NPC_SPORT_TCP] = "tcp source port",
+ [NPC_DPORT_TCP] = "tcp destination port",
+ [NPC_SPORT_UDP] = "udp source port",
+ [NPC_DPORT_UDP] = "udp destination port",
+ [NPC_SPORT_SCTP] = "sctp source port",
+ [NPC_DPORT_SCTP] = "sctp destination port",
+ [NPC_UNKNOWN] = "unknown",
+};
+
+const char *npc_get_field_name(u8 hdr)
+{
+ if (hdr >= ARRAY_SIZE(npc_flow_names))
+ return npc_flow_names[NPC_UNKNOWN];
+
+ return npc_flow_names[hdr];
+}
+
+/* Compute keyword masks and figure out the number of keywords a field
+ * spans in the key.
+ */
+static void npc_set_kw_masks(struct npc_mcam *mcam, u8 type,
+ u8 nr_bits, int start_kwi, int offset, u8 intf)
+{
+ struct npc_key_field *field = &mcam->rx_key_fields[type];
+ u8 bits_in_kw;
+ int max_kwi;
+
+ if (mcam->banks_per_entry == 1)
+ max_kwi = 1; /* NPC_MCAM_KEY_X1 */
+ else if (mcam->banks_per_entry == 2)
+ max_kwi = 3; /* NPC_MCAM_KEY_X2 */
+ else
+ max_kwi = 6; /* NPC_MCAM_KEY_X4 */
+
+ if (is_npc_intf_tx(intf))
+ field = &mcam->tx_key_fields[type];
+
+ if (offset + nr_bits <= 64) {
+ /* one KW only */
+ if (start_kwi > max_kwi)
+ return;
+ field->kw_mask[start_kwi] |= GENMASK_ULL(nr_bits - 1, 0)
+ << offset;
+ field->nr_kws = 1;
+ } else if (offset + nr_bits > 64 &&
+ offset + nr_bits <= 128) {
+ /* two KWs */
+ if (start_kwi + 1 > max_kwi)
+ return;
+ /* first KW mask */
+ bits_in_kw = 64 - offset;
+ field->kw_mask[start_kwi] |= GENMASK_ULL(bits_in_kw - 1, 0)
+ << offset;
+ /* second KW mask i.e. mask for rest of bits */
+ bits_in_kw = nr_bits + offset - 64;
+ field->kw_mask[start_kwi + 1] |= GENMASK_ULL(bits_in_kw - 1, 0);
+ field->nr_kws = 2;
+ } else {
+ /* three KWs */
+ if (start_kwi + 2 > max_kwi)
+ return;
+ /* first KW mask */
+ bits_in_kw = 64 - offset;
+ field->kw_mask[start_kwi] |= GENMASK_ULL(bits_in_kw - 1, 0)
+ << offset;
+ /* second KW mask */
+ field->kw_mask[start_kwi + 1] = ~0ULL;
+ /* third KW mask i.e. mask for rest of bits */
+ bits_in_kw = nr_bits + offset - 128;
+ field->kw_mask[start_kwi + 2] |= GENMASK_ULL(bits_in_kw - 1, 0);
+ field->nr_kws = 3;
+ }
+}
+
+/* Helper function to figure out whether field exists in the key */
+static bool npc_is_field_present(struct rvu *rvu, enum key_fields type, u8 intf)
+{
+ struct npc_mcam *mcam = &rvu->hw->mcam;
+ struct npc_key_field *input;
+
+ input = &mcam->rx_key_fields[type];
+ if (is_npc_intf_tx(intf))
+ input = &mcam->tx_key_fields[type];
+
+ return input->nr_kws > 0;
+}
+
+static bool npc_is_same(struct npc_key_field *input,
+ struct npc_key_field *field)
+{
+ int ret;
+
+ ret = memcmp(&input->layer_mdata, &field->layer_mdata,
+ sizeof(struct npc_layer_mdata));
+ return ret == 0;
+}
+
+static void npc_set_layer_mdata(struct npc_mcam *mcam, enum key_fields type,
+ u64 cfg, u8 lid, u8 lt, u8 intf)
+{
+ struct npc_key_field *input = &mcam->rx_key_fields[type];
+
+ if (is_npc_intf_tx(intf))
+ input = &mcam->tx_key_fields[type];
+
+ input->layer_mdata.hdr = FIELD_GET(NPC_HDR_OFFSET, cfg);
+ input->layer_mdata.key = FIELD_GET(NPC_KEY_OFFSET, cfg);
+ input->layer_mdata.len = FIELD_GET(NPC_BYTESM, cfg) + 1;
+ input->layer_mdata.ltype = lt;
+ input->layer_mdata.lid = lid;
+}
+
+static bool npc_check_overlap_fields(struct npc_key_field *input1,
+ struct npc_key_field *input2)
+{
+ int kwi;
+
+ /* Fields with same layer id and different ltypes are mutually
+ * exclusive hence they can be overlapped
+ */
+ if (input1->layer_mdata.lid == input2->layer_mdata.lid &&
+ input1->layer_mdata.ltype != input2->layer_mdata.ltype)
+ return false;
+
+ for (kwi = 0; kwi < NPC_MAX_KWS_IN_KEY; kwi++) {
+ if (input1->kw_mask[kwi] & input2->kw_mask[kwi])
+ return true;
+ }
+
+ return false;
+}
+
+/* Helper function to check whether given field overlaps with any other fields
+ * in the key. Due to limitations on key size and the key extraction profile in
+ * use higher layers can overwrite lower layer's header fields. Hence overlap
+ * needs to be checked.
+ */
+static bool npc_check_overlap(struct rvu *rvu, int blkaddr,
+ enum key_fields type, u8 start_lid, u8 intf)
+{
+ struct npc_mcam *mcam = &rvu->hw->mcam;
+ struct npc_key_field *dummy, *input;
+ int start_kwi, offset;
+ u8 nr_bits, lid, lt, ld;
+ u64 cfg;
+
+ dummy = &mcam->rx_key_fields[NPC_UNKNOWN];
+ input = &mcam->rx_key_fields[type];
+
+ if (is_npc_intf_tx(intf)) {
+ dummy = &mcam->tx_key_fields[NPC_UNKNOWN];
+ input = &mcam->tx_key_fields[type];
+ }
+
+ for (lid = start_lid; lid < NPC_MAX_LID; lid++) {
+ for (lt = 0; lt < NPC_MAX_LT; lt++) {
+ for (ld = 0; ld < NPC_MAX_LD; ld++) {
+ cfg = rvu_read64(rvu, blkaddr,
+ NPC_AF_INTFX_LIDX_LTX_LDX_CFG
+ (intf, lid, lt, ld));
+ if (!FIELD_GET(NPC_LDATA_EN, cfg))
+ continue;
+ memset(dummy, 0, sizeof(struct npc_key_field));
+ npc_set_layer_mdata(mcam, NPC_UNKNOWN, cfg,
+ lid, lt, intf);
+ /* exclude input */
+ if (npc_is_same(input, dummy))
+ continue;
+ start_kwi = dummy->layer_mdata.key / 8;
+ offset = (dummy->layer_mdata.key * 8) % 64;
+ nr_bits = dummy->layer_mdata.len * 8;
+ /* form KW masks */
+ npc_set_kw_masks(mcam, NPC_UNKNOWN, nr_bits,
+ start_kwi, offset, intf);
+ /* check any input field bits falls in any
+ * other field bits.
+ */
+ if (npc_check_overlap_fields(dummy, input))
+ return true;
+ }
+ }
+ }
+
+ return false;
+}
+
+static int npc_check_field(struct rvu *rvu, int blkaddr, enum key_fields type,
+ u8 intf)
+{
+ if (!npc_is_field_present(rvu, type, intf) ||
+ npc_check_overlap(rvu, blkaddr, type, 0, intf))
+ return -EOPNOTSUPP;
+ return 0;
+}
+
+static void npc_scan_parse_result(struct npc_mcam *mcam, u8 bit_number,
+ u8 key_nibble, u8 intf)
+{
+ u8 offset = (key_nibble * 4) % 64; /* offset within key word */
+ u8 kwi = (key_nibble * 4) / 64; /* which word in key */
+ u8 nr_bits = 4; /* bits in a nibble */
+ u8 type;
+
+ switch (bit_number) {
+ case 0 ... 2:
+ type = NPC_CHAN;
+ break;
+ case 3:
+ type = NPC_ERRLEV;
+ break;
+ case 4 ... 5:
+ type = NPC_ERRCODE;
+ break;
+ case 6:
+ type = NPC_LXMB;
+ break;
+ /* check for LTYPE only as of now */
+ case 9:
+ type = NPC_LA;
+ break;
+ case 12:
+ type = NPC_LB;
+ break;
+ case 15:
+ type = NPC_LC;
+ break;
+ case 18:
+ type = NPC_LD;
+ break;
+ case 21:
+ type = NPC_LE;
+ break;
+ case 24:
+ type = NPC_LF;
+ break;
+ case 27:
+ type = NPC_LG;
+ break;
+ case 30:
+ type = NPC_LH;
+ break;
+ default:
+ return;
+ };
+ npc_set_kw_masks(mcam, type, nr_bits, kwi, offset, intf);
+}
+
+static void npc_handle_multi_layer_fields(struct rvu *rvu, int blkaddr, u8 intf)
+{
+ struct npc_mcam *mcam = &rvu->hw->mcam;
+ struct npc_key_field *key_fields;
+ /* Ether type can come from three layers
+ * (ethernet, single tagged, double tagged)
+ */
+ struct npc_key_field *etype_ether;
+ struct npc_key_field *etype_tag1;
+ struct npc_key_field *etype_tag2;
+ /* Outer VLAN TCI can come from two layers
+ * (single tagged, double tagged)
+ */
+ struct npc_key_field *vlan_tag1;
+ struct npc_key_field *vlan_tag2;
+ u64 *features;
+ u8 start_lid;
+ int i;
+
+ key_fields = mcam->rx_key_fields;
+ features = &mcam->rx_features;
+
+ if (is_npc_intf_tx(intf)) {
+ key_fields = mcam->tx_key_fields;
+ features = &mcam->tx_features;
+ }
+
+ /* Handle header fields which can come from multiple layers like
+ * etype, outer vlan tci. These fields should have same position in
+ * the key otherwise to install a mcam rule more than one entry is
+ * needed which complicates mcam space management.
+ */
+ etype_ether = &key_fields[NPC_ETYPE_ETHER];
+ etype_tag1 = &key_fields[NPC_ETYPE_TAG1];
+ etype_tag2 = &key_fields[NPC_ETYPE_TAG2];
+ vlan_tag1 = &key_fields[NPC_VLAN_TAG1];
+ vlan_tag2 = &key_fields[NPC_VLAN_TAG2];
+
+ /* if key profile programmed does not extract Ethertype at all */
+ if (!etype_ether->nr_kws && !etype_tag1->nr_kws && !etype_tag2->nr_kws)
+ goto vlan_tci;
+
+ /* if key profile programmed extracts Ethertype from one layer */
+ if (etype_ether->nr_kws && !etype_tag1->nr_kws && !etype_tag2->nr_kws)
+ key_fields[NPC_ETYPE] = *etype_ether;
+ if (!etype_ether->nr_kws && etype_tag1->nr_kws && !etype_tag2->nr_kws)
+ key_fields[NPC_ETYPE] = *etype_tag1;
+ if (!etype_ether->nr_kws && !etype_tag1->nr_kws && etype_tag2->nr_kws)
+ key_fields[NPC_ETYPE] = *etype_tag2;
+
+ /* if key profile programmed extracts Ethertype from multiple layers */
+ if (etype_ether->nr_kws && etype_tag1->nr_kws) {
+ for (i = 0; i < NPC_MAX_KWS_IN_KEY; i++) {
+ if (etype_ether->kw_mask[i] != etype_tag1->kw_mask[i])
+ goto vlan_tci;
+ }
+ key_fields[NPC_ETYPE] = *etype_tag1;
+ }
+ if (etype_ether->nr_kws && etype_tag2->nr_kws) {
+ for (i = 0; i < NPC_MAX_KWS_IN_KEY; i++) {
+ if (etype_ether->kw_mask[i] != etype_tag2->kw_mask[i])
+ goto vlan_tci;
+ }
+ key_fields[NPC_ETYPE] = *etype_tag2;
+ }
+ if (etype_tag1->nr_kws && etype_tag2->nr_kws) {
+ for (i = 0; i < NPC_MAX_KWS_IN_KEY; i++) {
+ if (etype_tag1->kw_mask[i] != etype_tag2->kw_mask[i])
+ goto vlan_tci;
+ }
+ key_fields[NPC_ETYPE] = *etype_tag2;
+ }
+
+ /* check none of higher layers overwrite Ethertype */
+ start_lid = key_fields[NPC_ETYPE].layer_mdata.lid + 1;
+ if (npc_check_overlap(rvu, blkaddr, NPC_ETYPE, start_lid, intf))
+ goto vlan_tci;
+ *features |= BIT_ULL(NPC_ETYPE);
+vlan_tci:
+ /* if key profile does not extract outer vlan tci at all */
+ if (!vlan_tag1->nr_kws && !vlan_tag2->nr_kws)
+ goto done;
+
+ /* if key profile extracts outer vlan tci from one layer */
+ if (vlan_tag1->nr_kws && !vlan_tag2->nr_kws)
+ key_fields[NPC_OUTER_VID] = *vlan_tag1;
+ if (!vlan_tag1->nr_kws && vlan_tag2->nr_kws)
+ key_fields[NPC_OUTER_VID] = *vlan_tag2;
+
+ /* if key profile extracts outer vlan tci from multiple layers */
+ if (vlan_tag1->nr_kws && vlan_tag2->nr_kws) {
+ for (i = 0; i < NPC_MAX_KWS_IN_KEY; i++) {
+ if (vlan_tag1->kw_mask[i] != vlan_tag2->kw_mask[i])
+ goto done;
+ }
+ key_fields[NPC_OUTER_VID] = *vlan_tag2;
+ }
+ /* check none of higher layers overwrite outer vlan tci */
+ start_lid = key_fields[NPC_OUTER_VID].layer_mdata.lid + 1;
+ if (npc_check_overlap(rvu, blkaddr, NPC_OUTER_VID, start_lid, intf))
+ goto done;
+ *features |= BIT_ULL(NPC_OUTER_VID);
+done:
+ return;
+}
+
+static void npc_scan_ldata(struct rvu *rvu, int blkaddr, u8 lid,
+ u8 lt, u64 cfg, u8 intf)
+{
+ struct npc_mcam *mcam = &rvu->hw->mcam;
+ u8 hdr, key, nr_bytes, bit_offset;
+ u8 la_ltype, la_start;
+ /* starting KW index and starting bit position */
+ int start_kwi, offset;
+
+ nr_bytes = FIELD_GET(NPC_BYTESM, cfg) + 1;
+ hdr = FIELD_GET(NPC_HDR_OFFSET, cfg);
+ key = FIELD_GET(NPC_KEY_OFFSET, cfg);
+ start_kwi = key / 8;
+ offset = (key * 8) % 64;
+
+ /* For Tx, Layer A has NIX_INST_HDR_S(64 bytes) preceding
+ * ethernet header.
+ */
+ if (is_npc_intf_tx(intf)) {
+ la_ltype = NPC_LT_LA_IH_NIX_ETHER;
+ la_start = 8;
+ } else {
+ la_ltype = NPC_LT_LA_ETHER;
+ la_start = 0;
+ }
+
+#define NPC_SCAN_HDR(name, hlid, hlt, hstart, hlen) \
+do { \
+ if (lid == (hlid) && lt == (hlt)) { \
+ if ((hstart) >= hdr && \
+ ((hstart) + (hlen)) <= (hdr + nr_bytes)) { \
+ bit_offset = (hdr + nr_bytes - (hstart) - (hlen)) * 8; \
+ npc_set_layer_mdata(mcam, (name), cfg, lid, lt, intf); \
+ npc_set_kw_masks(mcam, (name), (hlen) * 8, \
+ start_kwi, offset + bit_offset, intf);\
+ } \
+ } \
+} while (0)
+
+ /* List LID, LTYPE, start offset from layer and length(in bytes) of
+ * packet header fields below.
+ * Example: Source IP is 4 bytes and starts at 12th byte of IP header
+ */
+ NPC_SCAN_HDR(NPC_SIP_IPV4, NPC_LID_LC, NPC_LT_LC_IP, 12, 4);
+ NPC_SCAN_HDR(NPC_DIP_IPV4, NPC_LID_LC, NPC_LT_LC_IP, 16, 4);
+ NPC_SCAN_HDR(NPC_SIP_IPV6, NPC_LID_LC, NPC_LT_LC_IP6, 8, 16);
+ NPC_SCAN_HDR(NPC_DIP_IPV6, NPC_LID_LC, NPC_LT_LC_IP6, 24, 16);
+ NPC_SCAN_HDR(NPC_SPORT_UDP, NPC_LID_LD, NPC_LT_LD_UDP, 0, 2);
+ NPC_SCAN_HDR(NPC_DPORT_UDP, NPC_LID_LD, NPC_LT_LD_UDP, 2, 2);
+ NPC_SCAN_HDR(NPC_SPORT_TCP, NPC_LID_LD, NPC_LT_LD_TCP, 0, 2);
+ NPC_SCAN_HDR(NPC_DPORT_TCP, NPC_LID_LD, NPC_LT_LD_TCP, 2, 2);
+ NPC_SCAN_HDR(NPC_SPORT_SCTP, NPC_LID_LD, NPC_LT_LD_SCTP, 0, 2);
+ NPC_SCAN_HDR(NPC_DPORT_SCTP, NPC_LID_LD, NPC_LT_LD_SCTP, 2, 2);
+ NPC_SCAN_HDR(NPC_ETYPE_ETHER, NPC_LID_LA, NPC_LT_LA_ETHER, 12, 2);
+ NPC_SCAN_HDR(NPC_ETYPE_TAG1, NPC_LID_LB, NPC_LT_LB_CTAG, 4, 2);
+ NPC_SCAN_HDR(NPC_ETYPE_TAG2, NPC_LID_LB, NPC_LT_LB_STAG_QINQ, 8, 2);
+ NPC_SCAN_HDR(NPC_VLAN_TAG1, NPC_LID_LB, NPC_LT_LB_CTAG, 2, 2);
+ NPC_SCAN_HDR(NPC_VLAN_TAG2, NPC_LID_LB, NPC_LT_LB_STAG_QINQ, 2, 2);
+ NPC_SCAN_HDR(NPC_DMAC, NPC_LID_LA, la_ltype, la_start, 6);
+ NPC_SCAN_HDR(NPC_SMAC, NPC_LID_LA, la_ltype, la_start, 6);
+ /* PF_FUNC is 2 bytes at 0th byte of NPC_LT_LA_IH_NIX_ETHER */
+ NPC_SCAN_HDR(NPC_PF_FUNC, NPC_LID_LA, NPC_LT_LA_IH_NIX_ETHER, 0, 2);
+}
+
+static void npc_set_features(struct rvu *rvu, int blkaddr, u8 intf)
+{
+ struct npc_mcam *mcam = &rvu->hw->mcam;
+ u64 *features = &mcam->rx_features;
+ u64 tcp_udp_sctp;
+ int err, hdr;
+
+ if (is_npc_intf_tx(intf))
+ features = &mcam->tx_features;
+
+ for (hdr = NPC_DMAC; hdr < NPC_HEADER_FIELDS_MAX; hdr++) {
+ err = npc_check_field(rvu, blkaddr, hdr, intf);
+ if (!err)
+ *features |= BIT_ULL(hdr);
+ }
+
+ tcp_udp_sctp = BIT_ULL(NPC_SPORT_TCP) | BIT_ULL(NPC_SPORT_UDP) |
+ BIT_ULL(NPC_DPORT_TCP) | BIT_ULL(NPC_DPORT_UDP) |
+ BIT_ULL(NPC_SPORT_SCTP) | BIT_ULL(NPC_DPORT_SCTP);
+
+ /* for tcp/udp/sctp corresponding layer type should be in the key */
+ if (*features & tcp_udp_sctp)
+ if (npc_check_field(rvu, blkaddr, NPC_LD, intf))
+ *features &= ~tcp_udp_sctp;
+
+ /* for vlan corresponding layer type should be in the key */
+ if (*features & BIT_ULL(NPC_OUTER_VID))
+ if (npc_check_field(rvu, blkaddr, NPC_LB, intf))
+ *features &= ~BIT_ULL(NPC_OUTER_VID);
+}
+
+/* Scan key extraction profile and record how fields of our interest
+ * fill the key structure. Also verify Channel and DMAC exists in
+ * key and not overwritten by other header fields.
+ */
+static int npc_scan_kex(struct rvu *rvu, int blkaddr, u8 intf)
+{
+ struct npc_mcam *mcam = &rvu->hw->mcam;
+ u8 lid, lt, ld, bitnr;
+ u8 key_nibble = 0;
+ u64 cfg;
+
+ /* Scan and note how parse result is going to be in key.
+ * A bit set in PARSE_NIBBLE_ENA corresponds to a nibble from
+ * parse result in the key. The enabled nibbles from parse result
+ * will be concatenated in key.
+ */
+ cfg = rvu_read64(rvu, blkaddr, NPC_AF_INTFX_KEX_CFG(intf));
+ cfg &= NPC_PARSE_NIBBLE;
+ for_each_set_bit(bitnr, (unsigned long *)&cfg, 31) {
+ npc_scan_parse_result(mcam, bitnr, key_nibble, intf);
+ key_nibble++;
+ }
+
+ /* Scan and note how layer data is going to be in key */
+ for (lid = 0; lid < NPC_MAX_LID; lid++) {
+ for (lt = 0; lt < NPC_MAX_LT; lt++) {
+ for (ld = 0; ld < NPC_MAX_LD; ld++) {
+ cfg = rvu_read64(rvu, blkaddr,
+ NPC_AF_INTFX_LIDX_LTX_LDX_CFG
+ (intf, lid, lt, ld));
+ if (!FIELD_GET(NPC_LDATA_EN, cfg))
+ continue;
+ npc_scan_ldata(rvu, blkaddr, lid, lt, cfg,
+ intf);
+ }
+ }
+ }
+
+ return 0;
+}
+
+static int npc_scan_verify_kex(struct rvu *rvu, int blkaddr)
+{
+ int err;
+
+ err = npc_scan_kex(rvu, blkaddr, NIX_INTF_RX);
+ if (err)
+ return err;
+
+ err = npc_scan_kex(rvu, blkaddr, NIX_INTF_TX);
+ if (err)
+ return err;
+
+ /* Channel is mandatory */
+ if (!npc_is_field_present(rvu, NPC_CHAN, NIX_INTF_RX)) {
+ dev_err(rvu->dev, "Channel not present in Key\n");
+ return -EINVAL;
+ }
+ /* check that none of the fields overwrite channel */
+ if (npc_check_overlap(rvu, blkaddr, NPC_CHAN, 0, NIX_INTF_RX)) {
+ dev_err(rvu->dev, "Channel cannot be overwritten\n");
+ return -EINVAL;
+ }
+ /* DMAC should be present in key for unicast filter to work */
+ if (!npc_is_field_present(rvu, NPC_DMAC, NIX_INTF_RX)) {
+ dev_err(rvu->dev, "DMAC not present in Key\n");
+ return -EINVAL;
+ }
+ /* check that none of the fields overwrite DMAC */
+ if (npc_check_overlap(rvu, blkaddr, NPC_DMAC, 0, NIX_INTF_RX)) {
+ dev_err(rvu->dev, "DMAC cannot be overwritten\n");
+ return -EINVAL;
+ }
+
+ npc_set_features(rvu, blkaddr, NIX_INTF_TX);
+ npc_set_features(rvu, blkaddr, NIX_INTF_RX);
+ npc_handle_multi_layer_fields(rvu, blkaddr, NIX_INTF_TX);
+ npc_handle_multi_layer_fields(rvu, blkaddr, NIX_INTF_RX);
+
+ return 0;
+}
+
+int npc_flow_steering_init(struct rvu *rvu, int blkaddr)
+{
+ struct npc_mcam *mcam = &rvu->hw->mcam;
+
+ INIT_LIST_HEAD(&mcam->mcam_rules);
+
+ return npc_scan_verify_kex(rvu, blkaddr);
+}
+
+static int npc_check_unsupported_flows(struct rvu *rvu, u64 features, u8 intf)
+{
+ struct npc_mcam *mcam = &rvu->hw->mcam;
+ u64 *mcam_features = &mcam->rx_features;
+ u64 unsupported;
+ u8 bit;
+
+ if (is_npc_intf_tx(intf))
+ mcam_features = &mcam->tx_features;
+
+ unsupported = (*mcam_features ^ features) & ~(*mcam_features);
+ if (unsupported) {
+ dev_info(rvu->dev, "Unsupported flow(s):\n");
+ for_each_set_bit(bit, (unsigned long *)&unsupported, 64)
+ dev_info(rvu->dev, "%s ", npc_get_field_name(bit));
+ return -EOPNOTSUPP;
+ }
+
+ return 0;
+}
+
+/* npc_update_entry - Based on the masks generated during
+ * the key scanning, updates the given entry with value and
+ * masks for the field of interest. Maximum 16 bytes of a packet
+ * header can be extracted by HW hence lo and hi are sufficient.
+ * When field bytes are less than or equal to 8 then hi should be
+ * 0 for value and mask.
+ *
+ * If exact match of value is required then mask should be all 1's.
+ * If any bits in mask are 0 then corresponding bits in value are
+ * dont care.
+ */
+static void npc_update_entry(struct rvu *rvu, enum key_fields type,
+ struct mcam_entry *entry, u64 val_lo,
+ u64 val_hi, u64 mask_lo, u64 mask_hi, u8 intf)
+{
+ struct npc_mcam *mcam = &rvu->hw->mcam;
+ struct mcam_entry dummy = { {0} };
+ struct npc_key_field *field;
+ u64 kw1, kw2, kw3;
+ u8 shift;
+ int i;
+
+ field = &mcam->rx_key_fields[type];
+ if (is_npc_intf_tx(intf))
+ field = &mcam->tx_key_fields[type];
+
+ if (!field->nr_kws)
+ return;
+
+ for (i = 0; i < NPC_MAX_KWS_IN_KEY; i++) {
+ if (!field->kw_mask[i])
+ continue;
+ /* place key value in kw[x] */
+ shift = __ffs64(field->kw_mask[i]);
+ /* update entry value */
+ kw1 = (val_lo << shift) & field->kw_mask[i];
+ dummy.kw[i] = kw1;
+ /* update entry mask */
+ kw1 = (mask_lo << shift) & field->kw_mask[i];
+ dummy.kw_mask[i] = kw1;
+
+ if (field->nr_kws == 1)
+ break;
+ /* place remaining bits of key value in kw[x + 1] */
+ if (field->nr_kws == 2) {
+ /* update entry value */
+ kw2 = shift ? val_lo >> (64 - shift) : 0;
+ kw2 |= (val_hi << shift);
+ kw2 &= field->kw_mask[i + 1];
+ dummy.kw[i + 1] = kw2;
+ /* update entry mask */
+ kw2 = shift ? mask_lo >> (64 - shift) : 0;
+ kw2 |= (mask_hi << shift);
+ kw2 &= field->kw_mask[i + 1];
+ dummy.kw_mask[i + 1] = kw2;
+ break;
+ }
+ /* place remaining bits of key value in kw[x + 1], kw[x + 2] */
+ if (field->nr_kws == 3) {
+ /* update entry value */
+ kw2 = shift ? val_lo >> (64 - shift) : 0;
+ kw2 |= (val_hi << shift);
+ kw2 &= field->kw_mask[i + 1];
+ kw3 = shift ? val_hi >> (64 - shift) : 0;
+ kw3 &= field->kw_mask[i + 2];
+ dummy.kw[i + 1] = kw2;
+ dummy.kw[i + 2] = kw3;
+ /* update entry mask */
+ kw2 = shift ? mask_lo >> (64 - shift) : 0;
+ kw2 |= (mask_hi << shift);
+ kw2 &= field->kw_mask[i + 1];
+ kw3 = shift ? mask_hi >> (64 - shift) : 0;
+ kw3 &= field->kw_mask[i + 2];
+ dummy.kw_mask[i + 1] = kw2;
+ dummy.kw_mask[i + 2] = kw3;
+ break;
+ }
+ }
+ /* dummy is ready with values and masks for given key
+ * field now clear and update input entry with those
+ */
+ for (i = 0; i < NPC_MAX_KWS_IN_KEY; i++) {
+ if (!field->kw_mask[i])
+ continue;
+ entry->kw[i] &= ~field->kw_mask[i];
+ entry->kw_mask[i] &= ~field->kw_mask[i];
+
+ entry->kw[i] |= dummy.kw[i];
+ entry->kw_mask[i] |= dummy.kw_mask[i];
+ }
+}
+
+#define IPV6_WORDS 4
+
+static void npc_update_ipv6_flow(struct rvu *rvu, struct mcam_entry *entry,
+ u64 features, struct flow_msg *pkt,
+ struct flow_msg *mask,
+ struct rvu_npc_mcam_rule *output, u8 intf)
+{
+ u32 src_ip[IPV6_WORDS], src_ip_mask[IPV6_WORDS];
+ u32 dst_ip[IPV6_WORDS], dst_ip_mask[IPV6_WORDS];
+ struct flow_msg *opkt = &output->packet;
+ struct flow_msg *omask = &output->mask;
+ u64 mask_lo, mask_hi;
+ u64 val_lo, val_hi;
+
+ /* For an ipv6 address fe80::2c68:63ff:fe5e:2d0a the packet
+ * values to be programmed in MCAM should as below:
+ * val_high: 0xfe80000000000000
+ * val_low: 0x2c6863fffe5e2d0a
+ */
+ if (features & BIT_ULL(NPC_SIP_IPV6)) {
+ be32_to_cpu_array(src_ip_mask, mask->ip6src, IPV6_WORDS);
+ be32_to_cpu_array(src_ip, pkt->ip6src, IPV6_WORDS);
+
+ mask_hi = (u64)src_ip_mask[0] << 32 | src_ip_mask[1];
+ mask_lo = (u64)src_ip_mask[2] << 32 | src_ip_mask[3];
+ val_hi = (u64)src_ip[0] << 32 | src_ip[1];
+ val_lo = (u64)src_ip[2] << 32 | src_ip[3];
+
+ npc_update_entry(rvu, NPC_SIP_IPV6, entry, val_lo, val_hi,
+ mask_lo, mask_hi, intf);
+ memcpy(opkt->ip6src, pkt->ip6src, sizeof(opkt->ip6src));
+ memcpy(omask->ip6src, mask->ip6src, sizeof(omask->ip6src));
+ }
+ if (features & BIT_ULL(NPC_DIP_IPV6)) {
+ be32_to_cpu_array(dst_ip_mask, mask->ip6dst, IPV6_WORDS);
+ be32_to_cpu_array(dst_ip, pkt->ip6dst, IPV6_WORDS);
+
+ mask_hi = (u64)dst_ip_mask[0] << 32 | dst_ip_mask[1];
+ mask_lo = (u64)dst_ip_mask[2] << 32 | dst_ip_mask[3];
+ val_hi = (u64)dst_ip[0] << 32 | dst_ip[1];
+ val_lo = (u64)dst_ip[2] << 32 | dst_ip[3];
+
+ npc_update_entry(rvu, NPC_DIP_IPV6, entry, val_lo, val_hi,
+ mask_lo, mask_hi, intf);
+ memcpy(opkt->ip6dst, pkt->ip6dst, sizeof(opkt->ip6dst));
+ memcpy(omask->ip6dst, mask->ip6dst, sizeof(omask->ip6dst));
+ }
+}
+
+static void npc_update_flow(struct rvu *rvu, struct mcam_entry *entry,
+ u64 features, struct flow_msg *pkt,
+ struct flow_msg *mask,
+ struct rvu_npc_mcam_rule *output, u8 intf)
+{
+ u64 dmac_mask = ether_addr_to_u64(mask->dmac);
+ u64 smac_mask = ether_addr_to_u64(mask->smac);
+ u64 dmac_val = ether_addr_to_u64(pkt->dmac);
+ u64 smac_val = ether_addr_to_u64(pkt->smac);
+ struct flow_msg *opkt = &output->packet;
+ struct flow_msg *omask = &output->mask;
+
+ if (!features)
+ return;
+
+ /* For tcp/udp/sctp LTYPE should be present in entry */
+ if (features & (BIT_ULL(NPC_SPORT_TCP) | BIT_ULL(NPC_DPORT_TCP)))
+ npc_update_entry(rvu, NPC_LD, entry, NPC_LT_LD_TCP,
+ 0, ~0ULL, 0, intf);
+ if (features & (BIT_ULL(NPC_SPORT_UDP) | BIT_ULL(NPC_DPORT_UDP)))
+ npc_update_entry(rvu, NPC_LD, entry, NPC_LT_LD_UDP,
+ 0, ~0ULL, 0, intf);
+ if (features & (BIT_ULL(NPC_SPORT_SCTP) | BIT_ULL(NPC_DPORT_SCTP)))
+ npc_update_entry(rvu, NPC_LD, entry, NPC_LT_LD_SCTP,
+ 0, ~0ULL, 0, intf);
+
+ if (features & BIT_ULL(NPC_OUTER_VID))
+ npc_update_entry(rvu, NPC_LB, entry,
+ NPC_LT_LB_STAG_QINQ | NPC_LT_LB_CTAG, 0,
+ NPC_LT_LB_STAG_QINQ & NPC_LT_LB_CTAG, 0, intf);
+
+#define NPC_WRITE_FLOW(field, member, val_lo, val_hi, mask_lo, mask_hi) \
+do { \
+ if (features & BIT_ULL((field))) { \
+ npc_update_entry(rvu, (field), entry, (val_lo), (val_hi), \
+ (mask_lo), (mask_hi), intf); \
+ memcpy(&opkt->member, &pkt->member, sizeof(pkt->member)); \
+ memcpy(&omask->member, &mask->member, sizeof(mask->member)); \
+ } \
+} while (0)
+
+ NPC_WRITE_FLOW(NPC_DMAC, dmac, dmac_val, 0, dmac_mask, 0);
+ NPC_WRITE_FLOW(NPC_SMAC, smac, smac_val, 0, smac_mask, 0);
+ NPC_WRITE_FLOW(NPC_ETYPE, etype, ntohs(pkt->etype), 0,
+ ntohs(mask->etype), 0);
+ NPC_WRITE_FLOW(NPC_SIP_IPV4, ip4src, ntohl(pkt->ip4src), 0,
+ ntohl(mask->ip4src), 0);
+ NPC_WRITE_FLOW(NPC_DIP_IPV4, ip4dst, ntohl(pkt->ip4dst), 0,
+ ntohl(mask->ip4dst), 0);
+ NPC_WRITE_FLOW(NPC_SPORT_TCP, sport, ntohs(pkt->sport), 0,
+ ntohs(mask->sport), 0);
+ NPC_WRITE_FLOW(NPC_SPORT_UDP, sport, ntohs(pkt->sport), 0,
+ ntohs(mask->sport), 0);
+ NPC_WRITE_FLOW(NPC_DPORT_TCP, dport, ntohs(pkt->dport), 0,
+ ntohs(mask->dport), 0);
+ NPC_WRITE_FLOW(NPC_DPORT_UDP, dport, ntohs(pkt->dport), 0,
+ ntohs(mask->dport), 0);
+ NPC_WRITE_FLOW(NPC_SPORT_SCTP, sport, ntohs(pkt->sport), 0,
+ ntohs(mask->sport), 0);
+ NPC_WRITE_FLOW(NPC_DPORT_SCTP, dport, ntohs(pkt->dport), 0,
+ ntohs(mask->dport), 0);
+
+ NPC_WRITE_FLOW(NPC_OUTER_VID, vlan_tci, ntohs(pkt->vlan_tci), 0,
+ ntohs(mask->vlan_tci), 0);
+
+ npc_update_ipv6_flow(rvu, entry, features, pkt, mask, output, intf);
+}
+
+static struct rvu_npc_mcam_rule *rvu_mcam_find_rule(struct npc_mcam *mcam,
+ u16 entry)
+{
+ struct rvu_npc_mcam_rule *iter;
+
+ mutex_lock(&mcam->lock);
+ list_for_each_entry(iter, &mcam->mcam_rules, list) {
+ if (iter->entry == entry) {
+ mutex_unlock(&mcam->lock);
+ return iter;
+ }
+ }
+ mutex_unlock(&mcam->lock);
+
+ return NULL;
+}
+
+static void rvu_mcam_add_rule(struct npc_mcam *mcam,
+ struct rvu_npc_mcam_rule *rule)
+{
+ struct list_head *head = &mcam->mcam_rules;
+ struct rvu_npc_mcam_rule *iter;
+
+ mutex_lock(&mcam->lock);
+ list_for_each_entry(iter, &mcam->mcam_rules, list) {
+ if (iter->entry > rule->entry)
+ break;
+ head = &iter->list;
+ }
+
+ list_add(&rule->list, head);
+ mutex_unlock(&mcam->lock);
+}
+
+static void rvu_mcam_remove_counter_from_rule(struct rvu *rvu, u16 pcifunc,
+ struct rvu_npc_mcam_rule *rule)
+{
+ struct npc_mcam_oper_counter_req free_req = { 0 };
+ struct msg_rsp free_rsp;
+
+ if (!rule->has_cntr)
+ return;
+
+ free_req.hdr.pcifunc = pcifunc;
+ free_req.cntr = rule->cntr;
+
+ rvu_mbox_handler_npc_mcam_free_counter(rvu, &free_req, &free_rsp);
+ rule->has_cntr = false;
+}
+
+static void rvu_mcam_add_counter_to_rule(struct rvu *rvu, u16 pcifunc,
+ struct rvu_npc_mcam_rule *rule,
+ struct npc_install_flow_rsp *rsp)
+{
+ struct npc_mcam_alloc_counter_req cntr_req = { 0 };
+ struct npc_mcam_alloc_counter_rsp cntr_rsp = { 0 };
+ int err;
+
+ cntr_req.hdr.pcifunc = pcifunc;
+ cntr_req.contig = true;
+ cntr_req.count = 1;
+
+ /* we try to allocate a counter to track the stats of this
+ * rule. If counter could not be allocated then proceed
+ * without counter because counters are limited than entries.
+ */
+ err = rvu_mbox_handler_npc_mcam_alloc_counter(rvu, &cntr_req,
+ &cntr_rsp);
+ if (!err && cntr_rsp.count) {
+ rule->cntr = cntr_rsp.cntr;
+ rule->has_cntr = true;
+ rsp->counter = rule->cntr;
+ } else {
+ rsp->counter = err;
+ }
+}
+
+static void npc_update_rx_entry(struct rvu *rvu, struct rvu_pfvf *pfvf,
+ struct mcam_entry *entry,
+ struct npc_install_flow_req *req, u16 target)
+{
+ struct nix_rx_action action;
+
+ npc_update_entry(rvu, NPC_CHAN, entry, req->channel, 0,
+ ~0ULL, 0, NIX_INTF_RX);
+
+ *(u64 *)&action = 0x00;
+ action.pf_func = target;
+ action.op = req->op;
+ action.index = req->index;
+ action.match_id = req->match_id;
+ action.flow_key_alg = req->flow_key_alg;
+
+ if (req->op == NIX_RX_ACTION_DEFAULT && pfvf->def_ucast_rule)
+ action = pfvf->def_ucast_rule->rx_action;
+
+ entry->action = *(u64 *)&action;
+
+ /* VTAG0 starts at 0th byte of LID_B.
+ * VTAG1 starts at 4th byte of LID_B.
+ */
+ entry->vtag_action = FIELD_PREP(RX_VTAG0_VALID_BIT, req->vtag0_valid) |
+ FIELD_PREP(RX_VTAG0_TYPE_MASK, req->vtag0_type) |
+ FIELD_PREP(RX_VTAG0_LID_MASK, NPC_LID_LB) |
+ FIELD_PREP(RX_VTAG0_RELPTR_MASK, 0) |
+ FIELD_PREP(RX_VTAG1_VALID_BIT, req->vtag1_valid) |
+ FIELD_PREP(RX_VTAG1_TYPE_MASK, req->vtag1_type) |
+ FIELD_PREP(RX_VTAG1_LID_MASK, NPC_LID_LB) |
+ FIELD_PREP(RX_VTAG1_RELPTR_MASK, 4);
+}
+
+static void npc_update_tx_entry(struct rvu *rvu, struct rvu_pfvf *pfvf,
+ struct mcam_entry *entry,
+ struct npc_install_flow_req *req, u16 target)
+{
+ struct nix_tx_action action;
+
+ npc_update_entry(rvu, NPC_PF_FUNC, entry, (__force u16)htons(target),
+ 0, ~0ULL, 0, NIX_INTF_TX);
+
+ *(u64 *)&action = 0x00;
+ action.op = req->op;
+ action.index = req->index;
+ action.match_id = req->match_id;
+
+ entry->action = *(u64 *)&action;
+
+ /* VTAG0 starts at 0th byte of LID_B.
+ * VTAG1 starts at 4th byte of LID_B.
+ */
+ entry->vtag_action = FIELD_PREP(TX_VTAG0_DEF_MASK, req->vtag0_def) |
+ FIELD_PREP(TX_VTAG0_OP_MASK, req->vtag0_op) |
+ FIELD_PREP(TX_VTAG0_LID_MASK, NPC_LID_LA) |
+ FIELD_PREP(TX_VTAG0_RELPTR_MASK, 20) |
+ FIELD_PREP(TX_VTAG1_DEF_MASK, req->vtag1_def) |
+ FIELD_PREP(TX_VTAG1_OP_MASK, req->vtag1_op) |
+ FIELD_PREP(TX_VTAG1_LID_MASK, NPC_LID_LA) |
+ FIELD_PREP(TX_VTAG1_RELPTR_MASK, 24);
+}
+
+static int npc_install_flow(struct rvu *rvu, int blkaddr, u16 target,
+ int nixlf, struct rvu_pfvf *pfvf,
+ struct npc_install_flow_req *req,
+ struct npc_install_flow_rsp *rsp, bool enable,
+ bool pf_set_vfs_mac)
+{
+ struct rvu_npc_mcam_rule *def_ucast_rule = pfvf->def_ucast_rule;
+ u64 features, installed_features, missing_features = 0;
+ struct npc_mcam_write_entry_req write_req = { 0 };
+ struct npc_mcam *mcam = &rvu->hw->mcam;
+ struct rvu_npc_mcam_rule dummy = { 0 };
+ struct rvu_npc_mcam_rule *rule;
+ bool new = false, msg_from_vf;
+ u16 owner = req->hdr.pcifunc;
+ struct msg_rsp write_rsp;
+ struct mcam_entry *entry;
+ int entry_index, err;
+
+ msg_from_vf = !!(owner & RVU_PFVF_FUNC_MASK);
+
+ installed_features = req->features;
+ features = req->features;
+ entry = &write_req.entry_data;
+ entry_index = req->entry;
+
+ npc_update_flow(rvu, entry, features, &req->packet, &req->mask, &dummy,
+ req->intf);
+
+ if (is_npc_intf_rx(req->intf))
+ npc_update_rx_entry(rvu, pfvf, entry, req, target);
+ else
+ npc_update_tx_entry(rvu, pfvf, entry, req, target);
+
+ /* Default unicast rules do not exist for TX */
+ if (is_npc_intf_tx(req->intf))
+ goto find_rule;
+
+ if (def_ucast_rule)
+ missing_features = (def_ucast_rule->features ^ features) &
+ def_ucast_rule->features;
+
+ if (req->default_rule && req->append) {
+ /* add to default rule */
+ if (missing_features)
+ npc_update_flow(rvu, entry, missing_features,
+ &def_ucast_rule->packet,
+ &def_ucast_rule->mask,
+ &dummy, req->intf);
+ enable = rvu_npc_write_default_rule(rvu, blkaddr,
+ nixlf, target,
+ pfvf->nix_rx_intf, entry,
+ &entry_index);
+ installed_features = req->features | missing_features;
+ } else if (req->default_rule && !req->append) {
+ /* overwrite default rule */
+ enable = rvu_npc_write_default_rule(rvu, blkaddr,
+ nixlf, target,
+ pfvf->nix_rx_intf, entry,
+ &entry_index);
+ } else if (msg_from_vf) {
+ /* normal rule - include default rule also to it for VF */
+ npc_update_flow(rvu, entry, missing_features,
+ &def_ucast_rule->packet, &def_ucast_rule->mask,
+ &dummy, req->intf);
+ installed_features = req->features | missing_features;
+ }
+
+find_rule:
+ rule = rvu_mcam_find_rule(mcam, entry_index);
+ if (!rule) {
+ rule = kzalloc(sizeof(*rule), GFP_KERNEL);
+ if (!rule)
+ return -ENOMEM;
+ new = true;
+ }
+ /* no counter for default rule */
+ if (req->default_rule)
+ goto update_rule;
+
+ /* allocate new counter if rule has no counter */
+ if (req->set_cntr && !rule->has_cntr)
+ rvu_mcam_add_counter_to_rule(rvu, owner, rule, rsp);
+
+ /* if user wants to delete an existing counter for a rule then
+ * free the counter
+ */
+ if (!req->set_cntr && rule->has_cntr)
+ rvu_mcam_remove_counter_from_rule(rvu, owner, rule);
+
+ write_req.hdr.pcifunc = owner;
+ write_req.entry = req->entry;
+ write_req.intf = req->intf;
+ write_req.enable_entry = (u8)enable;
+ /* if counter is available then clear and use it */
+ if (req->set_cntr && rule->has_cntr) {
+ rvu_write64(rvu, blkaddr, NPC_AF_MATCH_STATX(rule->cntr), 0x00);
+ write_req.set_cntr = 1;
+ write_req.cntr = rule->cntr;
+ }
+
+ err = rvu_mbox_handler_npc_mcam_write_entry(rvu, &write_req,
+ &write_rsp);
+ if (err) {
+ rvu_mcam_remove_counter_from_rule(rvu, owner, rule);
+ if (new)
+ kfree(rule);
+ return err;
+ }
+update_rule:
+ memcpy(&rule->packet, &dummy.packet, sizeof(rule->packet));
+ memcpy(&rule->mask, &dummy.mask, sizeof(rule->mask));
+ rule->entry = entry_index;
+ memcpy(&rule->rx_action, &entry->action, sizeof(struct nix_rx_action));
+ if (is_npc_intf_tx(req->intf))
+ memcpy(&rule->tx_action, &entry->action,
+ sizeof(struct nix_tx_action));
+ rule->vtag_action = entry->vtag_action;
+ rule->features = installed_features;
+ rule->default_rule = req->default_rule;
+ rule->owner = owner;
+ rule->enable = enable;
+ if (is_npc_intf_tx(req->intf))
+ rule->intf = pfvf->nix_tx_intf;
+ else
+ rule->intf = pfvf->nix_rx_intf;
+
+ if (new)
+ rvu_mcam_add_rule(mcam, rule);
+ if (req->default_rule)
+ pfvf->def_ucast_rule = rule;
+
+ /* VF's MAC address is being changed via PF */
+ if (pf_set_vfs_mac) {
+ ether_addr_copy(pfvf->default_mac, req->packet.dmac);
+ ether_addr_copy(pfvf->mac_addr, req->packet.dmac);
+ }
+
+ if (pfvf->pf_set_vf_cfg && req->vtag0_type == NIX_AF_LFX_RX_VTAG_TYPE7)
+ rule->vfvlan_cfg = true;
+
+ return 0;
+}
+
+int rvu_mbox_handler_npc_install_flow(struct rvu *rvu,
+ struct npc_install_flow_req *req,
+ struct npc_install_flow_rsp *rsp)
+{
+ bool from_vf = !!(req->hdr.pcifunc & RVU_PFVF_FUNC_MASK);
+ int blkaddr, nixlf, err;
+ struct rvu_pfvf *pfvf;
+ bool pf_set_vfs_mac = false;
+ bool enable = true;
+ u16 target;
+
+ blkaddr = rvu_get_blkaddr(rvu, BLKTYPE_NPC, 0);
+ if (blkaddr < 0) {
+ dev_err(rvu->dev, "%s: NPC block not implemented\n", __func__);
+ return -ENODEV;
+ }
+
+ if (!is_npc_interface_valid(rvu, req->intf))
+ return -EINVAL;
+
+ if (from_vf && req->default_rule)
+ return NPC_MCAM_PERM_DENIED;
+
+ /* Each PF/VF info is maintained in struct rvu_pfvf.
+ * rvu_pfvf for the target PF/VF needs to be retrieved
+ * hence modify pcifunc accordingly.
+ */
+
+ /* AF installing for a PF/VF */
+ if (!req->hdr.pcifunc)
+ target = req->vf;
+ /* PF installing for its VF */
+ else if (!from_vf && req->vf) {
+ target = (req->hdr.pcifunc & ~RVU_PFVF_FUNC_MASK) | req->vf;
+ pf_set_vfs_mac = req->default_rule &&
+ (req->features & BIT_ULL(NPC_DMAC));
+ }
+ /* msg received from PF/VF */
+ else
+ target = req->hdr.pcifunc;
+
+ if (npc_check_unsupported_flows(rvu, req->features, req->intf))
+ return -EOPNOTSUPP;
+
+ if (npc_mcam_verify_channel(rvu, target, req->intf, req->channel))
+ return -EINVAL;
+
+ pfvf = rvu_get_pfvf(rvu, target);
+
+ /* PF installing for its VF */
+ if (req->hdr.pcifunc && !from_vf && req->vf)
+ pfvf->pf_set_vf_cfg = 1;
+
+ /* update req destination mac addr */
+ if ((req->features & BIT_ULL(NPC_DMAC)) && is_npc_intf_rx(req->intf) &&
+ is_zero_ether_addr(req->packet.dmac)) {
+ ether_addr_copy(req->packet.dmac, pfvf->mac_addr);
+ eth_broadcast_addr((u8 *)&req->mask.dmac);
+ }
+
+ err = nix_get_nixlf(rvu, target, &nixlf, NULL);
+
+ /* If interface is uninitialized then do not enable entry */
+ if (err || (!req->default_rule && !pfvf->def_ucast_rule))
+ enable = false;
+
+ /* Packets reaching NPC in Tx path implies that a
+ * NIXLF is properly setup and transmitting.
+ * Hence rules can be enabled for Tx.
+ */
+ if (is_npc_intf_tx(req->intf))
+ enable = true;
+
+ /* Do not allow requests from uninitialized VFs */
+ if (from_vf && !enable)
+ return -EINVAL;
+
+ /* If message is from VF then its flow should not overlap with
+ * reserved unicast flow.
+ */
+ if (from_vf && pfvf->def_ucast_rule && is_npc_intf_rx(req->intf) &&
+ pfvf->def_ucast_rule->features & req->features)
+ return -EINVAL;
+
+ return npc_install_flow(rvu, blkaddr, target, nixlf, pfvf, req, rsp,
+ enable, pf_set_vfs_mac);
+}
+
+static int npc_delete_flow(struct rvu *rvu, struct rvu_npc_mcam_rule *rule,
+ u16 pcifunc)
+{
+ struct npc_mcam_ena_dis_entry_req dis_req = { 0 };
+ struct msg_rsp dis_rsp;
+
+ if (rule->default_rule)
+ return 0;
+
+ if (rule->has_cntr)
+ rvu_mcam_remove_counter_from_rule(rvu, pcifunc, rule);
+
+ dis_req.hdr.pcifunc = pcifunc;
+ dis_req.entry = rule->entry;
+
+ list_del(&rule->list);
+ kfree(rule);
+
+ return rvu_mbox_handler_npc_mcam_dis_entry(rvu, &dis_req, &dis_rsp);
+}
+
+int rvu_mbox_handler_npc_delete_flow(struct rvu *rvu,
+ struct npc_delete_flow_req *req,
+ struct msg_rsp *rsp)
+{
+ struct npc_mcam *mcam = &rvu->hw->mcam;
+ struct rvu_npc_mcam_rule *iter, *tmp;
+ u16 pcifunc = req->hdr.pcifunc;
+ struct list_head del_list;
+
+ INIT_LIST_HEAD(&del_list);
+
+ mutex_lock(&mcam->lock);
+ list_for_each_entry_safe(iter, tmp, &mcam->mcam_rules, list) {
+ if (iter->owner == pcifunc) {
+ /* All rules */
+ if (req->all) {
+ list_move_tail(&iter->list, &del_list);
+ /* Range of rules */
+ } else if (req->end && iter->entry >= req->start &&
+ iter->entry <= req->end) {
+ list_move_tail(&iter->list, &del_list);
+ /* single rule */
+ } else if (req->entry == iter->entry) {
+ list_move_tail(&iter->list, &del_list);
+ break;
+ }
+ }
+ }
+ mutex_unlock(&mcam->lock);
+
+ list_for_each_entry_safe(iter, tmp, &del_list, list) {
+ u16 entry = iter->entry;
+
+ /* clear the mcam entry target pcifunc */
+ mcam->entry2target_pffunc[entry] = 0x0;
+ if (npc_delete_flow(rvu, iter, pcifunc))
+ dev_err(rvu->dev, "rule deletion failed for entry:%u",
+ entry);
+ }
+
+ return 0;
+}
+
+static int npc_update_dmac_value(struct rvu *rvu, int npcblkaddr,
+ struct rvu_npc_mcam_rule *rule,
+ struct rvu_pfvf *pfvf)
+{
+ struct npc_mcam_write_entry_req write_req = { 0 };
+ struct mcam_entry *entry = &write_req.entry_data;
+ struct npc_mcam *mcam = &rvu->hw->mcam;
+ struct msg_rsp rsp;
+ u8 intf, enable;
+ int err;
+
+ ether_addr_copy(rule->packet.dmac, pfvf->mac_addr);
+
+ npc_read_mcam_entry(rvu, mcam, npcblkaddr, rule->entry,
+ entry, &intf, &enable);
+
+ npc_update_entry(rvu, NPC_DMAC, entry,
+ ether_addr_to_u64(pfvf->mac_addr), 0,
+ 0xffffffffffffull, 0, intf);
+
+ write_req.hdr.pcifunc = rule->owner;
+ write_req.entry = rule->entry;
+
+ mutex_unlock(&mcam->lock);
+ err = rvu_mbox_handler_npc_mcam_write_entry(rvu, &write_req, &rsp);
+ mutex_lock(&mcam->lock);
+
+ return err;
+}
+
+void npc_mcam_enable_flows(struct rvu *rvu, u16 target)
+{
+ struct rvu_pfvf *pfvf = rvu_get_pfvf(rvu, target);
+ struct rvu_npc_mcam_rule *def_ucast_rule;
+ struct npc_mcam *mcam = &rvu->hw->mcam;
+ struct rvu_npc_mcam_rule *rule;
+ int blkaddr, bank, index;
+ u64 def_action;
+
+ blkaddr = rvu_get_blkaddr(rvu, BLKTYPE_NPC, 0);
+ if (blkaddr < 0)
+ return;
+
+ def_ucast_rule = pfvf->def_ucast_rule;
+
+ mutex_lock(&mcam->lock);
+ list_for_each_entry(rule, &mcam->mcam_rules, list) {
+ if (is_npc_intf_rx(rule->intf) &&
+ rule->rx_action.pf_func == target && !rule->enable) {
+ if (rule->default_rule) {
+ npc_enable_mcam_entry(rvu, mcam, blkaddr,
+ rule->entry, true);
+ rule->enable = true;
+ continue;
+ }
+
+ if (rule->vfvlan_cfg)
+ npc_update_dmac_value(rvu, blkaddr, rule, pfvf);
+
+ if (rule->rx_action.op == NIX_RX_ACTION_DEFAULT) {
+ if (!def_ucast_rule)
+ continue;
+ /* Use default unicast entry action */
+ rule->rx_action = def_ucast_rule->rx_action;
+ def_action = *(u64 *)&def_ucast_rule->rx_action;
+ bank = npc_get_bank(mcam, rule->entry);
+ rvu_write64(rvu, blkaddr,
+ NPC_AF_MCAMEX_BANKX_ACTION
+ (rule->entry, bank), def_action);
+ }
+
+ npc_enable_mcam_entry(rvu, mcam, blkaddr,
+ rule->entry, true);
+ rule->enable = true;
+ }
+ }
+
+ /* Enable MCAM entries installed by PF with target as VF pcifunc */
+ for (index = 0; index < mcam->bmap_entries; index++) {
+ if (mcam->entry2target_pffunc[index] == target)
+ npc_enable_mcam_entry(rvu, mcam, blkaddr,
+ index, true);
+ }
+ mutex_unlock(&mcam->lock);
+}
+
+void npc_mcam_disable_flows(struct rvu *rvu, u16 target)
+{
+ struct npc_mcam *mcam = &rvu->hw->mcam;
+ int blkaddr, index;
+
+ blkaddr = rvu_get_blkaddr(rvu, BLKTYPE_NPC, 0);
+ if (blkaddr < 0)
+ return;
+
+ mutex_lock(&mcam->lock);
+ /* Disable MCAM entries installed by PF with target as VF pcifunc */
+ for (index = 0; index < mcam->bmap_entries; index++) {
+ if (mcam->entry2target_pffunc[index] == target)
+ npc_enable_mcam_entry(rvu, mcam, blkaddr,
+ index, false);
+ }
+ mutex_unlock(&mcam->lock);
+}
#define TIM_AF_LF_RST (0x20)
/* CPT */
-#define CPT_AF_CONSTANTS0 (0x0000)
-#define CPT_PRIV_LFX_CFG (0x41000)
-#define CPT_PRIV_LFX_INT_CFG (0x43000)
-#define CPT_AF_RVU_LF_CFG_DEBUG (0x45000)
-#define CPT_AF_LF_RST (0x44000)
-#define CPT_AF_BLK_RST (0x46000)
+#define CPT_AF_CONSTANTS0 (0x0000)
+#define CPT_AF_CONSTANTS1 (0x1000)
+#define CPT_AF_DIAG (0x3000)
+#define CPT_AF_ECO (0x4000)
+#define CPT_AF_FLTX_INT(a) (0xa000ull | (u64)(a) << 3)
+#define CPT_AF_FLTX_INT_W1S(a) (0xb000ull | (u64)(a) << 3)
+#define CPT_AF_FLTX_INT_ENA_W1C(a) (0xc000ull | (u64)(a) << 3)
+#define CPT_AF_FLTX_INT_ENA_W1S(a) (0xd000ull | (u64)(a) << 3)
+#define CPT_AF_PSNX_EXE(a) (0xe000ull | (u64)(a) << 3)
+#define CPT_AF_PSNX_EXE_W1S(a) (0xf000ull | (u64)(a) << 3)
+#define CPT_AF_PSNX_LF(a) (0x10000ull | (u64)(a) << 3)
+#define CPT_AF_PSNX_LF_W1S(a) (0x11000ull | (u64)(a) << 3)
+#define CPT_AF_EXEX_CTL2(a) (0x12000ull | (u64)(a) << 3)
+#define CPT_AF_EXEX_STS(a) (0x13000ull | (u64)(a) << 3)
+#define CPT_AF_EXE_ERR_INFO (0x14000)
+#define CPT_AF_EXEX_ACTIVE(a) (0x16000ull | (u64)(a) << 3)
+#define CPT_AF_INST_REQ_PC (0x17000)
+#define CPT_AF_INST_LATENCY_PC (0x18000)
+#define CPT_AF_RD_REQ_PC (0x19000)
+#define CPT_AF_RD_LATENCY_PC (0x1a000)
+#define CPT_AF_RD_UC_PC (0x1b000)
+#define CPT_AF_ACTIVE_CYCLES_PC (0x1c000)
+#define CPT_AF_EXE_DBG_CTL (0x1d000)
+#define CPT_AF_EXE_DBG_DATA (0x1e000)
+#define CPT_AF_EXE_REQ_TIMER (0x1f000)
+#define CPT_AF_EXEX_CTL(a) (0x20000ull | (u64)(a) << 3)
+#define CPT_AF_EXE_PERF_CTL (0x21000)
+#define CPT_AF_EXE_DBG_CNTX(a) (0x22000ull | (u64)(a) << 3)
+#define CPT_AF_EXE_PERF_EVENT_CNT (0x23000)
+#define CPT_AF_EXE_EPCI_INBX_CNT(a) (0x24000ull | (u64)(a) << 3)
+#define CPT_AF_EXE_EPCI_OUTBX_CNT(a) (0x25000ull | (u64)(a) << 3)
+#define CPT_AF_EXEX_UCODE_BASE(a) (0x26000ull | (u64)(a) << 3)
+#define CPT_AF_LFX_CTL(a) (0x27000ull | (u64)(a) << 3)
+#define CPT_AF_LFX_CTL2(a) (0x29000ull | (u64)(a) << 3)
+#define CPT_AF_CPTCLK_CNT (0x2a000)
+#define CPT_AF_PF_FUNC (0x2b000)
+#define CPT_AF_LFX_PTR_CTL(a) (0x2c000ull | (u64)(a) << 3)
+#define CPT_AF_GRPX_THR(a) (0x2d000ull | (u64)(a) << 3)
+#define CPT_AF_CTL (0x2e000ull)
+#define CPT_AF_XEX_THR(a) (0x2f000ull | (u64)(a) << 3)
+#define CPT_PRIV_LFX_CFG (0x41000)
+#define CPT_PRIV_AF_INT_CFG (0x42000)
+#define CPT_PRIV_LFX_INT_CFG (0x43000)
+#define CPT_AF_LF_RST (0x44000)
+#define CPT_AF_RVU_LF_CFG_DEBUG (0x45000)
+#define CPT_AF_BLK_RST (0x46000)
+#define CPT_AF_RVU_INT (0x47000)
+#define CPT_AF_RVU_INT_W1S (0x47008)
+#define CPT_AF_RVU_INT_ENA_W1S (0x47010)
+#define CPT_AF_RVU_INT_ENA_W1C (0x47018)
+#define CPT_AF_RAS_INT (0x47020)
+#define CPT_AF_RAS_INT_W1S (0x47028)
+#define CPT_AF_RAS_INT_ENA_W1S (0x47030)
+#define CPT_AF_RAS_INT_ENA_W1C (0x47038)
+
+#define CPT_AF_LF_CTL2_SHIFT 3
+#define CPT_AF_LF_SSO_PF_FUNC_SHIFT 32
#define NPC_AF_BLK_RST (0x00040)
VTAGSIZE_T4 = 0x0,
VTAGSIZE_T8 = 0x1,
};
+
+enum nix_tx_vtag_op {
+ NOP = 0x0,
+ VTAG_INSERT = 0x1,
+ VTAG_REPLACE = 0x2,
+};
+
+/* NIX RX VTAG actions */
+#define VTAG_STRIP BIT_ULL(4)
+#define VTAG_CAPTURE BIT_ULL(5)
+
#endif /* RVU_STRUCT_H */
obj-$(CONFIG_OCTEONTX2_VF) += octeontx2_nicvf.o
octeontx2_nicpf-y := otx2_pf.o otx2_common.o otx2_txrx.o otx2_ethtool.o \
- otx2_ptp.o
+ otx2_ptp.o otx2_flows.o
octeontx2_nicvf-y := otx2_vf.o
ccflags-y += -I$(srctree)/drivers/net/ethernet/marvell/octeontx2/af
if (!is_valid_ether_addr(addr->sa_data))
return -EADDRNOTAVAIL;
- if (!otx2_hw_set_mac_addr(pfvf, addr->sa_data))
+ if (!otx2_hw_set_mac_addr(pfvf, addr->sa_data)) {
memcpy(netdev->dev_addr, addr->sa_data, netdev->addr_len);
- else
+ /* update dmac field in vlan offload rule */
+ if (pfvf->flags & OTX2_FLAG_RX_VLAN_SUPPORT)
+ otx2_install_rxvlan_offload_flow(pfvf);
+ } else {
return -EPERM;
+ }
return 0;
}
rss->flowkey_cfg = rss->enable ? rss->flowkey_cfg :
NIX_FLOW_KEY_TYPE_IPV4 | NIX_FLOW_KEY_TYPE_IPV6 |
NIX_FLOW_KEY_TYPE_TCP | NIX_FLOW_KEY_TYPE_UDP |
- NIX_FLOW_KEY_TYPE_SCTP | NIX_FLOW_KEY_TYPE_VLAN;
+ NIX_FLOW_KEY_TYPE_SCTP | NIX_FLOW_KEY_TYPE_VLAN |
+ NIX_FLOW_KEY_TYPE_IPV4_PROTO;
ret = otx2_set_flowkey_cfg(pfvf);
if (ret)
#ifndef OTX2_COMMON_H
#define OTX2_COMMON_H
+#include <linux/ethtool.h>
#include <linux/pci.h>
#include <linux/iommu.h>
#include <linux/net_tstamp.h>
#include <linux/ptp_clock_kernel.h>
#include <linux/timecounter.h>
+#include <linux/soc/marvell/octeontx2/asm.h>
#include <mbox.h>
+#include <npc.h>
#include "otx2_reg.h"
#include "otx2_txrx.h"
#include <rvu_trace.h>
struct otx2_nic *pf;
struct delayed_work link_event_work;
bool intf_down; /* interface was either configured or not */
+ u8 mac[ETH_ALEN];
+ u16 vlan;
+ int tx_vtag_idx;
};
struct flr_work {
#define OTX2_HW_TIMESTAMP_LEN 8
+struct otx2_mac_table {
+ u8 addr[ETH_ALEN];
+ u16 mcam_entry;
+ bool inuse;
+};
+
+struct otx2_flow_config {
+ u16 entry[NPC_MAX_NONCONTIG_ENTRIES];
+ u32 nr_flows;
+#define OTX2_MAX_NTUPLE_FLOWS 32
+#define OTX2_MAX_UNICAST_FLOWS 8
+#define OTX2_MAX_VLAN_FLOWS 1
+#define OTX2_MCAM_COUNT (OTX2_MAX_NTUPLE_FLOWS + \
+ OTX2_MAX_UNICAST_FLOWS + \
+ OTX2_MAX_VLAN_FLOWS)
+ u32 ntuple_offset;
+ u32 unicast_offset;
+ u32 rx_vlan_offset;
+ u32 vf_vlan_offset;
+#define OTX2_PER_VF_VLAN_FLOWS 2 /* rx+tx per VF */
+#define OTX2_VF_VLAN_RX_INDEX 0
+#define OTX2_VF_VLAN_TX_INDEX 1
+ u32 ntuple_max_flows;
+ struct list_head flow_list;
+};
+
struct otx2_nic {
void __iomem *reg_base;
struct net_device *netdev;
#define OTX2_FLAG_RX_TSTAMP_ENABLED BIT_ULL(0)
#define OTX2_FLAG_TX_TSTAMP_ENABLED BIT_ULL(1)
#define OTX2_FLAG_INTF_DOWN BIT_ULL(2)
+#define OTX2_FLAG_MCAM_ENTRIES_ALLOC BIT_ULL(3)
+#define OTX2_FLAG_NTUPLE_SUPPORT BIT_ULL(4)
+#define OTX2_FLAG_UCAST_FLTR_SUPPORT BIT_ULL(5)
+#define OTX2_FLAG_RX_VLAN_SUPPORT BIT_ULL(6)
+#define OTX2_FLAG_VF_VLAN_SUPPORT BIT_ULL(7)
+#define OTX2_FLAG_PF_SHUTDOWN BIT_ULL(8)
#define OTX2_FLAG_RX_PAUSE_ENABLED BIT_ULL(9)
#define OTX2_FLAG_TX_PAUSE_ENABLED BIT_ULL(10)
u64 flags;
struct refill_work *refill_wrk;
struct workqueue_struct *otx2_wq;
struct work_struct rx_mode_work;
+ struct otx2_mac_table *mac_table;
/* Ethtool stuff */
u32 msg_enable;
struct otx2_ptp *ptp;
struct hwtstamp_config tstamp;
+
+ struct otx2_flow_config *flow_cfg;
};
static inline bool is_otx2_lbkvf(struct pci_dev *pdev)
return result;
}
-static inline u64 otx2_lmt_flush(uint64_t addr)
-{
- u64 result = 0;
-
- __asm__ volatile(".cpu generic+lse\n"
- "ldeor xzr,%x[rf],[%[rs]]"
- : [rf]"=r"(result)
- : [rs]"r"(addr));
- return result;
-}
-
#else
#define otx2_write128(lo, hi, addr)
#define otx2_atomic64_add(incr, ptr) ({ *ptr += incr; })
-#define otx2_lmt_flush(addr) ({ 0; })
#endif
/* Alloc pointer from pool/aura */
int otx2_stop(struct net_device *netdev);
int otx2_set_real_num_queues(struct net_device *netdev,
int tx_queues, int rx_queues);
+/* MCAM filter related APIs */
+int otx2_mcam_flow_init(struct otx2_nic *pf);
+int otx2_alloc_mcam_entries(struct otx2_nic *pfvf);
+void otx2_mcam_flow_del(struct otx2_nic *pf);
+int otx2_destroy_ntuple_flows(struct otx2_nic *pf);
+int otx2_destroy_mcam_flows(struct otx2_nic *pfvf);
+int otx2_get_flow(struct otx2_nic *pfvf,
+ struct ethtool_rxnfc *nfc, u32 location);
+int otx2_get_all_flows(struct otx2_nic *pfvf,
+ struct ethtool_rxnfc *nfc, u32 *rule_locs);
+int otx2_add_flow(struct otx2_nic *pfvf,
+ struct ethtool_rx_flow_spec *fsp);
+int otx2_remove_flow(struct otx2_nic *pfvf, u32 location);
+int otx2_prepare_flow_request(struct ethtool_rx_flow_spec *fsp,
+ struct npc_install_flow_req *req);
+int otx2_del_macfilter(struct net_device *netdev, const u8 *mac);
+int otx2_add_macfilter(struct net_device *netdev, const u8 *mac);
+int otx2_enable_rxvlan(struct otx2_nic *pf, bool enable);
+int otx2_install_rxvlan_offload_flow(struct otx2_nic *pfvf);
+
#endif /* OTX2_COMMON_H */
nfc->data = pfvf->hw.rx_queues;
ret = 0;
break;
+ case ETHTOOL_GRXCLSRLCNT:
+ nfc->rule_cnt = pfvf->flow_cfg->nr_flows;
+ ret = 0;
+ break;
+ case ETHTOOL_GRXCLSRULE:
+ ret = otx2_get_flow(pfvf, nfc, nfc->fs.location);
+ break;
+ case ETHTOOL_GRXCLSRLALL:
+ ret = otx2_get_all_flows(pfvf, nfc, rules);
+ break;
case ETHTOOL_GRXFH:
return otx2_get_rss_hash_opts(pfvf, nfc);
default:
}
static int otx2_set_rxnfc(struct net_device *dev, struct ethtool_rxnfc *nfc)
+{
+ bool ntuple = !!(dev->features & NETIF_F_NTUPLE);
+ struct otx2_nic *pfvf = netdev_priv(dev);
+ int ret = -EOPNOTSUPP;
+
+ switch (nfc->cmd) {
+ case ETHTOOL_SRXFH:
+ ret = otx2_set_rss_hash_opts(pfvf, nfc);
+ break;
+ case ETHTOOL_SRXCLSRLINS:
+ if (netif_running(dev) && ntuple)
+ ret = otx2_add_flow(pfvf, &nfc->fs);
+ break;
+ case ETHTOOL_SRXCLSRLDEL:
+ if (netif_running(dev) && ntuple)
+ ret = otx2_remove_flow(pfvf, nfc->fs.location);
+ break;
+ default:
+ break;
+ }
+
+ return ret;
+}
+
+static int otx2vf_get_rxnfc(struct net_device *dev,
+ struct ethtool_rxnfc *nfc, u32 *rules)
+{
+ struct otx2_nic *pfvf = netdev_priv(dev);
+ int ret = -EOPNOTSUPP;
+
+ switch (nfc->cmd) {
+ case ETHTOOL_GRXRINGS:
+ nfc->data = pfvf->hw.rx_queues;
+ ret = 0;
+ break;
+ case ETHTOOL_GRXFH:
+ return otx2_get_rss_hash_opts(pfvf, nfc);
+ default:
+ break;
+ }
+ return ret;
+}
+
+static int otx2vf_set_rxnfc(struct net_device *dev, struct ethtool_rxnfc *nfc)
{
struct otx2_nic *pfvf = netdev_priv(dev);
int ret = -EOPNOTSUPP;
.get_sset_count = otx2vf_get_sset_count,
.set_channels = otx2_set_channels,
.get_channels = otx2_get_channels,
- .get_rxnfc = otx2_get_rxnfc,
- .set_rxnfc = otx2_set_rxnfc,
+ .get_rxnfc = otx2vf_get_rxnfc,
+ .set_rxnfc = otx2vf_set_rxnfc,
.get_rxfh_key_size = otx2_get_rxfh_key_size,
.get_rxfh_indir_size = otx2_get_rxfh_indir_size,
.get_rxfh = otx2_get_rxfh,
--- /dev/null
+// SPDX-License-Identifier: GPL-2.0
+/* Marvell OcteonTx2 RVU Physcial Function ethernet driver
+ *
+ * Copyright (C) 2020 Marvell.
+ */
+
+#include <net/ipv6.h>
+
+#include "otx2_common.h"
+
+#define OTX2_DEFAULT_ACTION 0x1
+
+struct otx2_flow {
+ struct ethtool_rx_flow_spec flow_spec;
+ struct list_head list;
+ u32 location;
+ u16 entry;
+ bool is_vf;
+ int vf;
+};
+
+int otx2_alloc_mcam_entries(struct otx2_nic *pfvf)
+{
+ struct otx2_flow_config *flow_cfg = pfvf->flow_cfg;
+ struct npc_mcam_alloc_entry_req *req;
+ struct npc_mcam_alloc_entry_rsp *rsp;
+ int vf_vlan_max_flows;
+ int i;
+
+ mutex_lock(&pfvf->mbox.lock);
+
+ req = otx2_mbox_alloc_msg_npc_mcam_alloc_entry(&pfvf->mbox);
+ if (!req) {
+ mutex_unlock(&pfvf->mbox.lock);
+ return -ENOMEM;
+ }
+
+ vf_vlan_max_flows = pfvf->total_vfs * OTX2_PER_VF_VLAN_FLOWS;
+ req->contig = false;
+ req->count = OTX2_MCAM_COUNT + vf_vlan_max_flows;
+
+ /* Send message to AF */
+ if (otx2_sync_mbox_msg(&pfvf->mbox)) {
+ mutex_unlock(&pfvf->mbox.lock);
+ return -EINVAL;
+ }
+
+ rsp = (struct npc_mcam_alloc_entry_rsp *)otx2_mbox_get_rsp
+ (&pfvf->mbox.mbox, 0, &req->hdr);
+
+ if (rsp->count != req->count) {
+ netdev_info(pfvf->netdev,
+ "Unable to allocate %d MCAM entries, got %d\n",
+ req->count, rsp->count);
+ /* support only ntuples here */
+ flow_cfg->ntuple_max_flows = rsp->count;
+ flow_cfg->ntuple_offset = 0;
+ pfvf->flags |= OTX2_FLAG_NTUPLE_SUPPORT;
+ } else {
+ flow_cfg->vf_vlan_offset = 0;
+ flow_cfg->ntuple_offset = flow_cfg->vf_vlan_offset +
+ vf_vlan_max_flows;
+ flow_cfg->unicast_offset = flow_cfg->ntuple_offset +
+ OTX2_MAX_NTUPLE_FLOWS;
+ flow_cfg->rx_vlan_offset = flow_cfg->unicast_offset +
+ OTX2_MAX_UNICAST_FLOWS;
+ pfvf->flags |= OTX2_FLAG_NTUPLE_SUPPORT;
+ pfvf->flags |= OTX2_FLAG_UCAST_FLTR_SUPPORT;
+ pfvf->flags |= OTX2_FLAG_RX_VLAN_SUPPORT;
+ pfvf->flags |= OTX2_FLAG_VF_VLAN_SUPPORT;
+ }
+
+ for (i = 0; i < rsp->count; i++)
+ flow_cfg->entry[i] = rsp->entry_list[i];
+
+ pfvf->flags |= OTX2_FLAG_MCAM_ENTRIES_ALLOC;
+
+ mutex_unlock(&pfvf->mbox.lock);
+
+ return 0;
+}
+
+int otx2_mcam_flow_init(struct otx2_nic *pf)
+{
+ int err;
+
+ pf->flow_cfg = devm_kzalloc(pf->dev, sizeof(struct otx2_flow_config),
+ GFP_KERNEL);
+ if (!pf->flow_cfg)
+ return -ENOMEM;
+
+ INIT_LIST_HEAD(&pf->flow_cfg->flow_list);
+
+ pf->flow_cfg->ntuple_max_flows = OTX2_MAX_NTUPLE_FLOWS;
+
+ err = otx2_alloc_mcam_entries(pf);
+ if (err)
+ return err;
+
+ pf->mac_table = devm_kzalloc(pf->dev, sizeof(struct otx2_mac_table)
+ * OTX2_MAX_UNICAST_FLOWS, GFP_KERNEL);
+ if (!pf->mac_table)
+ return -ENOMEM;
+
+ return 0;
+}
+
+void otx2_mcam_flow_del(struct otx2_nic *pf)
+{
+ otx2_destroy_mcam_flows(pf);
+}
+
+/* On success adds mcam entry
+ * On failure enable promisous mode
+ */
+static int otx2_do_add_macfilter(struct otx2_nic *pf, const u8 *mac)
+{
+ struct otx2_flow_config *flow_cfg = pf->flow_cfg;
+ struct npc_install_flow_req *req;
+ int err, i;
+
+ if (!(pf->flags & OTX2_FLAG_UCAST_FLTR_SUPPORT))
+ return -ENOMEM;
+
+ /* dont have free mcam entries or uc list is greater than alloted */
+ if (netdev_uc_count(pf->netdev) > OTX2_MAX_UNICAST_FLOWS)
+ return -ENOMEM;
+
+ mutex_lock(&pf->mbox.lock);
+ req = otx2_mbox_alloc_msg_npc_install_flow(&pf->mbox);
+ if (!req) {
+ mutex_unlock(&pf->mbox.lock);
+ return -ENOMEM;
+ }
+
+ /* unicast offset starts with 32 0..31 for ntuple */
+ for (i = 0; i < OTX2_MAX_UNICAST_FLOWS; i++) {
+ if (pf->mac_table[i].inuse)
+ continue;
+ ether_addr_copy(pf->mac_table[i].addr, mac);
+ pf->mac_table[i].inuse = true;
+ pf->mac_table[i].mcam_entry =
+ flow_cfg->entry[i + flow_cfg->unicast_offset];
+ req->entry = pf->mac_table[i].mcam_entry;
+ break;
+ }
+
+ ether_addr_copy(req->packet.dmac, mac);
+ eth_broadcast_addr((u8 *)&req->mask.dmac);
+ req->features = BIT_ULL(NPC_DMAC);
+ req->channel = pf->hw.rx_chan_base;
+ req->intf = NIX_INTF_RX;
+ req->op = NIX_RX_ACTION_DEFAULT;
+ req->set_cntr = 1;
+
+ err = otx2_sync_mbox_msg(&pf->mbox);
+ mutex_unlock(&pf->mbox.lock);
+
+ return err;
+}
+
+int otx2_add_macfilter(struct net_device *netdev, const u8 *mac)
+{
+ struct otx2_nic *pf = netdev_priv(netdev);
+
+ return otx2_do_add_macfilter(pf, mac);
+}
+
+static bool otx2_get_mcamentry_for_mac(struct otx2_nic *pf, const u8 *mac,
+ int *mcam_entry)
+{
+ int i;
+
+ for (i = 0; i < OTX2_MAX_UNICAST_FLOWS; i++) {
+ if (!pf->mac_table[i].inuse)
+ continue;
+
+ if (ether_addr_equal(pf->mac_table[i].addr, mac)) {
+ *mcam_entry = pf->mac_table[i].mcam_entry;
+ pf->mac_table[i].inuse = false;
+ return true;
+ }
+ }
+ return false;
+}
+
+int otx2_del_macfilter(struct net_device *netdev, const u8 *mac)
+{
+ struct otx2_nic *pf = netdev_priv(netdev);
+ struct npc_delete_flow_req *req;
+ int err, mcam_entry;
+
+ /* check does mcam entry exists for given mac */
+ if (!otx2_get_mcamentry_for_mac(pf, mac, &mcam_entry))
+ return 0;
+
+ mutex_lock(&pf->mbox.lock);
+ req = otx2_mbox_alloc_msg_npc_delete_flow(&pf->mbox);
+ if (!req) {
+ mutex_unlock(&pf->mbox.lock);
+ return -ENOMEM;
+ }
+ req->entry = mcam_entry;
+ /* Send message to AF */
+ err = otx2_sync_mbox_msg(&pf->mbox);
+ mutex_unlock(&pf->mbox.lock);
+
+ return err;
+}
+
+static struct otx2_flow *otx2_find_flow(struct otx2_nic *pfvf, u32 location)
+{
+ struct otx2_flow *iter;
+
+ list_for_each_entry(iter, &pfvf->flow_cfg->flow_list, list) {
+ if (iter->location == location)
+ return iter;
+ }
+
+ return NULL;
+}
+
+static void otx2_add_flow_to_list(struct otx2_nic *pfvf, struct otx2_flow *flow)
+{
+ struct list_head *head = &pfvf->flow_cfg->flow_list;
+ struct otx2_flow *iter;
+
+ list_for_each_entry(iter, &pfvf->flow_cfg->flow_list, list) {
+ if (iter->location > flow->location)
+ break;
+ head = &iter->list;
+ }
+
+ list_add(&flow->list, head);
+}
+
+int otx2_get_flow(struct otx2_nic *pfvf, struct ethtool_rxnfc *nfc,
+ u32 location)
+{
+ struct otx2_flow *iter;
+
+ if (location >= pfvf->flow_cfg->ntuple_max_flows)
+ return -EINVAL;
+
+ list_for_each_entry(iter, &pfvf->flow_cfg->flow_list, list) {
+ if (iter->location == location) {
+ nfc->fs = iter->flow_spec;
+ return 0;
+ }
+ }
+
+ return -ENOENT;
+}
+
+int otx2_get_all_flows(struct otx2_nic *pfvf, struct ethtool_rxnfc *nfc,
+ u32 *rule_locs)
+{
+ u32 location = 0;
+ int idx = 0;
+ int err = 0;
+
+ nfc->data = pfvf->flow_cfg->ntuple_max_flows;
+ while ((!err || err == -ENOENT) && idx < nfc->rule_cnt) {
+ err = otx2_get_flow(pfvf, nfc, location);
+ if (!err)
+ rule_locs[idx++] = location;
+ location++;
+ }
+
+ return err;
+}
+
+static void otx2_prepare_ipv4_flow(struct ethtool_rx_flow_spec *fsp,
+ struct npc_install_flow_req *req,
+ u32 flow_type)
+{
+ struct ethtool_usrip4_spec *ipv4_usr_mask = &fsp->m_u.usr_ip4_spec;
+ struct ethtool_usrip4_spec *ipv4_usr_hdr = &fsp->h_u.usr_ip4_spec;
+ struct ethtool_tcpip4_spec *ipv4_l4_mask = &fsp->m_u.tcp_ip4_spec;
+ struct ethtool_tcpip4_spec *ipv4_l4_hdr = &fsp->h_u.tcp_ip4_spec;
+ struct flow_msg *pmask = &req->mask;
+ struct flow_msg *pkt = &req->packet;
+
+ switch (flow_type) {
+ case IP_USER_FLOW:
+ if (ipv4_usr_mask->ip4src) {
+ memcpy(&pkt->ip4src, &ipv4_usr_hdr->ip4src,
+ sizeof(pkt->ip4src));
+ memcpy(&pmask->ip4src, &ipv4_usr_mask->ip4src,
+ sizeof(pmask->ip4src));
+ req->features |= BIT_ULL(NPC_SIP_IPV4);
+ }
+ if (ipv4_usr_mask->ip4dst) {
+ memcpy(&pkt->ip4dst, &ipv4_usr_hdr->ip4dst,
+ sizeof(pkt->ip4dst));
+ memcpy(&pmask->ip4dst, &ipv4_usr_mask->ip4dst,
+ sizeof(pmask->ip4dst));
+ req->features |= BIT_ULL(NPC_DIP_IPV4);
+ }
+ break;
+ case TCP_V4_FLOW:
+ case UDP_V4_FLOW:
+ case SCTP_V4_FLOW:
+ if (ipv4_l4_mask->ip4src) {
+ memcpy(&pkt->ip4src, &ipv4_l4_hdr->ip4src,
+ sizeof(pkt->ip4src));
+ memcpy(&pmask->ip4src, &ipv4_l4_mask->ip4src,
+ sizeof(pmask->ip4src));
+ req->features |= BIT_ULL(NPC_SIP_IPV4);
+ }
+ if (ipv4_l4_mask->ip4dst) {
+ memcpy(&pkt->ip4dst, &ipv4_l4_hdr->ip4dst,
+ sizeof(pkt->ip4dst));
+ memcpy(&pmask->ip4dst, &ipv4_l4_mask->ip4dst,
+ sizeof(pmask->ip4dst));
+ req->features |= BIT_ULL(NPC_DIP_IPV4);
+ }
+ if (ipv4_l4_mask->psrc) {
+ memcpy(&pkt->sport, &ipv4_l4_hdr->psrc,
+ sizeof(pkt->sport));
+ memcpy(&pmask->sport, &ipv4_l4_mask->psrc,
+ sizeof(pmask->sport));
+ if (flow_type == UDP_V4_FLOW)
+ req->features |= BIT_ULL(NPC_SPORT_UDP);
+ else if (flow_type == TCP_V4_FLOW)
+ req->features |= BIT_ULL(NPC_SPORT_TCP);
+ else
+ req->features |= BIT_ULL(NPC_SPORT_SCTP);
+ }
+ if (ipv4_l4_mask->pdst) {
+ memcpy(&pkt->dport, &ipv4_l4_hdr->pdst,
+ sizeof(pkt->dport));
+ memcpy(&pmask->dport, &ipv4_l4_mask->pdst,
+ sizeof(pmask->dport));
+ if (flow_type == UDP_V4_FLOW)
+ req->features |= BIT_ULL(NPC_DPORT_UDP);
+ else if (flow_type == TCP_V4_FLOW)
+ req->features |= BIT_ULL(NPC_DPORT_TCP);
+ else
+ req->features |= BIT_ULL(NPC_DPORT_SCTP);
+ }
+ break;
+ default:
+ break;
+ }
+}
+
+static void otx2_prepare_ipv6_flow(struct ethtool_rx_flow_spec *fsp,
+ struct npc_install_flow_req *req,
+ u32 flow_type)
+{
+ struct ethtool_usrip6_spec *ipv6_usr_mask = &fsp->m_u.usr_ip6_spec;
+ struct ethtool_usrip6_spec *ipv6_usr_hdr = &fsp->h_u.usr_ip6_spec;
+ struct ethtool_tcpip6_spec *ipv6_l4_mask = &fsp->m_u.tcp_ip6_spec;
+ struct ethtool_tcpip6_spec *ipv6_l4_hdr = &fsp->h_u.tcp_ip6_spec;
+ struct flow_msg *pmask = &req->mask;
+ struct flow_msg *pkt = &req->packet;
+
+ switch (flow_type) {
+ case IPV6_USER_FLOW:
+ if (!ipv6_addr_any((struct in6_addr *)ipv6_usr_mask->ip6src)) {
+ memcpy(&pkt->ip6src, &ipv6_usr_hdr->ip6src,
+ sizeof(pkt->ip6src));
+ memcpy(&pmask->ip6src, &ipv6_usr_mask->ip6src,
+ sizeof(pmask->ip6src));
+ req->features |= BIT_ULL(NPC_SIP_IPV6);
+ }
+ if (!ipv6_addr_any((struct in6_addr *)ipv6_usr_mask->ip6dst)) {
+ memcpy(&pkt->ip6dst, &ipv6_usr_hdr->ip6dst,
+ sizeof(pkt->ip6dst));
+ memcpy(&pmask->ip6dst, &ipv6_usr_mask->ip6dst,
+ sizeof(pmask->ip6dst));
+ req->features |= BIT_ULL(NPC_DIP_IPV6);
+ }
+ break;
+ case TCP_V6_FLOW:
+ case UDP_V6_FLOW:
+ case SCTP_V6_FLOW:
+ if (!ipv6_addr_any((struct in6_addr *)ipv6_l4_mask->ip6src)) {
+ memcpy(&pkt->ip6src, &ipv6_l4_hdr->ip6src,
+ sizeof(pkt->ip6src));
+ memcpy(&pmask->ip6src, &ipv6_l4_mask->ip6src,
+ sizeof(pmask->ip6src));
+ req->features |= BIT_ULL(NPC_SIP_IPV6);
+ }
+ if (!ipv6_addr_any((struct in6_addr *)ipv6_l4_mask->ip6dst)) {
+ memcpy(&pkt->ip6dst, &ipv6_l4_hdr->ip6dst,
+ sizeof(pkt->ip6dst));
+ memcpy(&pmask->ip6dst, &ipv6_l4_mask->ip6dst,
+ sizeof(pmask->ip6dst));
+ req->features |= BIT_ULL(NPC_DIP_IPV6);
+ }
+ if (ipv6_l4_mask->psrc) {
+ memcpy(&pkt->sport, &ipv6_l4_hdr->psrc,
+ sizeof(pkt->sport));
+ memcpy(&pmask->sport, &ipv6_l4_mask->psrc,
+ sizeof(pmask->sport));
+ if (flow_type == UDP_V6_FLOW)
+ req->features |= BIT_ULL(NPC_SPORT_UDP);
+ else if (flow_type == TCP_V6_FLOW)
+ req->features |= BIT_ULL(NPC_SPORT_TCP);
+ else
+ req->features |= BIT_ULL(NPC_SPORT_SCTP);
+ }
+ if (ipv6_l4_mask->pdst) {
+ memcpy(&pkt->dport, &ipv6_l4_hdr->pdst,
+ sizeof(pkt->dport));
+ memcpy(&pmask->dport, &ipv6_l4_mask->pdst,
+ sizeof(pmask->dport));
+ if (flow_type == UDP_V6_FLOW)
+ req->features |= BIT_ULL(NPC_DPORT_UDP);
+ else if (flow_type == TCP_V6_FLOW)
+ req->features |= BIT_ULL(NPC_DPORT_TCP);
+ else
+ req->features |= BIT_ULL(NPC_DPORT_SCTP);
+ }
+ break;
+ default:
+ break;
+ }
+}
+
+int otx2_prepare_flow_request(struct ethtool_rx_flow_spec *fsp,
+ struct npc_install_flow_req *req)
+{
+ struct ethhdr *eth_mask = &fsp->m_u.ether_spec;
+ struct ethhdr *eth_hdr = &fsp->h_u.ether_spec;
+ struct flow_msg *pmask = &req->mask;
+ struct flow_msg *pkt = &req->packet;
+ u32 flow_type;
+
+ flow_type = fsp->flow_type & ~(FLOW_EXT | FLOW_MAC_EXT);
+ switch (flow_type) {
+ /* bits not set in mask are don't care */
+ case ETHER_FLOW:
+ if (!is_zero_ether_addr(eth_mask->h_source)) {
+ ether_addr_copy(pkt->smac, eth_hdr->h_source);
+ ether_addr_copy(pmask->smac, eth_mask->h_source);
+ req->features |= BIT_ULL(NPC_SMAC);
+ }
+ if (!is_zero_ether_addr(eth_mask->h_dest)) {
+ ether_addr_copy(pkt->dmac, eth_hdr->h_dest);
+ ether_addr_copy(pmask->dmac, eth_mask->h_dest);
+ req->features |= BIT_ULL(NPC_DMAC);
+ }
+ if (eth_mask->h_proto) {
+ memcpy(&pkt->etype, ð_hdr->h_proto,
+ sizeof(pkt->etype));
+ memcpy(&pmask->etype, ð_mask->h_proto,
+ sizeof(pmask->etype));
+ req->features |= BIT_ULL(NPC_ETYPE);
+ }
+ break;
+ case IP_USER_FLOW:
+ case TCP_V4_FLOW:
+ case UDP_V4_FLOW:
+ case SCTP_V4_FLOW:
+ otx2_prepare_ipv4_flow(fsp, req, flow_type);
+ break;
+ case IPV6_USER_FLOW:
+ case TCP_V6_FLOW:
+ case UDP_V6_FLOW:
+ case SCTP_V6_FLOW:
+ otx2_prepare_ipv6_flow(fsp, req, flow_type);
+ break;
+ default:
+ return -EOPNOTSUPP;
+ }
+ if (fsp->flow_type & FLOW_EXT) {
+ if (fsp->m_ext.vlan_etype)
+ return -EINVAL;
+ if (fsp->m_ext.vlan_tci) {
+ if (fsp->m_ext.vlan_tci != cpu_to_be16(VLAN_VID_MASK))
+ return -EINVAL;
+ if (be16_to_cpu(fsp->h_ext.vlan_tci) >= VLAN_N_VID)
+ return -EINVAL;
+
+ memcpy(&pkt->vlan_tci, &fsp->h_ext.vlan_tci,
+ sizeof(pkt->vlan_tci));
+ memcpy(&pmask->vlan_tci, &fsp->m_ext.vlan_tci,
+ sizeof(pmask->vlan_tci));
+ req->features |= BIT_ULL(NPC_OUTER_VID);
+ }
+
+ /* Not Drop/Direct to queue but use action in default entry */
+ if (fsp->m_ext.data[1] &&
+ fsp->h_ext.data[1] == cpu_to_be32(OTX2_DEFAULT_ACTION))
+ req->op = NIX_RX_ACTION_DEFAULT;
+ }
+
+ if (fsp->flow_type & FLOW_MAC_EXT &&
+ !is_zero_ether_addr(fsp->m_ext.h_dest)) {
+ ether_addr_copy(pkt->dmac, fsp->h_ext.h_dest);
+ ether_addr_copy(pmask->dmac, fsp->m_ext.h_dest);
+ req->features |= BIT_ULL(NPC_DMAC);
+ }
+
+ if (!req->features)
+ return -EOPNOTSUPP;
+
+ return 0;
+}
+
+static int otx2_add_flow_msg(struct otx2_nic *pfvf, struct otx2_flow *flow)
+{
+ u64 ring_cookie = flow->flow_spec.ring_cookie;
+ struct npc_install_flow_req *req;
+ int err, vf = 0;
+
+ mutex_lock(&pfvf->mbox.lock);
+ req = otx2_mbox_alloc_msg_npc_install_flow(&pfvf->mbox);
+ if (!req) {
+ mutex_unlock(&pfvf->mbox.lock);
+ return -ENOMEM;
+ }
+
+ err = otx2_prepare_flow_request(&flow->flow_spec, req);
+ if (err) {
+ /* free the allocated msg above */
+ otx2_mbox_reset(&pfvf->mbox.mbox, 0);
+ mutex_unlock(&pfvf->mbox.lock);
+ return err;
+ }
+
+ req->entry = flow->entry;
+ req->intf = NIX_INTF_RX;
+ req->set_cntr = 1;
+ req->channel = pfvf->hw.rx_chan_base;
+ if (ring_cookie == RX_CLS_FLOW_DISC) {
+ req->op = NIX_RX_ACTIONOP_DROP;
+ } else {
+ /* change to unicast only if action of default entry is not
+ * requested by user
+ */
+ if (req->op != NIX_RX_ACTION_DEFAULT)
+ req->op = NIX_RX_ACTIONOP_UCAST;
+ req->index = ethtool_get_flow_spec_ring(ring_cookie);
+ vf = ethtool_get_flow_spec_ring_vf(ring_cookie);
+ if (vf > pci_num_vf(pfvf->pdev)) {
+ mutex_unlock(&pfvf->mbox.lock);
+ return -EINVAL;
+ }
+ }
+
+ /* ethtool ring_cookie has (VF + 1) for VF */
+ if (vf) {
+ req->vf = vf;
+ flow->is_vf = true;
+ flow->vf = vf;
+ }
+
+ /* Send message to AF */
+ err = otx2_sync_mbox_msg(&pfvf->mbox);
+ mutex_unlock(&pfvf->mbox.lock);
+ return err;
+}
+
+int otx2_add_flow(struct otx2_nic *pfvf, struct ethtool_rx_flow_spec *fsp)
+{
+ struct otx2_flow_config *flow_cfg = pfvf->flow_cfg;
+ u32 ring = ethtool_get_flow_spec_ring(fsp->ring_cookie);
+ struct otx2_flow *flow;
+ bool new = false;
+ int err;
+
+ if (!(pfvf->flags & OTX2_FLAG_NTUPLE_SUPPORT))
+ return -ENOMEM;
+
+ if (ring >= pfvf->hw.rx_queues && fsp->ring_cookie != RX_CLS_FLOW_DISC)
+ return -EINVAL;
+
+ if (fsp->location >= flow_cfg->ntuple_max_flows)
+ return -EINVAL;
+
+ flow = otx2_find_flow(pfvf, fsp->location);
+ if (!flow) {
+ flow = kzalloc(sizeof(*flow), GFP_ATOMIC);
+ if (!flow)
+ return -ENOMEM;
+ flow->location = fsp->location;
+ flow->entry = flow_cfg->entry[flow_cfg->ntuple_offset +
+ flow->location];
+ new = true;
+ }
+ /* struct copy */
+ flow->flow_spec = *fsp;
+
+ err = otx2_add_flow_msg(pfvf, flow);
+ if (err) {
+ if (new)
+ kfree(flow);
+ return err;
+ }
+
+ /* add the new flow installed to list */
+ if (new) {
+ otx2_add_flow_to_list(pfvf, flow);
+ flow_cfg->nr_flows++;
+ }
+
+ return 0;
+}
+
+static int otx2_remove_flow_msg(struct otx2_nic *pfvf, u16 entry, bool all)
+{
+ struct npc_delete_flow_req *req;
+ int err;
+
+ mutex_lock(&pfvf->mbox.lock);
+ req = otx2_mbox_alloc_msg_npc_delete_flow(&pfvf->mbox);
+ if (!req) {
+ mutex_unlock(&pfvf->mbox.lock);
+ return -ENOMEM;
+ }
+
+ req->entry = entry;
+ if (all)
+ req->all = 1;
+
+ /* Send message to AF */
+ err = otx2_sync_mbox_msg(&pfvf->mbox);
+ mutex_unlock(&pfvf->mbox.lock);
+ return err;
+}
+
+int otx2_remove_flow(struct otx2_nic *pfvf, u32 location)
+{
+ struct otx2_flow_config *flow_cfg = pfvf->flow_cfg;
+ struct otx2_flow *flow;
+ int err;
+
+ if (location >= flow_cfg->ntuple_max_flows)
+ return -EINVAL;
+
+ flow = otx2_find_flow(pfvf, location);
+ if (!flow)
+ return -ENOENT;
+
+ err = otx2_remove_flow_msg(pfvf, flow->entry, false);
+ if (err)
+ return err;
+
+ list_del(&flow->list);
+ kfree(flow);
+ flow_cfg->nr_flows--;
+
+ return 0;
+}
+
+int otx2_destroy_ntuple_flows(struct otx2_nic *pfvf)
+{
+ struct otx2_flow_config *flow_cfg = pfvf->flow_cfg;
+ struct npc_delete_flow_req *req;
+ struct otx2_flow *iter, *tmp;
+ int err;
+
+ if (!(pfvf->flags & OTX2_FLAG_NTUPLE_SUPPORT))
+ return 0;
+
+ mutex_lock(&pfvf->mbox.lock);
+ req = otx2_mbox_alloc_msg_npc_delete_flow(&pfvf->mbox);
+ if (!req) {
+ mutex_unlock(&pfvf->mbox.lock);
+ return -ENOMEM;
+ }
+
+ req->start = flow_cfg->entry[flow_cfg->ntuple_offset];
+ req->end = flow_cfg->entry[flow_cfg->ntuple_offset +
+ flow_cfg->ntuple_max_flows - 1];
+ err = otx2_sync_mbox_msg(&pfvf->mbox);
+ mutex_unlock(&pfvf->mbox.lock);
+
+ list_for_each_entry_safe(iter, tmp, &flow_cfg->flow_list, list) {
+ list_del(&iter->list);
+ kfree(iter);
+ flow_cfg->nr_flows--;
+ }
+ return err;
+}
+
+int otx2_destroy_mcam_flows(struct otx2_nic *pfvf)
+{
+ struct otx2_flow_config *flow_cfg = pfvf->flow_cfg;
+ struct npc_mcam_free_entry_req *req;
+ struct otx2_flow *iter, *tmp;
+ int err;
+
+ if (!(pfvf->flags & OTX2_FLAG_MCAM_ENTRIES_ALLOC))
+ return 0;
+
+ /* remove all flows */
+ err = otx2_remove_flow_msg(pfvf, 0, true);
+ if (err)
+ return err;
+
+ list_for_each_entry_safe(iter, tmp, &flow_cfg->flow_list, list) {
+ list_del(&iter->list);
+ kfree(iter);
+ flow_cfg->nr_flows--;
+ }
+
+ mutex_lock(&pfvf->mbox.lock);
+ req = otx2_mbox_alloc_msg_npc_mcam_free_entry(&pfvf->mbox);
+ if (!req) {
+ mutex_unlock(&pfvf->mbox.lock);
+ return -ENOMEM;
+ }
+
+ req->all = 1;
+ /* Send message to AF to free MCAM entries */
+ err = otx2_sync_mbox_msg(&pfvf->mbox);
+ if (err) {
+ mutex_unlock(&pfvf->mbox.lock);
+ return err;
+ }
+
+ pfvf->flags &= ~OTX2_FLAG_MCAM_ENTRIES_ALLOC;
+ mutex_unlock(&pfvf->mbox.lock);
+
+ return 0;
+}
+
+int otx2_install_rxvlan_offload_flow(struct otx2_nic *pfvf)
+{
+ struct otx2_flow_config *flow_cfg = pfvf->flow_cfg;
+ struct npc_install_flow_req *req;
+ int err;
+
+ mutex_lock(&pfvf->mbox.lock);
+ req = otx2_mbox_alloc_msg_npc_install_flow(&pfvf->mbox);
+ if (!req) {
+ mutex_unlock(&pfvf->mbox.lock);
+ return -ENOMEM;
+ }
+
+ req->entry = flow_cfg->entry[flow_cfg->rx_vlan_offset];
+ req->intf = NIX_INTF_RX;
+ ether_addr_copy(req->packet.dmac, pfvf->netdev->dev_addr);
+ eth_broadcast_addr((u8 *)&req->mask.dmac);
+ req->channel = pfvf->hw.rx_chan_base;
+ req->op = NIX_RX_ACTION_DEFAULT;
+ req->features = BIT_ULL(NPC_OUTER_VID) | BIT_ULL(NPC_DMAC);
+ req->vtag0_valid = true;
+ req->vtag0_type = NIX_AF_LFX_RX_VTAG_TYPE0;
+
+ /* Send message to AF */
+ err = otx2_sync_mbox_msg(&pfvf->mbox);
+ mutex_unlock(&pfvf->mbox.lock);
+ return err;
+}
+
+static int otx2_delete_rxvlan_offload_flow(struct otx2_nic *pfvf)
+{
+ struct otx2_flow_config *flow_cfg = pfvf->flow_cfg;
+ struct npc_delete_flow_req *req;
+ int err;
+
+ mutex_lock(&pfvf->mbox.lock);
+ req = otx2_mbox_alloc_msg_npc_delete_flow(&pfvf->mbox);
+ if (!req) {
+ mutex_unlock(&pfvf->mbox.lock);
+ return -ENOMEM;
+ }
+
+ req->entry = flow_cfg->entry[flow_cfg->rx_vlan_offset];
+ /* Send message to AF */
+ err = otx2_sync_mbox_msg(&pfvf->mbox);
+ mutex_unlock(&pfvf->mbox.lock);
+ return err;
+}
+
+int otx2_enable_rxvlan(struct otx2_nic *pf, bool enable)
+{
+ struct nix_vtag_config *req;
+ struct mbox_msghdr *rsp_hdr;
+ int err;
+
+ /* Dont have enough mcam entries */
+ if (!(pf->flags & OTX2_FLAG_RX_VLAN_SUPPORT))
+ return -ENOMEM;
+
+ if (enable) {
+ err = otx2_install_rxvlan_offload_flow(pf);
+ if (err)
+ return err;
+ } else {
+ err = otx2_delete_rxvlan_offload_flow(pf);
+ if (err)
+ return err;
+ }
+
+ mutex_lock(&pf->mbox.lock);
+ req = otx2_mbox_alloc_msg_nix_vtag_cfg(&pf->mbox);
+ if (!req) {
+ mutex_unlock(&pf->mbox.lock);
+ return -ENOMEM;
+ }
+
+ /* config strip, capture and size */
+ req->vtag_size = VTAGSIZE_T4;
+ req->cfg_type = 1; /* rx vlan cfg */
+ req->rx.vtag_type = NIX_AF_LFX_RX_VTAG_TYPE0;
+ req->rx.strip_vtag = enable;
+ req->rx.capture_vtag = enable;
+
+ err = otx2_sync_mbox_msg(&pf->mbox);
+ if (err) {
+ mutex_unlock(&pf->mbox.lock);
+ return err;
+ }
+
+ rsp_hdr = otx2_mbox_get_rsp(&pf->mbox.mbox, 0, &req->hdr);
+ if (IS_ERR(rsp_hdr)) {
+ mutex_unlock(&pf->mbox.lock);
+ return PTR_ERR(rsp_hdr);
+ }
+
+ mutex_unlock(&pf->mbox.lock);
+ return rsp_hdr->rc;
+}
static int otx2_init_hw_resources(struct otx2_nic *pf)
{
+ struct nix_lf_free_req *free_req;
struct mbox *mbox = &pf->mbox;
struct otx2_hw *hw = &pf->hw;
struct msg_req *req;
otx2_aura_pool_free(pf);
err_free_nix_lf:
mutex_lock(&mbox->lock);
- req = otx2_mbox_alloc_msg_nix_lf_free(mbox);
- if (req) {
+ free_req = otx2_mbox_alloc_msg_nix_lf_free(mbox);
+ if (free_req) {
+ free_req->flags = NIX_LF_DISABLE_FLOWS;
if (otx2_sync_mbox_msg(mbox))
dev_err(pf->dev, "%s failed to free nixlf\n", __func__);
}
static void otx2_free_hw_resources(struct otx2_nic *pf)
{
struct otx2_qset *qset = &pf->qset;
+ struct nix_lf_free_req *free_req;
struct mbox *mbox = &pf->mbox;
struct otx2_cq_queue *cq;
struct msg_req *req;
mutex_lock(&mbox->lock);
/* Reset NIX LF */
- req = otx2_mbox_alloc_msg_nix_lf_free(mbox);
- if (req) {
+ free_req = otx2_mbox_alloc_msg_nix_lf_free(mbox);
+ if (free_req) {
+ free_req->flags = NIX_LF_DISABLE_FLOWS;
+ if (!(pf->flags & OTX2_FLAG_PF_SHUTDOWN))
+ free_req->flags |= NIX_LF_DONT_FREE_TX_VTAG;
if (otx2_sync_mbox_msg(mbox))
dev_err(pf->dev, "%s failed to free nixlf\n", __func__);
}
otx2_set_cints_affinity(pf);
+ if (pf->flags & OTX2_FLAG_RX_VLAN_SUPPORT)
+ otx2_enable_rxvlan(pf, true);
+
/* When reinitializing enable time stamping if it is enabled before */
if (pf->flags & OTX2_FLAG_TX_TSTAMP_ENABLED) {
pf->flags &= ~OTX2_FLAG_TX_TSTAMP_ENABLED;
struct otx2_nic *pf = container_of(work, struct otx2_nic, rx_mode_work);
struct net_device *netdev = pf->netdev;
struct nix_rx_mode *req;
+ bool promisc = false;
if (!(netdev->flags & IFF_UP))
return;
+ if ((netdev->flags & IFF_PROMISC) ||
+ (netdev_uc_count(netdev) > OTX2_MAX_UNICAST_FLOWS)) {
+ promisc = true;
+ }
+
+ /* Write unicast address to mcam entries or del from mcam */
+ if (!promisc && netdev->priv_flags & IFF_UNICAST_FLT)
+ __dev_uc_sync(netdev, otx2_add_macfilter, otx2_del_macfilter);
+
mutex_lock(&pf->mbox.lock);
req = otx2_mbox_alloc_msg_nix_set_rx_mode(&pf->mbox);
if (!req) {
req->mode = NIX_RX_MODE_UCAST;
- /* We don't support MAC address filtering yet */
- if (netdev->flags & IFF_PROMISC)
+ if (promisc)
req->mode |= NIX_RX_MODE_PROMISC;
else if (netdev->flags & (IFF_ALLMULTI | IFF_MULTICAST))
req->mode |= NIX_RX_MODE_ALLMULTI;
netdev_features_t features)
{
netdev_features_t changed = features ^ netdev->features;
+ bool ntuple = !!(features & NETIF_F_NTUPLE);
struct otx2_nic *pf = netdev_priv(netdev);
if ((changed & NETIF_F_LOOPBACK) && netif_running(netdev))
return otx2_cgx_config_loopback(pf,
features & NETIF_F_LOOPBACK);
+
+ if ((changed & NETIF_F_HW_VLAN_CTAG_RX) && netif_running(netdev))
+ return otx2_enable_rxvlan(pf,
+ features & NETIF_F_HW_VLAN_CTAG_RX);
+
+ if ((changed & NETIF_F_NTUPLE) && !ntuple)
+ otx2_destroy_ntuple_flows(pf);
+
return 0;
}
}
}
+static int otx2_do_set_vf_mac(struct otx2_nic *pf, int vf, const u8 *mac)
+{
+ struct npc_install_flow_req *req;
+ int err;
+
+ mutex_lock(&pf->mbox.lock);
+ req = otx2_mbox_alloc_msg_npc_install_flow(&pf->mbox);
+ if (!req) {
+ err = -ENOMEM;
+ goto out;
+ }
+
+ ether_addr_copy(req->packet.dmac, mac);
+ eth_broadcast_addr((u8 *)&req->mask.dmac);
+ req->features = BIT_ULL(NPC_DMAC);
+ req->channel = pf->hw.rx_chan_base;
+ req->intf = NIX_INTF_RX;
+ req->default_rule = 1;
+ req->append = 1;
+ req->vf = vf + 1;
+ req->op = NIX_RX_ACTION_DEFAULT;
+
+ err = otx2_sync_mbox_msg(&pf->mbox);
+out:
+ mutex_unlock(&pf->mbox.lock);
+ return err;
+}
+
+static int otx2_set_vf_mac(struct net_device *netdev, int vf, u8 *mac)
+{
+ struct otx2_nic *pf = netdev_priv(netdev);
+ struct pci_dev *pdev = pf->pdev;
+ struct otx2_vf_config *config;
+ int ret;
+
+ if (!netif_running(netdev))
+ return -EAGAIN;
+
+ if (vf >= pci_num_vf(pdev))
+ return -EINVAL;
+
+ if (!is_valid_ether_addr(mac))
+ return -EINVAL;
+
+ config = &pf->vf_configs[vf];
+ ether_addr_copy(config->mac, mac);
+
+ ret = otx2_do_set_vf_mac(pf, vf, mac);
+ if (ret == 0)
+ dev_info(&pdev->dev, "Reload VF driver to apply the changes\n");
+
+ return ret;
+}
+
+static int otx2_do_set_vf_vlan(struct otx2_nic *pf, int vf, u16 vlan, u8 qos,
+ __be16 proto)
+{
+ struct otx2_flow_config *flow_cfg = pf->flow_cfg;
+ struct nix_vtag_config_rsp *vtag_rsp;
+ struct npc_delete_flow_req *del_req;
+ struct nix_vtag_config *vtag_req;
+ struct npc_install_flow_req *req;
+ struct otx2_vf_config *config;
+ int err = 0;
+ u32 idx;
+
+ config = &pf->vf_configs[vf];
+
+ if (!vlan && !config->vlan)
+ goto out;
+
+ mutex_lock(&pf->mbox.lock);
+
+ /* free old tx vtag entry */
+ if (config->vlan) {
+ vtag_req = otx2_mbox_alloc_msg_nix_vtag_cfg(&pf->mbox);
+ if (!vtag_req) {
+ err = -ENOMEM;
+ goto out;
+ }
+ vtag_req->cfg_type = 0;
+ vtag_req->tx.free_vtag0 = 1;
+ vtag_req->tx.vtag0_idx = config->tx_vtag_idx;
+
+ err = otx2_sync_mbox_msg(&pf->mbox);
+ if (err)
+ goto out;
+ }
+
+ if (!vlan && config->vlan) {
+ /* rx */
+ del_req = otx2_mbox_alloc_msg_npc_delete_flow(&pf->mbox);
+ if (!del_req) {
+ err = -ENOMEM;
+ goto out;
+ }
+ idx = ((vf * OTX2_PER_VF_VLAN_FLOWS) + OTX2_VF_VLAN_RX_INDEX);
+ del_req->entry =
+ flow_cfg->entry[flow_cfg->vf_vlan_offset + idx];
+ err = otx2_sync_mbox_msg(&pf->mbox);
+ if (err)
+ goto out;
+
+ /* tx */
+ del_req = otx2_mbox_alloc_msg_npc_delete_flow(&pf->mbox);
+ if (!del_req) {
+ err = -ENOMEM;
+ goto out;
+ }
+ idx = ((vf * OTX2_PER_VF_VLAN_FLOWS) + OTX2_VF_VLAN_TX_INDEX);
+ del_req->entry =
+ flow_cfg->entry[flow_cfg->vf_vlan_offset + idx];
+ err = otx2_sync_mbox_msg(&pf->mbox);
+
+ goto out;
+ }
+
+ /* rx */
+ req = otx2_mbox_alloc_msg_npc_install_flow(&pf->mbox);
+ if (!req) {
+ err = -ENOMEM;
+ goto out;
+ }
+
+ idx = ((vf * OTX2_PER_VF_VLAN_FLOWS) + OTX2_VF_VLAN_RX_INDEX);
+ req->entry = flow_cfg->entry[flow_cfg->vf_vlan_offset + idx];
+ req->packet.vlan_tci = htons(vlan);
+ req->mask.vlan_tci = htons(VLAN_VID_MASK);
+ /* af fills the destination mac addr */
+ eth_broadcast_addr((u8 *)&req->mask.dmac);
+ req->features = BIT_ULL(NPC_OUTER_VID) | BIT_ULL(NPC_DMAC);
+ req->channel = pf->hw.rx_chan_base;
+ req->intf = NIX_INTF_RX;
+ req->vf = vf + 1;
+ req->op = NIX_RX_ACTION_DEFAULT;
+ req->vtag0_valid = true;
+ req->vtag0_type = NIX_AF_LFX_RX_VTAG_TYPE7;
+ req->set_cntr = 1;
+
+ err = otx2_sync_mbox_msg(&pf->mbox);
+ if (err)
+ goto out;
+
+ /* tx */
+ vtag_req = otx2_mbox_alloc_msg_nix_vtag_cfg(&pf->mbox);
+ if (!vtag_req) {
+ err = -ENOMEM;
+ goto out;
+ }
+
+ /* configure tx vtag params */
+ vtag_req->vtag_size = VTAGSIZE_T4;
+ vtag_req->cfg_type = 0; /* tx vlan cfg */
+ vtag_req->tx.cfg_vtag0 = 1;
+ vtag_req->tx.vtag0 = ((u64)ntohs(proto) << 16) | vlan;
+
+ err = otx2_sync_mbox_msg(&pf->mbox);
+ if (err)
+ goto out;
+
+ vtag_rsp = (struct nix_vtag_config_rsp *)otx2_mbox_get_rsp
+ (&pf->mbox.mbox, 0, &vtag_req->hdr);
+ if (IS_ERR(vtag_rsp)) {
+ err = PTR_ERR(vtag_rsp);
+ goto out;
+ }
+ config->tx_vtag_idx = vtag_rsp->vtag0_idx;
+
+ req = otx2_mbox_alloc_msg_npc_install_flow(&pf->mbox);
+ if (!req) {
+ err = -ENOMEM;
+ goto out;
+ }
+
+ eth_zero_addr((u8 *)&req->mask.dmac);
+ idx = ((vf * OTX2_PER_VF_VLAN_FLOWS) + OTX2_VF_VLAN_TX_INDEX);
+ req->entry = flow_cfg->entry[flow_cfg->vf_vlan_offset + idx];
+ req->features = BIT_ULL(NPC_DMAC);
+ req->channel = pf->hw.tx_chan_base;
+ req->intf = NIX_INTF_TX;
+ req->vf = vf + 1;
+ req->op = NIX_TX_ACTIONOP_UCAST_DEFAULT;
+ req->vtag0_def = vtag_rsp->vtag0_idx;
+ req->vtag0_op = VTAG_INSERT;
+ req->set_cntr = 1;
+
+ err = otx2_sync_mbox_msg(&pf->mbox);
+out:
+ config->vlan = vlan;
+ mutex_unlock(&pf->mbox.lock);
+ return err;
+}
+
+static int otx2_set_vf_vlan(struct net_device *netdev, int vf, u16 vlan, u8 qos,
+ __be16 proto)
+{
+ struct otx2_nic *pf = netdev_priv(netdev);
+ struct pci_dev *pdev = pf->pdev;
+
+ if (!netif_running(netdev))
+ return -EAGAIN;
+
+ if (vf >= pci_num_vf(pdev))
+ return -EINVAL;
+
+ /* qos is currently unsupported */
+ if (vlan >= VLAN_N_VID || qos)
+ return -EINVAL;
+
+ if (proto != htons(ETH_P_8021Q))
+ return -EPROTONOSUPPORT;
+
+ if (!(pf->flags & OTX2_FLAG_VF_VLAN_SUPPORT))
+ return -EOPNOTSUPP;
+
+ return otx2_do_set_vf_vlan(pf, vf, vlan, qos, proto);
+}
+
+static int otx2_get_vf_config(struct net_device *netdev, int vf,
+ struct ifla_vf_info *ivi)
+{
+ struct otx2_nic *pf = netdev_priv(netdev);
+ struct pci_dev *pdev = pf->pdev;
+ struct otx2_vf_config *config;
+
+ if (!netif_running(netdev))
+ return -EAGAIN;
+
+ if (vf >= pci_num_vf(pdev))
+ return -EINVAL;
+
+ config = &pf->vf_configs[vf];
+ ivi->vf = vf;
+ ether_addr_copy(ivi->mac, config->mac);
+ ivi->vlan = config->vlan;
+
+ return 0;
+}
+
static const struct net_device_ops otx2_netdev_ops = {
.ndo_open = otx2_open,
.ndo_stop = otx2_stop,
.ndo_tx_timeout = otx2_tx_timeout,
.ndo_get_stats64 = otx2_get_stats64,
.ndo_do_ioctl = otx2_ioctl,
+ .ndo_set_vf_mac = otx2_set_vf_mac,
+ .ndo_set_vf_vlan = otx2_set_vf_vlan,
+ .ndo_get_vf_config = otx2_get_vf_config,
};
static int otx2_wq_init(struct otx2_nic *pf)
netdev->hw_features |= NETIF_F_LOOPBACK | NETIF_F_RXALL;
+ err = otx2_mcam_flow_init(pf);
+ if (err)
+ goto err_ptp_destroy;
+
+ if (pf->flags & OTX2_FLAG_NTUPLE_SUPPORT)
+ netdev->hw_features |= NETIF_F_NTUPLE;
+
+ if (pf->flags & OTX2_FLAG_UCAST_FLTR_SUPPORT)
+ netdev->priv_flags |= IFF_UNICAST_FLT;
+
+ /* Support TSO on tag interface */
+ netdev->vlan_features |= netdev->features;
+ netdev->hw_features |= NETIF_F_HW_VLAN_CTAG_TX |
+ NETIF_F_HW_VLAN_STAG_TX;
+ if (pf->flags & OTX2_FLAG_RX_VLAN_SUPPORT)
+ netdev->hw_features |= NETIF_F_HW_VLAN_CTAG_RX |
+ NETIF_F_HW_VLAN_STAG_RX;
+ netdev->features |= netdev->hw_features;
+
netdev->gso_max_segs = OTX2_MAX_GSO_SEGS;
netdev->watchdog_timeo = OTX2_TX_TIMEOUT;
err = register_netdev(netdev);
if (err) {
dev_err(dev, "Failed to register netdevice\n");
- goto err_ptp_destroy;
+ goto err_del_mcam_entries;
}
err = otx2_wq_init(pf);
err_unreg_netdev:
unregister_netdev(netdev);
+err_del_mcam_entries:
+ otx2_mcam_flow_del(pf);
err_ptp_destroy:
otx2_ptp_destroy(pf);
err_detach_rsrc:
pf = netdev_priv(netdev);
+ pf->flags |= OTX2_FLAG_PF_SHUTDOWN;
+
if (pf->flags & OTX2_FLAG_TX_TSTAMP_ENABLED)
otx2_config_hw_tx_tstamp(pf, false);
if (pf->flags & OTX2_FLAG_RX_TSTAMP_ENABLED)
destroy_workqueue(pf->otx2_wq);
otx2_ptp_destroy(pf);
+ otx2_mcam_flow_del(pf);
otx2_detach_resources(&pf->mbox);
otx2_disable_mbox_intr(pf);
otx2_pfaf_mbox_destroy(pf);
ext->tstmp = 1;
}
+#define OTX2_VLAN_PTR_OFFSET (ETH_HLEN - ETH_TLEN)
+ if (skb_vlan_tag_present(skb)) {
+ if (skb->vlan_proto == htons(ETH_P_8021Q)) {
+ ext->vlan1_ins_ena = 1;
+ ext->vlan1_ins_ptr = OTX2_VLAN_PTR_OFFSET;
+ ext->vlan1_ins_tci = skb_vlan_tag_get(skb);
+ } else if (skb->vlan_proto == htons(ETH_P_8021AD)) {
+ ext->vlan0_ins_ena = 1;
+ ext->vlan0_ins_ptr = OTX2_VLAN_PTR_OFFSET;
+ ext->vlan0_ins_tci = skb_vlan_tag_get(skb);
+ }
+ }
+
*offset += sizeof(*ext);
}
}
if (skb_shinfo(skb)->gso_size && !is_hw_tso_supported(pfvf, skb)) {
+ /* Insert vlan tag before giving pkt to tso */
+ if (skb_vlan_tag_present(skb))
+ skb = __vlan_hwaccel_push_inside(skb);
otx2_sq_append_tso(pfvf, sq, skb, qidx);
return true;
}
NETIF_F_SG | NETIF_F_TSO | NETIF_F_TSO6 |
NETIF_F_GSO_UDP_L4;
netdev->features = netdev->hw_features;
+ /* Support TSO on tag interface */
+ netdev->vlan_features |= netdev->features;
+ netdev->hw_features |= NETIF_F_HW_VLAN_CTAG_TX |
+ NETIF_F_HW_VLAN_STAG_TX;
+ netdev->features |= netdev->hw_features;
netdev->gso_max_segs = OTX2_MAX_GSO_SEGS;
netdev->watchdog_timeo = OTX2_TX_TIMEOUT;
if (err)
return err;
- if (dma_set_mask_and_coherent(&pdev->dev, DMA_BIT_MASK(30))) {
+ err = dma_set_mask_and_coherent(&pdev->dev, DMA_BIT_MASK(30));
+ if (err) {
dev_err(&pdev->dev, "fail to set DMA mask\n");
goto err_dma_mask;
}
dev_info(fw->dev.dev, "Prestera FW is ready\n");
fw->wq = alloc_workqueue("prestera_fw_wq", WQ_HIGHPRI, 1);
- if (!fw->wq)
+ if (!fw->wq) {
+ err = -ENOMEM;
goto err_wq_alloc;
+ }
INIT_WORK(&fw->evt_work, prestera_fw_evt_work_fn);
mtk_star_adjust_link, 0, priv->phy_intf);
if (!priv->phydev) {
netdev_err(ndev, "failed to connect to PHY\n");
+ ret = -ENODEV;
goto err_free_irq;
}
err_drop_packet:
dev_kfree_skb(skb);
ndev->stats.tx_dropped++;
- return NETDEV_TX_BUSY;
+ return NETDEV_TX_OK;
}
/* Returns the number of bytes sent or a negative number on the first
if (mlx4_en_DUMP_ETH_STATS(mdev, priv->port, 1))
en_dbg(HW, priv, "Failed dumping statistics\n");
- memset(&priv->pstats, 0, sizeof(priv->pstats));
memset(&priv->pkstats, 0, sizeof(priv->pkstats));
memset(&priv->port_stats, 0, sizeof(priv->port_stats));
memset(&priv->rx_flowstats, 0, sizeof(priv->rx_flowstats));
wmb(); /* ensure HW sees CQ consumer before we post new buffers */
ring->cons = cq->mcq.cons_index;
}
- AVG_PERF_COUNTER(priv->pstats.rx_coal_avg, polled);
mlx4_en_refill_rx_buffers(priv, ring);
/* in case we got here because of !clean_complete */
done = budget;
- INC_PERF_COUNTER(priv->pstats.napi_quota);
-
cpu_curr = smp_processor_id();
idata = irq_desc_get_irq_data(cq->irq_desc);
aff = irq_data_get_affinity_mask(idata);
if (unlikely(!priv->port_up))
goto tx_drop;
- /* fetch ring->cons far ahead before needing it to avoid stall */
- ring_cons = READ_ONCE(ring->cons);
-
real_size = get_real_size(skb, shinfo, dev, &lso_header_size,
&inline_ok, &fragptr);
if (unlikely(!real_size))
netdev_txq_bql_enqueue_prefetchw(ring->tx_queue);
- /* Track current inflight packets for performance analysis */
- AVG_PERF_COUNTER(priv->pstats.inflight_avg,
- (u32)(ring->prod - ring_cons - 1));
-
/* Packet is good - grab an index and transmit it */
index = ring->prod & ring->size_mask;
bf_index = ring->prod;
ring->packets++;
}
ring->bytes += tx_info->nr_bytes;
- AVG_PERF_COUNTER(priv->pstats.tx_pktsz_avg, skb->len);
if (tx_info->inl)
build_inline_wqe(tx_desc, skb, shinfo, fragptr);
index = ring->prod & ring->size_mask;
tx_info = &ring->tx_info[index];
- /* Track current inflight packets for performance analysis */
- AVG_PERF_COUNTER(priv->pstats.inflight_avg,
- (u32)(ring->prod - READ_ONCE(ring->cons) - 1));
-
tx_desc = ring->buf + (index << LOG_TXBB_SIZE);
data = &tx_desc->data;
cpu_to_be32(MLX4_EN_BIT_DESC_OWN) : 0);
rx_ring->xdp_tx++;
- AVG_PERF_COUNTER(priv->pstats.tx_pktsz_avg, length);
ring->prod += MLX4_EN_XDP_TX_NRTXBB;
#define INIT_HCA_LOG_RD_OFFSET (INIT_HCA_QPC_OFFSET + 0x77)
#define INIT_HCA_MCAST_OFFSET 0x0c0
#define INIT_HCA_MC_BASE_OFFSET (INIT_HCA_MCAST_OFFSET + 0x00)
-#define INIT_HCA_LOG_MC_ENTRY_SZ_OFFSET (INIT_HCA_MCAST_OFFSET + 0x12)
-#define INIT_HCA_LOG_MC_HASH_SZ_OFFSET (INIT_HCA_MCAST_OFFSET + 0x16)
+#define INIT_HCA_LOG_MC_ENTRY_SZ_OFFSET (INIT_HCA_MCAST_OFFSET + 0x13)
+#define INIT_HCA_LOG_MC_HASH_SZ_OFFSET (INIT_HCA_MCAST_OFFSET + 0x17)
#define INIT_HCA_UC_STEERING_OFFSET (INIT_HCA_MCAST_OFFSET + 0x18)
#define INIT_HCA_LOG_MC_TABLE_SZ_OFFSET (INIT_HCA_MCAST_OFFSET + 0x1b)
#define INIT_HCA_DEVICE_MANAGED_FLOW_STEERING_EN 0x6
#define INIT_HCA_DRIVER_VERSION_SZ 0x40
#define INIT_HCA_FS_PARAM_OFFSET 0x1d0
#define INIT_HCA_FS_BASE_OFFSET (INIT_HCA_FS_PARAM_OFFSET + 0x00)
-#define INIT_HCA_FS_LOG_ENTRY_SZ_OFFSET (INIT_HCA_FS_PARAM_OFFSET + 0x12)
+#define INIT_HCA_FS_LOG_ENTRY_SZ_OFFSET (INIT_HCA_FS_PARAM_OFFSET + 0x13)
#define INIT_HCA_FS_A0_OFFSET (INIT_HCA_FS_PARAM_OFFSET + 0x18)
#define INIT_HCA_FS_LOG_TABLE_SZ_OFFSET (INIT_HCA_FS_PARAM_OFFSET + 0x1b)
#define INIT_HCA_FS_ETH_BITS_OFFSET (INIT_HCA_FS_PARAM_OFFSET + 0x21)
u64 cmpt_base;
u64 mtt_base;
u64 global_caps;
- u16 log_mc_entry_sz;
- u16 log_mc_hash_sz;
+ u8 log_mc_entry_sz;
+ u8 log_mc_hash_sz;
u16 hca_core_clock; /* Internal Clock Frequency (in MHz) */
u8 log_num_qps;
u8 log_num_srqs;
#include <linux/bitops.h>
#include <linux/compiler.h>
+#include <linux/ethtool.h>
#include <linux/list.h>
#include <linux/mutex.h>
#include <linux/netdevice.h>
#define MLX4_EN_LOOPBACK_RETRIES 5
#define MLX4_EN_LOOPBACK_TIMEOUT 100
-#ifdef MLX4_EN_PERF_STAT
-/* Number of samples to 'average' */
-#define AVG_SIZE 128
-#define AVG_FACTOR 1024
-
-#define INC_PERF_COUNTER(cnt) (++(cnt))
-#define ADD_PERF_COUNTER(cnt, add) ((cnt) += (add))
-#define AVG_PERF_COUNTER(cnt, sample) \
- ((cnt) = ((cnt) * (AVG_SIZE - 1) + (sample) * AVG_FACTOR) / AVG_SIZE)
-#define GET_PERF_COUNTER(cnt) (cnt)
-#define GET_AVG_PERF_COUNTER(cnt) ((cnt) / AVG_FACTOR)
-
-#else
-
-#define INC_PERF_COUNTER(cnt) do {} while (0)
-#define ADD_PERF_COUNTER(cnt, add) do {} while (0)
-#define AVG_PERF_COUNTER(cnt, sample) do {} while (0)
-#define GET_PERF_COUNTER(cnt) (0)
-#define GET_AVG_PERF_COUNTER(cnt) (0)
-#endif /* MLX4_EN_PERF_STAT */
-
/* Constants for TX flow */
enum {
MAX_INLINE = 104, /* 128 - 16 - 4 - 4 */
struct work_struct linkstate_task;
struct delayed_work stats_task;
struct delayed_work service_task;
- struct mlx4_en_perf_stats pstats;
struct mlx4_en_pkt_stats pkstats;
struct mlx4_en_counter_stats pf_stats;
struct mlx4_en_flow_stats_rx rx_priority_flowstats[MLX4_NUM_PRIORITIES];
#ifndef _MLX4_STATS_
#define _MLX4_STATS_
-#ifdef MLX4_EN_PERF_STAT
-#define NUM_PERF_STATS NUM_PERF_COUNTERS
-#else
-#define NUM_PERF_STATS 0
-#endif
-
#define NUM_PRIORITIES 9
#define NUM_PRIORITY_STATS 2
#define NUM_PORT_STATS 10
};
-struct mlx4_en_perf_stats {
- u32 tx_poll;
- u64 tx_pktsz_avg;
- u32 inflight_avg;
- u16 tx_coal_avg;
- u16 rx_coal_avg;
- u32 napi_quota;
-#define NUM_PERF_COUNTERS 6
-};
-
struct mlx4_en_xdp_stats {
unsigned long rx_xdp_drop;
unsigned long rx_xdp_tx;
};
#define NUM_ALL_STATS (NUM_MAIN_STATS + NUM_PORT_STATS + NUM_PKT_STATS + \
- NUM_FLOW_STATS + NUM_PERF_STATS + NUM_PF_STATS + \
+ NUM_FLOW_STATS + NUM_PF_STATS + \
NUM_XDP_STATS + NUM_PHY_STATS)
#define MLX4_FIND_NETDEV_STAT(n) (offsetof(struct net_device_stats, n) / \
kvfree(cmd->stats);
return err;
}
-EXPORT_SYMBOL(mlx5_cmd_init);
void mlx5_cmd_cleanup(struct mlx5_core_dev *dev)
{
dma_pool_destroy(cmd->pool);
kvfree(cmd->stats);
}
-EXPORT_SYMBOL(mlx5_cmd_cleanup);
void mlx5_cmd_set_state(struct mlx5_core_dev *dev,
enum mlx5_cmdif_state cmdif_state)
{
dev->cmd.state = cmdif_state;
}
-EXPORT_SYMBOL(mlx5_cmd_set_state);
struct netlink_ext_ack *extack)
{
struct mlx5_core_dev *dev = devlink_priv(devlink);
- const struct firmware *fw;
- int err;
-
- err = request_firmware_direct(&fw, params->file_name, &dev->pdev->dev);
- if (err)
- return err;
-
- err = mlx5_firmware_flash(dev, fw, extack);
- release_firmware(fw);
- return err;
+ return mlx5_firmware_flash(dev, params->fw, extack);
}
static u8 mlx5_fw_ver_major(u32 version)
case MLX5_FLOW_DESTINATION_TYPE_TIR:
trace_seq_printf(p, "tir=%u\n", dst->tir_num);
break;
+ case MLX5_FLOW_DESTINATION_TYPE_FLOW_SAMPLER:
+ trace_seq_printf(p, "sampler_id=%u\n", dst->sampler_id);
+ break;
case MLX5_FLOW_DESTINATION_TYPE_COUNTER:
trace_seq_printf(p, "counter_id=%u\n", counter_id);
break;
return (ioread32be(&dev->iseg->initializing) >> MLX5_ECPU_BIT_NUM) & 1;
}
-static int mlx5_peer_pf_init(struct mlx5_core_dev *dev)
+static bool mlx5_ecpf_esw_admins_host_pf(const struct mlx5_core_dev *dev)
{
- u32 in[MLX5_ST_SZ_DW(enable_hca_in)] = {};
- int err;
+ /* In separate host mode, PF enables itself.
+ * When ECPF is eswitch manager, eswitch enables host PF after
+ * eswitch is setup.
+ */
+ return mlx5_core_is_ecpf_esw_manager(dev);
+}
+
+int mlx5_cmd_host_pf_enable_hca(struct mlx5_core_dev *dev)
+{
+ u32 out[MLX5_ST_SZ_DW(enable_hca_out)] = {};
+ u32 in[MLX5_ST_SZ_DW(enable_hca_in)] = {};
MLX5_SET(enable_hca_in, in, opcode, MLX5_CMD_OP_ENABLE_HCA);
- err = mlx5_cmd_exec_in(dev, enable_hca, in);
+ MLX5_SET(enable_hca_in, in, function_id, 0);
+ MLX5_SET(enable_hca_in, in, embedded_cpu_function, 0);
+ return mlx5_cmd_exec(dev, &in, sizeof(in), &out, sizeof(out));
+}
+
+int mlx5_cmd_host_pf_disable_hca(struct mlx5_core_dev *dev)
+{
+ u32 out[MLX5_ST_SZ_DW(disable_hca_out)] = {};
+ u32 in[MLX5_ST_SZ_DW(disable_hca_in)] = {};
+
+ MLX5_SET(disable_hca_in, in, opcode, MLX5_CMD_OP_DISABLE_HCA);
+ MLX5_SET(disable_hca_in, in, function_id, 0);
+ MLX5_SET(disable_hca_in, in, embedded_cpu_function, 0);
+ return mlx5_cmd_exec(dev, in, sizeof(in), out, sizeof(out));
+}
+
+static int mlx5_host_pf_init(struct mlx5_core_dev *dev)
+{
+ int err;
+
+ if (mlx5_ecpf_esw_admins_host_pf(dev))
+ return 0;
+
+ /* ECPF shall enable HCA for host PF in the same way a PF
+ * does this for its VFs when ECPF is not a eswitch manager.
+ */
+ err = mlx5_cmd_host_pf_enable_hca(dev);
if (err)
- mlx5_core_err(dev, "Failed to enable peer PF HCA err(%d)\n",
- err);
+ mlx5_core_err(dev, "Failed to enable external host PF HCA err(%d)\n", err);
return err;
}
-static void mlx5_peer_pf_cleanup(struct mlx5_core_dev *dev)
+static void mlx5_host_pf_cleanup(struct mlx5_core_dev *dev)
{
- u32 in[MLX5_ST_SZ_DW(disable_hca_in)] = {};
int err;
- MLX5_SET(disable_hca_in, in, opcode, MLX5_CMD_OP_DISABLE_HCA);
- err = mlx5_cmd_exec_in(dev, disable_hca, in);
+ if (mlx5_ecpf_esw_admins_host_pf(dev))
+ return;
+
+ err = mlx5_cmd_host_pf_disable_hca(dev);
if (err) {
- mlx5_core_err(dev, "Failed to disable peer PF HCA err(%d)\n",
- err);
+ mlx5_core_err(dev, "Failed to disable external host PF HCA err(%d)\n", err);
return;
}
-
- err = mlx5_wait_for_pages(dev, &dev->priv.peer_pf_pages);
- if (err)
- mlx5_core_warn(dev, "Timeout reclaiming peer PF pages err(%d)\n",
- err);
}
int mlx5_ec_init(struct mlx5_core_dev *dev)
if (!mlx5_core_is_ecpf(dev))
return 0;
- /* ECPF shall enable HCA for peer PF in the same way a PF
- * does this for its VFs.
- */
- return mlx5_peer_pf_init(dev);
+ return mlx5_host_pf_init(dev);
}
void mlx5_ec_cleanup(struct mlx5_core_dev *dev)
{
+ int err;
+
if (!mlx5_core_is_ecpf(dev))
return;
- mlx5_peer_pf_cleanup(dev);
+ mlx5_host_pf_cleanup(dev);
+
+ err = mlx5_wait_for_pages(dev, &dev->priv.host_pf_pages);
+ if (err)
+ mlx5_core_warn(dev, "Timeout reclaiming external host PF pages err(%d)\n", err);
}
int mlx5_ec_init(struct mlx5_core_dev *dev);
void mlx5_ec_cleanup(struct mlx5_core_dev *dev);
+int mlx5_cmd_host_pf_enable_hca(struct mlx5_core_dev *dev);
+int mlx5_cmd_host_pf_disable_hca(struct mlx5_core_dev *dev);
+
#else /* CONFIG_MLX5_ESWITCH */
static inline bool
struct mlx5e_priv *priv;
/* A given netdev is not a representor or not a slave of LAG configuration */
- if (!mlx5e_eswitch_rep(netdev) || !bond_slave_get_rtnl(netdev))
+ if (!mlx5e_eswitch_rep(netdev) || !netif_is_lag_port(netdev))
return false;
priv = netdev_priv(netdev);
outer_headers.dst_ipv4_dst_ipv6.ipv4_layout.ipv4);
}
+#if IS_ENABLED(CONFIG_IPV6)
static void accel_fs_tcp_set_ipv6_flow(struct mlx5_flow_spec *spec, struct sock *sk)
{
MLX5_SET_TO_ONES(fte_match_param, spec->match_criteria, outer_headers.ip_protocol);
outer_headers.dst_ipv4_dst_ipv6.ipv6_layout.ipv6),
0xff, 16);
}
+#endif
void mlx5e_accel_fs_del_sk(struct mlx5_flow_handle *rule)
{
if (!spec)
return -ENOMEM;
- /* Action to copy 7 bit ipsec_syndrome to regB[0:6] */
+ /* Action to copy 7 bit ipsec_syndrome to regB[24:30] */
MLX5_SET(copy_action_in, action, action_type, MLX5_ACTION_TYPE_COPY);
MLX5_SET(copy_action_in, action, src_field, MLX5_ACTION_IN_FIELD_IPSEC_SYNDROME);
MLX5_SET(copy_action_in, action, src_offset, 0);
MLX5_SET(copy_action_in, action, length, 7);
MLX5_SET(copy_action_in, action, dst_field, MLX5_ACTION_IN_FIELD_METADATA_REG_B);
- MLX5_SET(copy_action_in, action, dst_offset, 0);
+ MLX5_SET(copy_action_in, action, dst_offset, 24);
modify_hdr = mlx5_modify_header_alloc(mdev, MLX5_FLOW_NAMESPACE_KERNEL,
1, action);
setup_fte_common(attrs, ipsec_obj_id, spec, &flow_act);
- /* Set 1 bit ipsec marker */
- /* Set 24 bit ipsec_obj_id */
+ /* Set bit[31] ipsec marker */
+ /* Set bit[23-0] ipsec_obj_id */
MLX5_SET(set_action_in, action, action_type, MLX5_ACTION_TYPE_SET);
MLX5_SET(set_action_in, action, field, MLX5_ACTION_IN_FIELD_METADATA_REG_B);
- MLX5_SET(set_action_in, action, data, (ipsec_obj_id << 1) | 0x1);
- MLX5_SET(set_action_in, action, offset, 7);
- MLX5_SET(set_action_in, action, length, 25);
+ MLX5_SET(set_action_in, action, data, (ipsec_obj_id | BIT(31)));
+ MLX5_SET(set_action_in, action, offset, 0);
+ MLX5_SET(set_action_in, action, length, 32);
modify_hdr = mlx5_modify_header_alloc(priv->mdev, MLX5_FLOW_NAMESPACE_KERNEL,
1, action);
struct mlx5_cqe64 *cqe)
{
u32 ipsec_meta_data = be32_to_cpu(cqe->ft_metadata);
- u8 ipsec_syndrome = ipsec_meta_data & 0xFF;
struct mlx5e_priv *priv;
struct xfrm_offload *xo;
struct xfrm_state *xs;
xo = xfrm_offload(skb);
xo->flags = CRYPTO_DONE;
- switch (ipsec_syndrome & MLX5_IPSEC_METADATA_SYNDROM_MASK) {
+ switch (MLX5_IPSEC_METADATA_SYNDROM(ipsec_meta_data)) {
case MLX5E_IPSEC_OFFLOAD_RX_SYNDROME_DECRYPTED:
xo->status = CRYPTO_SUCCESS;
if (WARN_ON_ONCE(priv->ipsec->no_trailer))
#include "en.h"
#include "en/txrx.h"
-#define MLX5_IPSEC_METADATA_MARKER_MASK (0x80)
-#define MLX5_IPSEC_METADATA_SYNDROM_MASK (0x7F)
-#define MLX5_IPSEC_METADATA_HANDLE(metadata) (((metadata) >> 8) & 0xFF)
+/* Bit31: IPsec marker, Bit30-24: IPsec syndrome, Bit23-0: IPsec obj id */
+#define MLX5_IPSEC_METADATA_MARKER(metadata) (((metadata) >> 31) & 0x1)
+#define MLX5_IPSEC_METADATA_SYNDROM(metadata) (((metadata) >> 24) & GENMASK(6, 0))
+#define MLX5_IPSEC_METADATA_HANDLE(metadata) ((metadata) & GENMASK(23, 0))
struct mlx5e_accel_tx_ipsec_state {
struct xfrm_offload *xo;
static inline bool mlx5_ipsec_is_rx_flow(struct mlx5_cqe64 *cqe)
{
- return !!(MLX5_IPSEC_METADATA_MARKER_MASK & be32_to_cpu(cqe->ft_metadata));
+ return MLX5_IPSEC_METADATA_MARKER(be32_to_cpu(cqe->ft_metadata));
}
static inline bool mlx5e_ipsec_is_tx_flow(struct mlx5e_accel_tx_ipsec_state *ipsec_st)
depth += sizeof(struct tcphdr);
- if (unlikely(!sk || sk->sk_state == TCP_TIME_WAIT))
+ if (unlikely(!sk))
return;
- if (unlikely(!resync_queue_get_psv(sk)))
- return;
+ if (unlikely(sk->sk_state == TCP_TIME_WAIT))
+ goto unref;
- skb->sk = sk;
- skb->destructor = sock_edemux;
+ if (unlikely(!resync_queue_get_psv(sk)))
+ goto unref;
seq = th->seq;
datalen = skb->len - depth;
tls_offload_rx_resync_async_request_start(sk, seq, datalen);
rq->stats->tls_resync_req_start++;
+
+unref:
+ sock_gen_put(sk);
}
void mlx5e_ktls_rx_resync(struct net_device *netdev, struct sock *sk,
tc->ct = mlx5_tc_ct_init(priv, tc->chains, &priv->fs.tc.mod_hdr,
MLX5_FLOW_NAMESPACE_KERNEL);
- if (IS_ERR(tc->ct))
+ if (IS_ERR(tc->ct)) {
+ err = PTR_ERR(tc->ct);
goto err_ct;
+ }
tc->netdevice_nb.notifier_call = mlx5e_tc_netdev_event;
err = register_netdevice_notifier_dev_net(priv->netdev,
reg_b = be32_to_cpu(cqe->ft_metadata);
+ if (reg_b >> (MLX5E_TC_TABLE_CHAIN_TAG_BITS + ZONE_RESTORE_BITS))
+ return false;
+
chain = reg_b & MLX5E_TC_TABLE_CHAIN_TAG_MASK;
if (chain)
return true;
memcpy(&vhdr->h_vlan_encapsulated_proto, skb->data + cpy1_sz, cpy2_sz);
}
-/* RM 2311217: no L4 inner checksum for IPsec tunnel type packet */
+/* If packet is not IP's CHECKSUM_PARTIAL (e.g. icmd packet),
+ * need to set L3 checksum flag for IPsec
+ */
static void
ipsec_txwqe_build_eseg_csum(struct mlx5e_txqsq *sq, struct sk_buff *skb,
struct mlx5_wqe_eth_seg *eseg)
eseg->cs_flags |= MLX5_ETH_WQE_L3_INNER_CSUM;
sq->stats->csum_partial_inner++;
} else {
- eseg->cs_flags |= MLX5_ETH_WQE_L4_CSUM;
sq->stats->csum_partial++;
}
}
static inline void
-mlx5e_txwqe_build_eseg_csum(struct mlx5e_txqsq *sq, struct sk_buff *skb, struct mlx5_wqe_eth_seg *eseg)
+mlx5e_txwqe_build_eseg_csum(struct mlx5e_txqsq *sq, struct sk_buff *skb,
+ struct mlx5e_accel_tx_state *accel,
+ struct mlx5_wqe_eth_seg *eseg)
{
- if (unlikely(eseg->flow_table_metadata & cpu_to_be32(MLX5_ETH_WQE_FT_META_IPSEC))) {
- ipsec_txwqe_build_eseg_csum(sq, skb, eseg);
- return;
- }
-
if (likely(skb->ip_summed == CHECKSUM_PARTIAL)) {
eseg->cs_flags = MLX5_ETH_WQE_L3_CSUM;
if (skb->encapsulation) {
eseg->cs_flags |= MLX5_ETH_WQE_L4_CSUM;
sq->stats->csum_partial++;
}
+#ifdef CONFIG_MLX5_EN_TLS
+ } else if (unlikely(accel && accel->tls.tls_tisn)) {
+ eseg->cs_flags = MLX5_ETH_WQE_L3_CSUM | MLX5_ETH_WQE_L4_CSUM;
+ sq->stats->csum_partial++;
+#endif
+ } else if (unlikely(eseg->flow_table_metadata & cpu_to_be32(MLX5_ETH_WQE_FT_META_IPSEC))) {
+ ipsec_txwqe_build_eseg_csum(sq, skb, eseg);
+
} else
sq->stats->csum_none++;
}
}
static bool mlx5e_txwqe_build_eseg(struct mlx5e_priv *priv, struct mlx5e_txqsq *sq,
- struct sk_buff *skb, struct mlx5_wqe_eth_seg *eseg)
+ struct sk_buff *skb, struct mlx5e_accel_tx_state *accel,
+ struct mlx5_wqe_eth_seg *eseg)
{
if (unlikely(!mlx5e_accel_tx_eseg(priv, skb, eseg)))
return false;
- mlx5e_txwqe_build_eseg_csum(sq, skb, eseg);
+ mlx5e_txwqe_build_eseg_csum(sq, skb, accel, eseg);
return true;
}
if (mlx5e_tx_skb_supports_mpwqe(skb, &attr)) {
struct mlx5_wqe_eth_seg eseg = {};
- if (unlikely(!mlx5e_txwqe_build_eseg(priv, sq, skb, &eseg)))
+ if (unlikely(!mlx5e_txwqe_build_eseg(priv, sq, skb, &accel, &eseg)))
return NETDEV_TX_OK;
mlx5e_sq_xmit_mpwqe(sq, skb, &eseg, netdev_xmit_more());
/* May update the WQE, but may not post other WQEs. */
mlx5e_accel_tx_finish(sq, wqe, &accel,
(struct mlx5_wqe_inline_seg *)(wqe->data + wqe_attr.ds_cnt_inl));
- if (unlikely(!mlx5e_txwqe_build_eseg(priv, sq, skb, &wqe->eth)))
+ if (unlikely(!mlx5e_txwqe_build_eseg(priv, sq, skb, &accel, &wqe->eth)))
return NETDEV_TX_OK;
mlx5e_sq_xmit_wqe(sq, skb, &attr, &wqe_attr, wqe, pi, netdev_xmit_more());
mlx5e_sq_calc_wqe_attr(skb, &attr, &wqe_attr);
pi = mlx5e_txqsq_get_next_pi(sq, wqe_attr.num_wqebbs);
wqe = MLX5E_TX_FETCH_WQE(sq, pi);
- mlx5e_txwqe_build_eseg_csum(sq, skb, &wqe->eth);
+ mlx5e_txwqe_build_eseg_csum(sq, skb, NULL, &wqe->eth);
mlx5e_sq_xmit_wqe(sq, skb, &attr, &wqe_attr, wqe, pi, xmit_more);
}
mlx5i_txwqe_build_datagram(av, dqpn, dqkey, datagram);
- mlx5e_txwqe_build_eseg_csum(sq, skb, eseg);
+ mlx5e_txwqe_build_eseg_csum(sq, skb, NULL, eseg);
eseg->mss = attr.mss;
struct mlx5_flow_table *
esw_acl_table_create(struct mlx5_eswitch *esw, u16 vport_num, int ns, int size)
{
+ struct mlx5_flow_table_attr ft_attr = {};
struct mlx5_core_dev *dev = esw->dev;
struct mlx5_flow_namespace *root_ns;
struct mlx5_flow_table *acl;
return ERR_PTR(-EOPNOTSUPP);
}
- acl = mlx5_create_vport_flow_table(root_ns, 0, size, 0, vport_num);
+ ft_attr.max_fte = size;
+ ft_attr.flags = MLX5_FLOW_TABLE_OTHER_VPORT;
+ acl = mlx5_create_vport_flow_table(root_ns, &ft_attr, vport_num);
if (IS_ERR(acl)) {
err = PTR_ERR(acl);
esw_warn(dev, "vport[%d] create %s ACL table, err(%d)\n", vport_num,
struct mlx5_vport *vport;
vport = mlx5_eswitch_get_vport(esw, vport_num);
+
+ if (!vport->qos.enabled)
+ return -EOPNOTSUPP;
+
MLX5_SET(scheduling_context, ctx, max_average_bw, rate_mbps);
return mlx5_modify_scheduling_element_cmd(esw->dev,
int i;
mlx5_esw_for_each_vf_vport(esw, i, vport, esw->esw_funcs.num_vfs) {
+ memset(&vport->qos, 0, sizeof(vport->qos));
memset(&vport->info, 0, sizeof(vport->info));
vport->info.link_state = MLX5_VPORT_ADMIN_STATE_AUTO;
}
return err;
}
+static int host_pf_enable_hca(struct mlx5_core_dev *dev)
+{
+ if (!mlx5_core_is_ecpf(dev))
+ return 0;
+
+ /* Once vport and representor are ready, take out the external host PF
+ * out of initializing state. Enabling HCA clears the iser->initializing
+ * bit and host PF driver loading can progress.
+ */
+ return mlx5_cmd_host_pf_enable_hca(dev);
+}
+
+static void host_pf_disable_hca(struct mlx5_core_dev *dev)
+{
+ if (!mlx5_core_is_ecpf(dev))
+ return;
+
+ mlx5_cmd_host_pf_disable_hca(dev);
+}
+
/* mlx5_eswitch_enable_pf_vf_vports() enables vports of PF, ECPF and VFs
* whichever are present on the eswitch.
*/
if (ret)
return ret;
+ /* Enable external host PF HCA */
+ ret = host_pf_enable_hca(esw->dev);
+ if (ret)
+ goto pf_hca_err;
+
/* Enable ECPF vport */
if (mlx5_ecpf_vport_exists(esw->dev)) {
ret = mlx5_eswitch_load_vport(esw, MLX5_VPORT_ECPF, enabled_events);
vf_err:
if (mlx5_ecpf_vport_exists(esw->dev))
mlx5_eswitch_unload_vport(esw, MLX5_VPORT_ECPF);
-
ecpf_err:
+ host_pf_disable_hca(esw->dev);
+pf_hca_err:
mlx5_eswitch_unload_vport(esw, MLX5_VPORT_PF);
return ret;
}
if (mlx5_ecpf_vport_exists(esw->dev))
mlx5_eswitch_unload_vport(esw, MLX5_VPORT_ECPF);
+ host_pf_disable_hca(esw->dev);
mlx5_eswitch_unload_vport(esw, MLX5_VPORT_PF);
}
max_guarantee = evport->info.min_rate;
}
- return max_t(u32, max_guarantee / fw_max_bw_share, 1);
+ if (max_guarantee)
+ return max_t(u32, max_guarantee / fw_max_bw_share, 1);
+ return 0;
}
-static int normalize_vports_min_rate(struct mlx5_eswitch *esw, u32 divider)
+static int normalize_vports_min_rate(struct mlx5_eswitch *esw)
{
u32 fw_max_bw_share = MLX5_CAP_QOS(esw->dev, max_tsar_bw_share);
+ u32 divider = calculate_vports_min_rate_divider(esw);
struct mlx5_vport *evport;
u32 vport_max_rate;
u32 vport_min_rate;
continue;
vport_min_rate = evport->info.min_rate;
vport_max_rate = evport->info.max_rate;
- bw_share = MLX5_MIN_BW_SHARE;
+ bw_share = 0;
- if (vport_min_rate)
+ if (divider)
bw_share = MLX5_RATE_TO_BW_SHARE(vport_min_rate,
divider,
fw_max_bw_share);
struct mlx5_vport *evport = mlx5_eswitch_get_vport(esw, vport);
u32 fw_max_bw_share;
u32 previous_min_rate;
- u32 divider;
bool min_rate_supported;
bool max_rate_supported;
int err = 0;
previous_min_rate = evport->info.min_rate;
evport->info.min_rate = min_rate;
- divider = calculate_vports_min_rate_divider(esw);
- err = normalize_vports_min_rate(esw, divider);
+ err = normalize_vports_min_rate(esw);
if (err) {
evport->info.min_rate = previous_min_rate;
goto unlock;
MLX5_SET(set_flow_table_root_in, in, table_id, ft->id);
MLX5_SET(set_flow_table_root_in, in, underlay_qpn, underlay_qpn);
- if (ft->vport) {
- MLX5_SET(set_flow_table_root_in, in, vport_number, ft->vport);
- MLX5_SET(set_flow_table_root_in, in, other_vport, 1);
- }
+ MLX5_SET(set_flow_table_root_in, in, vport_number, ft->vport);
+ MLX5_SET(set_flow_table_root_in, in, other_vport,
+ !!(ft->flags & MLX5_FLOW_TABLE_OTHER_VPORT));
return mlx5_cmd_exec_in(dev, set_flow_table_root, in);
}
MLX5_SET(create_flow_table_in, in, table_type, ft->type);
MLX5_SET(create_flow_table_in, in, flow_table_context.level, ft->level);
MLX5_SET(create_flow_table_in, in, flow_table_context.log_size, log_size);
- if (ft->vport) {
- MLX5_SET(create_flow_table_in, in, vport_number, ft->vport);
- MLX5_SET(create_flow_table_in, in, other_vport, 1);
- }
+ MLX5_SET(create_flow_table_in, in, vport_number, ft->vport);
+ MLX5_SET(create_flow_table_in, in, other_vport,
+ !!(ft->flags & MLX5_FLOW_TABLE_OTHER_VPORT));
MLX5_SET(create_flow_table_in, in, flow_table_context.decap_en,
en_decap);
MLX5_CMD_OP_DESTROY_FLOW_TABLE);
MLX5_SET(destroy_flow_table_in, in, table_type, ft->type);
MLX5_SET(destroy_flow_table_in, in, table_id, ft->id);
- if (ft->vport) {
- MLX5_SET(destroy_flow_table_in, in, vport_number, ft->vport);
- MLX5_SET(destroy_flow_table_in, in, other_vport, 1);
- }
+ MLX5_SET(destroy_flow_table_in, in, vport_number, ft->vport);
+ MLX5_SET(destroy_flow_table_in, in, other_vport,
+ !!(ft->flags & MLX5_FLOW_TABLE_OTHER_VPORT));
return mlx5_cmd_exec_in(dev, destroy_flow_table, in);
}
flow_table_context.lag_master_next_table_id, 0);
}
} else {
- if (ft->vport) {
- MLX5_SET(modify_flow_table_in, in, vport_number,
- ft->vport);
- MLX5_SET(modify_flow_table_in, in, other_vport, 1);
- }
+ MLX5_SET(modify_flow_table_in, in, vport_number, ft->vport);
+ MLX5_SET(modify_flow_table_in, in, other_vport,
+ !!(ft->flags & MLX5_FLOW_TABLE_OTHER_VPORT));
MLX5_SET(modify_flow_table_in, in, modify_field_select,
MLX5_MODIFY_FLOW_TABLE_MISS_TABLE_ID);
if (next_ft) {
MLX5_SET(create_flow_group_in, in, other_vport, 1);
}
+ MLX5_SET(create_flow_group_in, in, vport_number, ft->vport);
+ MLX5_SET(create_flow_group_in, in, other_vport,
+ !!(ft->flags & MLX5_FLOW_TABLE_OTHER_VPORT));
err = mlx5_cmd_exec_inout(dev, create_flow_group, in, out);
if (!err)
fg->id = MLX5_GET(create_flow_group_out, out,
MLX5_SET(destroy_flow_group_in, in, table_type, ft->type);
MLX5_SET(destroy_flow_group_in, in, table_id, ft->id);
MLX5_SET(destroy_flow_group_in, in, group_id, fg->id);
- if (ft->vport) {
- MLX5_SET(destroy_flow_group_in, in, vport_number, ft->vport);
- MLX5_SET(destroy_flow_group_in, in, other_vport, 1);
- }
-
+ MLX5_SET(destroy_flow_group_in, in, vport_number, ft->vport);
+ MLX5_SET(destroy_flow_group_in, in, other_vport,
+ !!(ft->flags & MLX5_FLOW_TABLE_OTHER_VPORT));
return mlx5_cmd_exec_in(dev, destroy_flow_group, in);
}
MLX5_SET(set_fte_in, in, ignore_flow_level,
!!(fte->action.flags & FLOW_ACT_IGNORE_FLOW_LEVEL));
- if (ft->vport) {
- MLX5_SET(set_fte_in, in, vport_number, ft->vport);
- MLX5_SET(set_fte_in, in, other_vport, 1);
- }
+ MLX5_SET(set_fte_in, in, vport_number, ft->vport);
+ MLX5_SET(set_fte_in, in, other_vport,
+ !!(ft->flags & MLX5_FLOW_TABLE_OTHER_VPORT));
in_flow_context = MLX5_ADDR_OF(set_fte_in, in, flow_context);
MLX5_SET(flow_context, in_flow_context, group_id, group_id);
dst->dest_attr.vport.pkt_reformat->id);
}
break;
+ case MLX5_FLOW_DESTINATION_TYPE_FLOW_SAMPLER:
+ id = dst->dest_attr.sampler_id;
+ break;
default:
id = dst->dest_attr.tir_num;
}
MLX5_SET(delete_fte_in, in, table_type, ft->type);
MLX5_SET(delete_fte_in, in, table_id, ft->id);
MLX5_SET(delete_fte_in, in, flow_index, fte->index);
- if (ft->vport) {
- MLX5_SET(delete_fte_in, in, vport_number, ft->vport);
- MLX5_SET(delete_fte_in, in, other_vport, 1);
- }
+ MLX5_SET(delete_fte_in, in, vport_number, ft->vport);
+ MLX5_SET(delete_fte_in, in, other_vport,
+ !!(ft->flags & MLX5_FLOW_TABLE_OTHER_VPORT));
return mlx5_cmd_exec_in(dev, delete_fte, in);
}
goto out;
}
+ if (rule->dest_attr.type == MLX5_FLOW_DESTINATION_TYPE_PORT &&
+ --fte->dests_size) {
+ fte->modify_mask |= BIT(MLX5_SET_FTE_MODIFY_ENABLE_MASK_ACTION);
+ fte->action.action &= ~MLX5_FLOW_CONTEXT_ACTION_ALLOW;
+ goto out;
+ }
+
if ((fte->action.action & MLX5_FLOW_CONTEXT_ACTION_FWD_DEST) &&
--fte->dests_size) {
fte->modify_mask |=
{
return __mlx5_create_flow_table(ns, ft_attr, FS_FT_OP_MOD_NORMAL, 0);
}
+EXPORT_SYMBOL(mlx5_create_flow_table);
-struct mlx5_flow_table *mlx5_create_vport_flow_table(struct mlx5_flow_namespace *ns,
- int prio, int max_fte,
- u32 level, u16 vport)
+struct mlx5_flow_table *
+mlx5_create_vport_flow_table(struct mlx5_flow_namespace *ns,
+ struct mlx5_flow_table_attr *ft_attr, u16 vport)
{
- struct mlx5_flow_table_attr ft_attr = {};
-
- ft_attr.max_fte = max_fte;
- ft_attr.level = level;
- ft_attr.prio = prio;
-
- return __mlx5_create_flow_table(ns, &ft_attr, FS_FT_OP_MOD_NORMAL, vport);
+ return __mlx5_create_flow_table(ns, ft_attr, FS_FT_OP_MOD_NORMAL, vport);
}
struct mlx5_flow_table*
return fg;
}
+EXPORT_SYMBOL(mlx5_create_flow_group);
static struct mlx5_flow_rule *alloc_rule(struct mlx5_flow_destination *dest)
{
mlx5_core_warn(get_dev(&fg->node), "Flow group %d wasn't destroyed, refcount > 1\n",
fg->id);
}
+EXPORT_SYMBOL(mlx5_destroy_flow_group);
struct mlx5_flow_namespace *mlx5_get_fdb_sub_ns(struct mlx5_core_dev *dev,
int n)
u32 qpn;
};
-#define MLX5_FTE_MATCH_PARAM_RESERVED reserved_at_a00
+#define MLX5_FTE_MATCH_PARAM_RESERVED reserved_at_c00
/* Calculate the fte_match_param length and without the reserved length.
* Make sure the reserved field is the last.
*/
goto err_sriov;
}
- err = mlx5_sriov_attach(dev);
- if (err) {
- mlx5_core_err(dev, "sriov init failed %d\n", err);
- goto err_sriov;
- }
-
err = mlx5_ec_init(dev);
if (err) {
mlx5_core_err(dev, "Failed to init embedded CPU\n");
goto err_ec;
}
+ err = mlx5_sriov_attach(dev);
+ if (err) {
+ mlx5_core_err(dev, "sriov init failed %d\n", err);
+ goto err_sriov;
+ }
+
return 0;
-err_ec:
- mlx5_sriov_detach(dev);
err_sriov:
+ mlx5_ec_cleanup(dev);
+err_ec:
mlx5_cleanup_fs(dev);
err_fs:
mlx5_accel_tls_cleanup(dev);
static void mlx5_unload(struct mlx5_core_dev *dev)
{
- mlx5_ec_cleanup(dev);
mlx5_sriov_detach(dev);
+ mlx5_ec_cleanup(dev);
mlx5_cleanup_fs(dev);
mlx5_accel_ipsec_cleanup(dev);
mlx5_accel_tls_cleanup(dev);
{ PCI_VDEVICE(MELLANOX, 0xa2d2) }, /* BlueField integrated ConnectX-5 network controller */
{ PCI_VDEVICE(MELLANOX, 0xa2d3), MLX5_PCI_DEV_IS_VF}, /* BlueField integrated ConnectX-5 network controller VF */
{ PCI_VDEVICE(MELLANOX, 0xa2d6) }, /* BlueField-2 integrated ConnectX-6 Dx network controller */
+ { PCI_VDEVICE(MELLANOX, 0xa2dc) }, /* BlueField-3 integrated ConnectX-7 network controller */
{ 0, }
};
int mlx5_query_hca_caps(struct mlx5_core_dev *dev);
int mlx5_query_board_id(struct mlx5_core_dev *dev);
+int mlx5_cmd_init(struct mlx5_core_dev *dev);
+void mlx5_cmd_cleanup(struct mlx5_core_dev *dev);
+void mlx5_cmd_set_state(struct mlx5_core_dev *dev,
+ enum mlx5_cmdif_state cmdif_state);
int mlx5_cmd_init_hca(struct mlx5_core_dev *dev, uint32_t *sw_owner_id);
int mlx5_cmd_teardown_hca(struct mlx5_core_dev *dev);
int mlx5_cmd_force_teardown_hca(struct mlx5_core_dev *dev);
if (func_id)
dev->priv.vfs_pages += npages;
else if (mlx5_core_is_ecpf(dev) && !ec_function)
- dev->priv.peer_pf_pages += npages;
+ dev->priv.host_pf_pages += npages;
mlx5_core_dbg(dev, "npages %d, ec_function %d, func_id 0x%x, err %d\n",
npages, ec_function, func_id, err);
if (func_id)
dev->priv.vfs_pages -= npages;
else if (mlx5_core_is_ecpf(dev) && !ec_function)
- dev->priv.peer_pf_pages -= npages;
+ dev->priv.host_pf_pages -= npages;
mlx5_core_dbg(dev, "npages %d, ec_function %d, func_id 0x%x\n",
npages, ec_function, func_id);
}
+static u32 fwp_fill_manage_pages_out(struct fw_page *fwp, u32 *out, u32 index,
+ u32 npages)
+{
+ u32 pages_set = 0;
+ unsigned int n;
+
+ for_each_clear_bit(n, &fwp->bitmask, MLX5_NUM_4K_IN_PAGE) {
+ MLX5_ARRAY_SET64(manage_pages_out, out, pas, index + pages_set,
+ fwp->addr + (n * MLX5_ADAPTER_PAGE_SIZE));
+ pages_set++;
+
+ if (!--npages)
+ break;
+ }
+
+ return pages_set;
+}
+
static int reclaim_pages_cmd(struct mlx5_core_dev *dev,
u32 *in, int in_size, u32 *out, int out_size)
{
fwp = rb_entry(p, struct fw_page, rb_node);
p = rb_next(p);
- MLX5_ARRAY_SET64(manage_pages_out, out, pas, i, fwp->addr);
- i++;
+ i += fwp_fill_manage_pages_out(fwp, out, i, npages - i);
}
MLX5_SET(manage_pages_out, out, output_num_entries, i);
if (func_id)
dev->priv.vfs_pages -= num_claimed;
else if (mlx5_core_is_ecpf(dev) && !ec_function)
- dev->priv.peer_pf_pages -= num_claimed;
+ dev->priv.host_pf_pages -= num_claimed;
out_free:
kvfree(out);
WARN(dev->priv.vfs_pages,
"VFs FW pages counter is %d after reclaiming all pages\n",
dev->priv.vfs_pages);
- WARN(dev->priv.peer_pf_pages,
- "Peer PF FW pages counter is %d after reclaiming all pages\n",
- dev->priv.peer_pf_pages);
+ WARN(dev->priv.host_pf_pages,
+ "External host PF FW pages counter is %d after reclaiming all pages\n",
+ dev->priv.host_pf_pages);
return 0;
}
caps->eswitch_manager = MLX5_CAP_GEN(mdev, eswitch_manager);
caps->gvmi = MLX5_CAP_GEN(mdev, vhca_id);
caps->flex_protocols = MLX5_CAP_GEN(mdev, flex_parser_protocols);
+ caps->sw_format_ver = MLX5_CAP_GEN(mdev, steering_format_version);
if (caps->flex_protocols & MLX5_FLEX_PARSER_ICMP_V4_ENABLED) {
caps->flex_parser_id_icmp_dw0 = MLX5_CAP_GEN(mdev, flex_parser_id_icmp_dw0);
if (ret)
return ret;
+ if (dmn->info.caps.sw_format_ver != MLX5_STEERING_FORMAT_CONNECTX_5) {
+ mlx5dr_err(dmn, "SW steering is not supported on this device\n");
+ return -EOPNOTSUPP;
+ }
+
ret = dr_domain_query_fdb_caps(mdev, dmn);
if (ret)
return ret;
}
if (mask) {
- if (mask->match_sz > sizeof(struct mlx5dr_match_param)) {
+ if (mask->match_sz > DR_SZ_MATCH_PARAM) {
mlx5dr_err(dmn, "Invalid match size attribute\n");
return -EINVAL;
}
u32 s_idx, e_idx;
if (!value_size ||
- (value_size > sizeof(struct mlx5dr_match_param) ||
- (value_size % sizeof(u32)))) {
+ (value_size > DR_SZ_MATCH_PARAM || (value_size % sizeof(u32)))) {
mlx5dr_err(matcher->tbl->dmn, "Rule parameters length is incorrect\n");
return false;
}
#define WIRE_PORT 0xFFFF
#define DR_STE_SVLAN 0x1
#define DR_STE_CVLAN 0x2
+#define DR_SZ_MATCH_PARAM (MLX5_ST_SZ_DW_MATCH_PARAM * 4)
#define mlx5dr_err(dmn, arg...) mlx5_core_err((dmn)->mdev, ##arg)
#define mlx5dr_info(dmn, arg...) mlx5_core_info((dmn)->mdev, ##arg)
u8 max_ft_level;
u16 roce_min_src_udp;
u8 num_esw_ports;
+ u8 sw_format_ver;
bool eswitch_manager;
bool rx_sw_owner;
bool tx_sw_owner;
}
mlxfw_info(mlxfw_dev, "Initialize firmware flash process\n");
- devlink_flash_update_begin_notify(mlxfw_dev->devlink);
mlxfw_status_notify(mlxfw_dev, "Initializing firmware flash process",
NULL, 0, 0);
err = mlxfw_dev->ops->fsm_lock(mlxfw_dev, &fwhandle);
mlxfw_info(mlxfw_dev, "Firmware flash done\n");
mlxfw_status_notify(mlxfw_dev, "Firmware flash done", NULL, 0, 0);
mlxfw_mfa2_file_fini(mfa2_file);
- devlink_flash_update_end_notify(mlxfw_dev->devlink);
return 0;
err_state_wait_activate_to_locked:
mlxfw_dev->ops->fsm_release(mlxfw_dev, fwhandle);
err_fsm_lock:
mlxfw_mfa2_file_fini(mfa2_file);
- devlink_flash_update_end_notify(mlxfw_dev->devlink);
return err;
}
EXPORT_SYMBOL(mlxfw_firmware_flash);
config MLXSW_CORE
tristate "Mellanox Technologies Switch ASICs support"
select NET_DEVLINK
+ select MLXFW
help
This driver supports Mellanox Technologies Switch ASICs family.
select GENERIC_ALLOCATOR
select PARMAN
select OBJAGG
- select MLXFW
imply PTP_1588_CLOCK
select NET_PTP_CLASSIFY if PTP_1588_CLOCK
default m
if (trans->core->fw_flash_in_progress)
timeout = msecs_to_jiffies(MLXSW_EMAD_TIMEOUT_DURING_FW_FLASH_MS);
- queue_delayed_work(trans->core->emad_wq, &trans->timeout_dw, timeout);
+ queue_delayed_work(trans->core->emad_wq, &trans->timeout_dw,
+ timeout << trans->retries);
}
static int mlxsw_emad_transmit(struct mlxsw_core *mlxsw_core,
struct devlink_flash_update_params *params,
struct netlink_ext_ack *extack)
{
- const struct firmware *firmware;
- int err;
-
- err = request_firmware_direct(&firmware, params->file_name, mlxsw_core->bus_info->dev);
- if (err)
- return err;
- err = mlxsw_core_fw_flash(mlxsw_core, firmware, extack);
- release_firmware(firmware);
-
- return err;
+ return mlxsw_core_fw_flash(mlxsw_core, params->fw, extack);
}
static int mlxsw_core_devlink_param_fw_load_policy_validate(struct devlink *devlink, u32 id,
#ifndef _MLXSW_CORE_ENV_H
#define _MLXSW_CORE_ENV_H
+struct ethtool_modinfo;
+struct ethtool_eeprom;
+
int mlxsw_env_module_temp_thresholds_get(struct mlxsw_core *core, int module,
int off, int *temp);
*/
MLXSW_ITEM32(reg, spvid, sub_port, 0x00, 8, 8);
+/* reg_spvid_et_vlan
+ * EtherType used for when VLAN is pushed at ingress (for untagged
+ * packets or for QinQ push mode).
+ * 0: ether_type0 - (default)
+ * 1: ether_type1
+ * 2: ether_type2 - Reserved when Spectrum-1, supported by Spectrum-2
+ * Ethertype IDs are configured by SVER.
+ * Access: RW
+ */
+MLXSW_ITEM32(reg, spvid, et_vlan, 0x04, 16, 2);
+
/* reg_spvid_pvid
* Port default VID
* Access: RW
*/
MLXSW_ITEM32(reg, spvid, pvid, 0x04, 0, 12);
-static inline void mlxsw_reg_spvid_pack(char *payload, u8 local_port, u16 pvid)
+static inline void mlxsw_reg_spvid_pack(char *payload, u8 local_port, u16 pvid,
+ u8 et_vlan)
{
MLXSW_REG_ZERO(spvid, payload);
mlxsw_reg_spvid_local_port_set(payload, local_port);
mlxsw_reg_spvid_pvid_set(payload, pvid);
+ mlxsw_reg_spvid_et_vlan_set(payload, et_vlan);
}
/* SPVM - Switch Port VLAN Membership
}
}
+/* SPVC - Switch Port VLAN Classification Register
+ * -----------------------------------------------
+ * Configures the port to identify packets as untagged / single tagged /
+ * double packets based on the packet EtherTypes.
+ * Ethertype IDs are configured by SVER.
+ */
+#define MLXSW_REG_SPVC_ID 0x2026
+#define MLXSW_REG_SPVC_LEN 0x0C
+
+MLXSW_REG_DEFINE(spvc, MLXSW_REG_SPVC_ID, MLXSW_REG_SPVC_LEN);
+
+/* reg_spvc_local_port
+ * Local port.
+ * Access: Index
+ *
+ * Note: applies both to Rx port and Tx port, so if a packet traverses
+ * through Rx port i and a Tx port j then port i and port j must have the
+ * same configuration.
+ */
+MLXSW_ITEM32(reg, spvc, local_port, 0x00, 16, 8);
+
+/* reg_spvc_inner_et2
+ * Vlan Tag1 EtherType2 enable.
+ * Packet is initially classified as double VLAN Tag if in addition to
+ * being classified with a tag0 VLAN Tag its tag1 EtherType value is
+ * equal to ether_type2.
+ * 0: disable (default)
+ * 1: enable
+ * Access: RW
+ */
+MLXSW_ITEM32(reg, spvc, inner_et2, 0x08, 17, 1);
+
+/* reg_spvc_et2
+ * Vlan Tag0 EtherType2 enable.
+ * Packet is initially classified as VLAN Tag if its tag0 EtherType is
+ * equal to ether_type2.
+ * 0: disable (default)
+ * 1: enable
+ * Access: RW
+ */
+MLXSW_ITEM32(reg, spvc, et2, 0x08, 16, 1);
+
+/* reg_spvc_inner_et1
+ * Vlan Tag1 EtherType1 enable.
+ * Packet is initially classified as double VLAN Tag if in addition to
+ * being classified with a tag0 VLAN Tag its tag1 EtherType value is
+ * equal to ether_type1.
+ * 0: disable
+ * 1: enable (default)
+ * Access: RW
+ */
+MLXSW_ITEM32(reg, spvc, inner_et1, 0x08, 9, 1);
+
+/* reg_spvc_et1
+ * Vlan Tag0 EtherType1 enable.
+ * Packet is initially classified as VLAN Tag if its tag0 EtherType is
+ * equal to ether_type1.
+ * 0: disable
+ * 1: enable (default)
+ * Access: RW
+ */
+MLXSW_ITEM32(reg, spvc, et1, 0x08, 8, 1);
+
+/* reg_inner_et0
+ * Vlan Tag1 EtherType0 enable.
+ * Packet is initially classified as double VLAN Tag if in addition to
+ * being classified with a tag0 VLAN Tag its tag1 EtherType value is
+ * equal to ether_type0.
+ * 0: disable
+ * 1: enable (default)
+ * Access: RW
+ */
+MLXSW_ITEM32(reg, spvc, inner_et0, 0x08, 1, 1);
+
+/* reg_et0
+ * Vlan Tag0 EtherType0 enable.
+ * Packet is initially classified as VLAN Tag if its tag0 EtherType is
+ * equal to ether_type0.
+ * 0: disable
+ * 1: enable (default)
+ * Access: RW
+ */
+MLXSW_ITEM32(reg, spvc, et0, 0x08, 0, 1);
+
+static inline void mlxsw_reg_spvc_pack(char *payload, u8 local_port, bool et1,
+ bool et0)
+{
+ MLXSW_REG_ZERO(spvc, payload);
+ mlxsw_reg_spvc_local_port_set(payload, local_port);
+ /* Enable inner_et1 and inner_et0 to enable identification of double
+ * tagged packets.
+ */
+ mlxsw_reg_spvc_inner_et1_set(payload, 1);
+ mlxsw_reg_spvc_inner_et0_set(payload, 1);
+ mlxsw_reg_spvc_et1_set(payload, et1);
+ mlxsw_reg_spvc_et0_set(payload, et0);
+}
+
/* CWTP - Congetion WRED ECN TClass Profile
* ----------------------------------------
* Configures the profiles for queues of egress port and traffic class
MLXSW_REG(svpe),
MLXSW_REG(sfmr),
MLXSW_REG(spvmlr),
+ MLXSW_REG(spvc),
MLXSW_REG(cwtp),
MLXSW_REG(cwtpm),
MLXSW_REG(pgcr),
return err;
}
+static int mlxsw_sp_ethtype_to_sver_type(u16 ethtype, u8 *p_sver_type)
+{
+ switch (ethtype) {
+ case ETH_P_8021Q:
+ *p_sver_type = 0;
+ break;
+ case ETH_P_8021AD:
+ *p_sver_type = 1;
+ break;
+ default:
+ return -EINVAL;
+ }
+
+ return 0;
+}
+
static int __mlxsw_sp_port_pvid_set(struct mlxsw_sp_port *mlxsw_sp_port,
- u16 vid)
+ u16 vid, u16 ethtype)
{
struct mlxsw_sp *mlxsw_sp = mlxsw_sp_port->mlxsw_sp;
char spvid_pl[MLXSW_REG_SPVID_LEN];
+ u8 sver_type;
+ int err;
+
+ err = mlxsw_sp_ethtype_to_sver_type(ethtype, &sver_type);
+ if (err)
+ return err;
+
+ mlxsw_reg_spvid_pack(spvid_pl, mlxsw_sp_port->local_port, vid,
+ sver_type);
- mlxsw_reg_spvid_pack(spvid_pl, mlxsw_sp_port->local_port, vid);
return mlxsw_reg_write(mlxsw_sp->core, MLXSW_REG(spvid), spvid_pl);
}
return mlxsw_reg_write(mlxsw_sp->core, MLXSW_REG(spaft), spaft_pl);
}
-int mlxsw_sp_port_pvid_set(struct mlxsw_sp_port *mlxsw_sp_port, u16 vid)
+int mlxsw_sp_port_pvid_set(struct mlxsw_sp_port *mlxsw_sp_port, u16 vid,
+ u16 ethtype)
{
int err;
if (err)
return err;
} else {
- err = __mlxsw_sp_port_pvid_set(mlxsw_sp_port, vid);
+ err = __mlxsw_sp_port_pvid_set(mlxsw_sp_port, vid, ethtype);
if (err)
return err;
err = mlxsw_sp_port_allow_untagged_set(mlxsw_sp_port, true);
return 0;
err_port_allow_untagged_set:
- __mlxsw_sp_port_pvid_set(mlxsw_sp_port, mlxsw_sp_port->pvid);
+ __mlxsw_sp_port_pvid_set(mlxsw_sp_port, mlxsw_sp_port->pvid, ethtype);
return err;
}
return 0;
}
+int
+mlxsw_sp_port_vlan_classification_set(struct mlxsw_sp_port *mlxsw_sp_port,
+ bool is_8021ad_tagged,
+ bool is_8021q_tagged)
+{
+ struct mlxsw_sp *mlxsw_sp = mlxsw_sp_port->mlxsw_sp;
+ char spvc_pl[MLXSW_REG_SPVC_LEN];
+
+ mlxsw_reg_spvc_pack(spvc_pl, mlxsw_sp_port->local_port,
+ is_8021ad_tagged, is_8021q_tagged);
+ return mlxsw_reg_write(mlxsw_sp->core, MLXSW_REG(spvc), spvc_pl);
+}
+
static int mlxsw_sp_port_create(struct mlxsw_sp *mlxsw_sp, u8 local_port,
u8 split_base_local_port,
struct mlxsw_sp_port_mapping *port_mapping)
goto err_port_nve_init;
}
- err = mlxsw_sp_port_pvid_set(mlxsw_sp_port, MLXSW_SP_DEFAULT_VID);
+ err = mlxsw_sp_port_pvid_set(mlxsw_sp_port, MLXSW_SP_DEFAULT_VID,
+ ETH_P_8021Q);
if (err) {
dev_err(mlxsw_sp->bus_info->dev, "Port %d: Failed to set PVID\n",
mlxsw_sp_port->local_port);
}
mlxsw_sp_port->default_vlan = mlxsw_sp_port_vlan;
+ /* Set SPVC.et0=true and SPVC.et1=false to make the local port to treat
+ * only packets with 802.1q header as tagged packets.
+ */
+ err = mlxsw_sp_port_vlan_classification_set(mlxsw_sp_port, false, true);
+ if (err) {
+ dev_err(mlxsw_sp->bus_info->dev, "Port %d: Failed to set default VLAN classification\n",
+ local_port);
+ goto err_port_vlan_classification_set;
+ }
+
INIT_DELAYED_WORK(&mlxsw_sp_port->ptp.shaper_dw,
mlxsw_sp->ptp_ops->shaper_work);
err_register_netdev:
err_port_overheat_init_val_set:
+ mlxsw_sp_port_vlan_classification_set(mlxsw_sp_port, true, true);
+err_port_vlan_classification_set:
mlxsw_sp->ports[local_port] = NULL;
mlxsw_sp_port_vlan_destroy(mlxsw_sp_port_vlan);
err_port_vlan_create:
mlxsw_sp_port_ptp_clear(mlxsw_sp_port);
mlxsw_core_port_clear(mlxsw_sp->core, local_port, mlxsw_sp);
unregister_netdev(mlxsw_sp_port->dev); /* This calls ndo_stop */
+ mlxsw_sp_port_vlan_classification_set(mlxsw_sp_port, true, true);
mlxsw_sp->ports[local_port] = NULL;
mlxsw_sp_port_vlan_flush(mlxsw_sp_port, true);
mlxsw_sp_port_nve_fini(mlxsw_sp_port);
lag->ref_count--;
/* Make sure untagged frames are allowed to ingress */
- mlxsw_sp_port_pvid_set(mlxsw_sp_port, MLXSW_SP_DEFAULT_VID);
+ mlxsw_sp_port_pvid_set(mlxsw_sp_port, MLXSW_SP_DEFAULT_VID,
+ ETH_P_8021Q);
}
static int mlxsw_sp_lag_dist_port_add(struct mlxsw_sp_port *mlxsw_sp_port,
struct net_device *upper_dev;
struct mlxsw_sp *mlxsw_sp;
int err = 0;
+ u16 proto;
mlxsw_sp_port = netdev_priv(dev);
mlxsw_sp = mlxsw_sp_port->mlxsw_sp;
NL_SET_ERR_MSG_MOD(extack, "Can not put a VLAN on an OVS port");
return -EINVAL;
}
+ if (netif_is_bridge_master(upper_dev)) {
+ br_vlan_get_proto(upper_dev, &proto);
+ if (br_vlan_enabled(upper_dev) &&
+ proto != ETH_P_8021Q && proto != ETH_P_8021AD) {
+ NL_SET_ERR_MSG_MOD(extack, "Enslaving a port to a bridge with unknown VLAN protocol is not supported");
+ return -EOPNOTSUPP;
+ }
+ if (vlan_uses_dev(lower_dev) &&
+ br_vlan_enabled(upper_dev) &&
+ proto == ETH_P_8021AD) {
+ NL_SET_ERR_MSG_MOD(extack, "Enslaving a port that already has a VLAN upper to an 802.1ad bridge is not supported");
+ return -EOPNOTSUPP;
+ }
+ }
+ if (netif_is_bridge_port(lower_dev) && is_vlan_dev(upper_dev)) {
+ struct net_device *br_dev = netdev_master_upper_dev_get(lower_dev);
+
+ if (br_vlan_enabled(br_dev)) {
+ br_vlan_get_proto(br_dev, &proto);
+ if (proto == ETH_P_8021AD) {
+ NL_SET_ERR_MSG_MOD(extack, "VLAN uppers are not supported on a port enslaved to an 802.1ad bridge");
+ return -EOPNOTSUPP;
+ }
+ }
+ }
+ if (is_vlan_dev(upper_dev) &&
+ ntohs(vlan_dev_vlan_proto(upper_dev)) != ETH_P_8021Q) {
+ NL_SET_ERR_MSG_MOD(extack, "VLAN uppers are only supported with 802.1q VLAN protocol");
+ return -EOPNOTSUPP;
+ }
break;
case NETDEV_CHANGEUPPER:
upper_dev = info->upper_dev;
struct netdev_notifier_changeupper_info *info = ptr;
struct netlink_ext_ack *extack;
struct net_device *upper_dev;
+ u16 proto;
if (!mlxsw_sp)
return 0;
}
if (!info->linking)
break;
+ if (br_vlan_enabled(br_dev)) {
+ br_vlan_get_proto(br_dev, &proto);
+ if (proto == ETH_P_8021AD) {
+ NL_SET_ERR_MSG_MOD(extack, "Uppers are not supported on top of an 802.1ad bridge");
+ return -EOPNOTSUPP;
+ }
+ }
+ if (is_vlan_dev(upper_dev) &&
+ ntohs(vlan_dev_vlan_proto(upper_dev)) != ETH_P_8021Q) {
+ NL_SET_ERR_MSG_MOD(extack, "VLAN uppers are only supported with 802.1q VLAN protocol");
+ return -EOPNOTSUPP;
+ }
if (netif_is_macvlan(upper_dev) &&
!mlxsw_sp_rif_exists(mlxsw_sp, br_dev)) {
NL_SET_ERR_MSG_MOD(extack, "macvlan is only supported on top of router interfaces");
#ifndef _MLXSW_SPECTRUM_H
#define _MLXSW_SPECTRUM_H
+#include <linux/ethtool.h>
#include <linux/types.h>
#include <linux/netdevice.h>
#include <linux/rhashtable.h>
int prio, char *ppcnt_pl);
int mlxsw_sp_port_admin_status_set(struct mlxsw_sp_port *mlxsw_sp_port,
bool is_up);
+int
+mlxsw_sp_port_vlan_classification_set(struct mlxsw_sp_port *mlxsw_sp_port,
+ bool is_8021ad_tagged,
+ bool is_8021q_tagged);
/* spectrum_buffers.c */
struct mlxsw_sp_hdroom_prio {
int mlxsw_sp_port_vp_mode_set(struct mlxsw_sp_port *mlxsw_sp_port, bool enable);
int mlxsw_sp_port_vid_learning_set(struct mlxsw_sp_port *mlxsw_sp_port, u16 vid,
bool learn_enable);
-int mlxsw_sp_port_pvid_set(struct mlxsw_sp_port *mlxsw_sp_port, u16 vid);
+int mlxsw_sp_port_pvid_set(struct mlxsw_sp_port *mlxsw_sp_port, u16 vid,
+ u16 ethtype);
struct mlxsw_sp_port_vlan *
mlxsw_sp_port_vlan_create(struct mlxsw_sp_port *mlxsw_sp_port, u16 vid);
void mlxsw_sp_port_vlan_destroy(struct mlxsw_sp_port_vlan *mlxsw_sp_port_vlan);
mlxsw_sp_nexthop_for_each(nh, mlxsw_sp->router)
if (mlxsw_sp_nexthop_offload(nh) &&
- !mlxsw_sp_nexthop_group_has_ipip(nh))
+ !mlxsw_sp_nexthop_group_has_ipip(nh) &&
+ !mlxsw_sp_nexthop_is_discard(nh))
size++;
return size;
}
nh_count = 0;
mlxsw_sp_nexthop_for_each(nh, mlxsw_sp->router) {
if (!mlxsw_sp_nexthop_offload(nh) ||
- mlxsw_sp_nexthop_group_has_ipip(nh))
+ mlxsw_sp_nexthop_group_has_ipip(nh) ||
+ mlxsw_sp_nexthop_is_discard(nh))
continue;
if (nh_count < nh_skip)
mlxsw_sp_nexthop_for_each(nh, mlxsw_sp->router) {
if (!mlxsw_sp_nexthop_offload(nh) ||
- mlxsw_sp_nexthop_group_has_ipip(nh))
+ mlxsw_sp_nexthop_group_has_ipip(nh) ||
+ mlxsw_sp_nexthop_is_discard(nh))
continue;
mlxsw_sp_nexthop_indexes(nh, &adj_index, &adj_size,
}
static bool mlxsw_sp_ipip_can_offload_gre4(const struct mlxsw_sp *mlxsw_sp,
- const struct net_device *ol_dev,
- enum mlxsw_sp_l3proto ol_proto)
+ const struct net_device *ol_dev)
{
struct ip_tunnel *tunnel = netdev_priv(ol_dev);
__be16 okflags = TUNNEL_KEY; /* We can't offload any other features. */
struct mlxsw_sp_ipip_entry *ipip_entry);
bool (*can_offload)(const struct mlxsw_sp *mlxsw_sp,
- const struct net_device *ol_dev,
- enum mlxsw_sp_l3proto ol_proto);
+ const struct net_device *ol_dev);
/* Return a configuration for creating an overlay loopback RIF. */
struct mlxsw_sp_rif_ipip_lb_config
goto err_hashtable_init;
/* Delive these message types as PTP0. */
- message_type = BIT(MLXSW_SP_PTP_MESSAGE_TYPE_SYNC) |
- BIT(MLXSW_SP_PTP_MESSAGE_TYPE_DELAY_REQ) |
- BIT(MLXSW_SP_PTP_MESSAGE_TYPE_PDELAY_REQ) |
- BIT(MLXSW_SP_PTP_MESSAGE_TYPE_PDELAY_RESP);
+ message_type = BIT(PTP_MSGTYPE_SYNC) |
+ BIT(PTP_MSGTYPE_DELAY_REQ) |
+ BIT(PTP_MSGTYPE_PDELAY_REQ) |
+ BIT(PTP_MSGTYPE_PDELAY_RESP);
err = mlxsw_sp_ptp_mtptpt_set(mlxsw_sp, MLXSW_REG_MTPTPT_TRAP_ID_PTP0,
message_type);
if (err)
struct mlxsw_sp_port;
struct mlxsw_sp_ptp_clock;
-enum {
- MLXSW_SP_PTP_MESSAGE_TYPE_SYNC,
- MLXSW_SP_PTP_MESSAGE_TYPE_DELAY_REQ,
- MLXSW_SP_PTP_MESSAGE_TYPE_PDELAY_REQ,
- MLXSW_SP_PTP_MESSAGE_TYPE_PDELAY_RESP,
-};
-
static inline int mlxsw_sp_ptp_get_ts_info_noptp(struct ethtool_ts_info *info)
{
info->so_timestamping = SOF_TIMESTAMPING_RX_SOFTWARE |
MLXSW_SP_FIB_ENTRY_TYPE_NVE_DECAP,
};
+struct mlxsw_sp_nexthop_group_info;
struct mlxsw_sp_nexthop_group;
struct mlxsw_sp_fib_entry;
struct mlxsw_sp_fib4_entry {
struct mlxsw_sp_fib_entry common;
+ struct fib_info *fi;
u32 tb_id;
- u32 prio;
u8 tos;
u8 type;
};
/* Given decap parameters, find the corresponding IPIP entry. */
static struct mlxsw_sp_ipip_entry *
-mlxsw_sp_ipip_entry_find_by_decap(struct mlxsw_sp *mlxsw_sp,
- const struct net_device *ul_dev,
+mlxsw_sp_ipip_entry_find_by_decap(struct mlxsw_sp *mlxsw_sp, int ul_dev_ifindex,
enum mlxsw_sp_l3proto ul_proto,
union mlxsw_sp_l3addr ul_dip)
{
- struct mlxsw_sp_ipip_entry *ipip_entry;
+ struct mlxsw_sp_ipip_entry *ipip_entry = NULL;
+ struct net_device *ul_dev;
+
+ rcu_read_lock();
+
+ ul_dev = dev_get_by_index_rcu(mlxsw_sp_net(mlxsw_sp), ul_dev_ifindex);
+ if (!ul_dev)
+ goto out_unlock;
list_for_each_entry(ipip_entry, &mlxsw_sp->router->ipip_list,
ipip_list_node)
if (mlxsw_sp_ipip_entry_matches_decap(mlxsw_sp, ul_dev,
ul_proto, ul_dip,
ipip_entry))
- return ipip_entry;
+ goto out_unlock;
+
+ rcu_read_unlock();
return NULL;
+
+out_unlock:
+ rcu_read_unlock();
+ return ipip_entry;
}
static bool mlxsw_sp_netdev_ipip_type(const struct mlxsw_sp *mlxsw_sp,
const struct mlxsw_sp_ipip_ops *ops
= mlxsw_sp->router->ipip_ops_arr[ipipt];
- /* For deciding whether decap should be offloaded, we don't care about
- * overlay protocol, so ask whether either one is supported.
- */
- return ops->can_offload(mlxsw_sp, ol_dev, MLXSW_SP_L3_PROTO_IPV4) ||
- ops->can_offload(mlxsw_sp, ol_dev, MLXSW_SP_L3_PROTO_IPV6);
+ return ops->can_offload(mlxsw_sp, ol_dev);
}
static int mlxsw_sp_netdevice_ipip_ol_reg_event(struct mlxsw_sp *mlxsw_sp,
struct list_head neigh_list_node; /* member of neigh entry list */
struct list_head rif_list_node;
struct list_head router_list_node;
- struct mlxsw_sp_nexthop_group *nh_grp; /* pointer back to the group
- * this belongs to
- */
+ struct mlxsw_sp_nexthop_group_info *nhgi; /* pointer back to the group
+ * this nexthop belongs to
+ */
struct rhash_head ht_node;
+ struct neigh_table *neigh_tbl;
struct mlxsw_sp_nexthop_key key;
unsigned char gw_addr[sizeof(struct in6_addr)];
int ifindex;
offloaded:1, /* set in case the neigh is actually put into
* KVD linear area of this group.
*/
- update:1; /* set indicates that MAC of this neigh should be
+ update:1, /* set indicates that MAC of this neigh should be
* updated in HW
*/
+ discard:1; /* nexthop is programmed to discard packets */
enum mlxsw_sp_nexthop_type type;
union {
struct mlxsw_sp_neigh_entry *neigh_entry;
bool counter_valid;
};
-struct mlxsw_sp_nexthop_group {
- void *priv;
- struct rhash_head ht_node;
- struct list_head fib_list; /* list of fib entries that use this group */
- struct neigh_table *neigh_tbl;
- u8 adj_index_valid:1,
- gateway:1; /* routes using the group use a gateway */
+enum mlxsw_sp_nexthop_group_type {
+ MLXSW_SP_NEXTHOP_GROUP_TYPE_IPV4,
+ MLXSW_SP_NEXTHOP_GROUP_TYPE_IPV6,
+ MLXSW_SP_NEXTHOP_GROUP_TYPE_OBJ,
+};
+
+struct mlxsw_sp_nexthop_group_info {
+ struct mlxsw_sp_nexthop_group *nh_grp;
u32 adj_index;
u16 ecmp_size;
u16 count;
int sum_norm_weight;
+ u8 adj_index_valid:1,
+ gateway:1; /* routes using the group use a gateway */
struct mlxsw_sp_nexthop nexthops[0];
#define nh_rif nexthops[0].rif
};
+struct mlxsw_sp_nexthop_group_vr_key {
+ u16 vr_id;
+ enum mlxsw_sp_l3proto proto;
+};
+
+struct mlxsw_sp_nexthop_group_vr_entry {
+ struct list_head list; /* member in vr_list */
+ struct rhash_head ht_node; /* member in vr_ht */
+ refcount_t ref_count;
+ struct mlxsw_sp_nexthop_group_vr_key key;
+};
+
+struct mlxsw_sp_nexthop_group {
+ struct rhash_head ht_node;
+ struct list_head fib_list; /* list of fib entries that use this group */
+ union {
+ struct {
+ struct fib_info *fi;
+ } ipv4;
+ struct {
+ u32 id;
+ } obj;
+ };
+ struct mlxsw_sp_nexthop_group_info *nhgi;
+ struct list_head vr_list;
+ struct rhashtable vr_ht;
+ enum mlxsw_sp_nexthop_group_type type;
+ bool can_destroy;
+};
+
void mlxsw_sp_nexthop_counter_alloc(struct mlxsw_sp *mlxsw_sp,
struct mlxsw_sp_nexthop *nh)
{
int mlxsw_sp_nexthop_indexes(struct mlxsw_sp_nexthop *nh, u32 *p_adj_index,
u32 *p_adj_size, u32 *p_adj_hash_index)
{
- struct mlxsw_sp_nexthop_group *nh_grp = nh->nh_grp;
+ struct mlxsw_sp_nexthop_group_info *nhgi = nh->nhgi;
u32 adj_hash_index = 0;
int i;
- if (!nh->offloaded || !nh_grp->adj_index_valid)
+ if (!nh->offloaded || !nhgi->adj_index_valid)
return -EINVAL;
- *p_adj_index = nh_grp->adj_index;
- *p_adj_size = nh_grp->ecmp_size;
+ *p_adj_index = nhgi->adj_index;
+ *p_adj_size = nhgi->ecmp_size;
- for (i = 0; i < nh_grp->count; i++) {
- struct mlxsw_sp_nexthop *nh_iter = &nh_grp->nexthops[i];
+ for (i = 0; i < nhgi->count; i++) {
+ struct mlxsw_sp_nexthop *nh_iter = &nhgi->nexthops[i];
if (nh_iter == nh)
break;
bool mlxsw_sp_nexthop_group_has_ipip(struct mlxsw_sp_nexthop *nh)
{
- struct mlxsw_sp_nexthop_group *nh_grp = nh->nh_grp;
+ struct mlxsw_sp_nexthop_group_info *nhgi = nh->nhgi;
int i;
- for (i = 0; i < nh_grp->count; i++) {
- struct mlxsw_sp_nexthop *nh_iter = &nh_grp->nexthops[i];
+ for (i = 0; i < nhgi->count; i++) {
+ struct mlxsw_sp_nexthop *nh_iter = &nhgi->nexthops[i];
if (nh_iter->type == MLXSW_SP_NEXTHOP_TYPE_IPIP)
return true;
return false;
}
-static struct fib_info *
-mlxsw_sp_nexthop4_group_fi(const struct mlxsw_sp_nexthop_group *nh_grp)
+bool mlxsw_sp_nexthop_is_discard(const struct mlxsw_sp_nexthop *nh)
{
- return nh_grp->priv;
+ return nh->discard;
+}
+
+static const struct rhashtable_params mlxsw_sp_nexthop_group_vr_ht_params = {
+ .key_offset = offsetof(struct mlxsw_sp_nexthop_group_vr_entry, key),
+ .head_offset = offsetof(struct mlxsw_sp_nexthop_group_vr_entry, ht_node),
+ .key_len = sizeof(struct mlxsw_sp_nexthop_group_vr_key),
+ .automatic_shrinking = true,
+};
+
+static struct mlxsw_sp_nexthop_group_vr_entry *
+mlxsw_sp_nexthop_group_vr_entry_lookup(struct mlxsw_sp_nexthop_group *nh_grp,
+ const struct mlxsw_sp_fib *fib)
+{
+ struct mlxsw_sp_nexthop_group_vr_key key;
+
+ memset(&key, 0, sizeof(key));
+ key.vr_id = fib->vr->id;
+ key.proto = fib->proto;
+ return rhashtable_lookup_fast(&nh_grp->vr_ht, &key,
+ mlxsw_sp_nexthop_group_vr_ht_params);
+}
+
+static int
+mlxsw_sp_nexthop_group_vr_entry_create(struct mlxsw_sp_nexthop_group *nh_grp,
+ const struct mlxsw_sp_fib *fib)
+{
+ struct mlxsw_sp_nexthop_group_vr_entry *vr_entry;
+ int err;
+
+ vr_entry = kzalloc(sizeof(*vr_entry), GFP_KERNEL);
+ if (!vr_entry)
+ return -ENOMEM;
+
+ vr_entry->key.vr_id = fib->vr->id;
+ vr_entry->key.proto = fib->proto;
+ refcount_set(&vr_entry->ref_count, 1);
+
+ err = rhashtable_insert_fast(&nh_grp->vr_ht, &vr_entry->ht_node,
+ mlxsw_sp_nexthop_group_vr_ht_params);
+ if (err)
+ goto err_hashtable_insert;
+
+ list_add(&vr_entry->list, &nh_grp->vr_list);
+
+ return 0;
+
+err_hashtable_insert:
+ kfree(vr_entry);
+ return err;
+}
+
+static void
+mlxsw_sp_nexthop_group_vr_entry_destroy(struct mlxsw_sp_nexthop_group *nh_grp,
+ struct mlxsw_sp_nexthop_group_vr_entry *vr_entry)
+{
+ list_del(&vr_entry->list);
+ rhashtable_remove_fast(&nh_grp->vr_ht, &vr_entry->ht_node,
+ mlxsw_sp_nexthop_group_vr_ht_params);
+ kfree(vr_entry);
+}
+
+static int
+mlxsw_sp_nexthop_group_vr_link(struct mlxsw_sp_nexthop_group *nh_grp,
+ const struct mlxsw_sp_fib *fib)
+{
+ struct mlxsw_sp_nexthop_group_vr_entry *vr_entry;
+
+ vr_entry = mlxsw_sp_nexthop_group_vr_entry_lookup(nh_grp, fib);
+ if (vr_entry) {
+ refcount_inc(&vr_entry->ref_count);
+ return 0;
+ }
+
+ return mlxsw_sp_nexthop_group_vr_entry_create(nh_grp, fib);
+}
+
+static void
+mlxsw_sp_nexthop_group_vr_unlink(struct mlxsw_sp_nexthop_group *nh_grp,
+ const struct mlxsw_sp_fib *fib)
+{
+ struct mlxsw_sp_nexthop_group_vr_entry *vr_entry;
+
+ vr_entry = mlxsw_sp_nexthop_group_vr_entry_lookup(nh_grp, fib);
+ if (WARN_ON_ONCE(!vr_entry))
+ return;
+
+ if (!refcount_dec_and_test(&vr_entry->ref_count))
+ return;
+
+ mlxsw_sp_nexthop_group_vr_entry_destroy(nh_grp, vr_entry);
}
struct mlxsw_sp_nexthop_group_cmp_arg {
- enum mlxsw_sp_l3proto proto;
+ enum mlxsw_sp_nexthop_group_type type;
union {
struct fib_info *fi;
struct mlxsw_sp_fib6_entry *fib6_entry;
+ u32 id;
};
};
{
int i;
- for (i = 0; i < nh_grp->count; i++) {
+ for (i = 0; i < nh_grp->nhgi->count; i++) {
const struct mlxsw_sp_nexthop *nh;
- nh = &nh_grp->nexthops[i];
+ nh = &nh_grp->nhgi->nexthops[i];
if (nh->ifindex == ifindex && nh->nh_weight == weight &&
ipv6_addr_equal(gw, (struct in6_addr *) nh->gw_addr))
return true;
{
struct mlxsw_sp_rt6 *mlxsw_sp_rt6;
- if (nh_grp->count != fib6_entry->nrt6)
+ if (nh_grp->nhgi->count != fib6_entry->nrt6)
return false;
list_for_each_entry(mlxsw_sp_rt6, &fib6_entry->rt6_list, list) {
const struct mlxsw_sp_nexthop_group_cmp_arg *cmp_arg = arg->key;
const struct mlxsw_sp_nexthop_group *nh_grp = ptr;
- switch (cmp_arg->proto) {
- case MLXSW_SP_L3_PROTO_IPV4:
- return cmp_arg->fi != mlxsw_sp_nexthop4_group_fi(nh_grp);
- case MLXSW_SP_L3_PROTO_IPV6:
+ if (nh_grp->type != cmp_arg->type)
+ return 1;
+
+ switch (cmp_arg->type) {
+ case MLXSW_SP_NEXTHOP_GROUP_TYPE_IPV4:
+ return cmp_arg->fi != nh_grp->ipv4.fi;
+ case MLXSW_SP_NEXTHOP_GROUP_TYPE_IPV6:
return !mlxsw_sp_nexthop6_group_cmp(nh_grp,
cmp_arg->fib6_entry);
+ case MLXSW_SP_NEXTHOP_GROUP_TYPE_OBJ:
+ return cmp_arg->id != nh_grp->obj.id;
default:
WARN_ON(1);
return 1;
}
}
-static int
-mlxsw_sp_nexthop_group_type(const struct mlxsw_sp_nexthop_group *nh_grp)
-{
- return nh_grp->neigh_tbl->family;
-}
-
static u32 mlxsw_sp_nexthop_group_hash_obj(const void *data, u32 len, u32 seed)
{
const struct mlxsw_sp_nexthop_group *nh_grp = data;
unsigned int val;
int i;
- switch (mlxsw_sp_nexthop_group_type(nh_grp)) {
- case AF_INET:
- fi = mlxsw_sp_nexthop4_group_fi(nh_grp);
+ switch (nh_grp->type) {
+ case MLXSW_SP_NEXTHOP_GROUP_TYPE_IPV4:
+ fi = nh_grp->ipv4.fi;
return jhash(&fi, sizeof(fi), seed);
- case AF_INET6:
- val = nh_grp->count;
- for (i = 0; i < nh_grp->count; i++) {
- nh = &nh_grp->nexthops[i];
+ case MLXSW_SP_NEXTHOP_GROUP_TYPE_IPV6:
+ val = nh_grp->nhgi->count;
+ for (i = 0; i < nh_grp->nhgi->count; i++) {
+ nh = &nh_grp->nhgi->nexthops[i];
val ^= jhash(&nh->ifindex, sizeof(nh->ifindex), seed);
val ^= jhash(&nh->gw_addr, sizeof(nh->gw_addr), seed);
}
return jhash(&val, sizeof(val), seed);
+ case MLXSW_SP_NEXTHOP_GROUP_TYPE_OBJ:
+ return jhash(&nh_grp->obj.id, sizeof(nh_grp->obj.id), seed);
default:
WARN_ON(1);
return 0;
{
const struct mlxsw_sp_nexthop_group_cmp_arg *cmp_arg = data;
- switch (cmp_arg->proto) {
- case MLXSW_SP_L3_PROTO_IPV4:
+ switch (cmp_arg->type) {
+ case MLXSW_SP_NEXTHOP_GROUP_TYPE_IPV4:
return jhash(&cmp_arg->fi, sizeof(cmp_arg->fi), seed);
- case MLXSW_SP_L3_PROTO_IPV6:
+ case MLXSW_SP_NEXTHOP_GROUP_TYPE_IPV6:
return mlxsw_sp_nexthop6_group_hash(cmp_arg->fib6_entry, seed);
+ case MLXSW_SP_NEXTHOP_GROUP_TYPE_OBJ:
+ return jhash(&cmp_arg->id, sizeof(cmp_arg->id), seed);
default:
WARN_ON(1);
return 0;
static int mlxsw_sp_nexthop_group_insert(struct mlxsw_sp *mlxsw_sp,
struct mlxsw_sp_nexthop_group *nh_grp)
{
- if (mlxsw_sp_nexthop_group_type(nh_grp) == AF_INET6 &&
- !nh_grp->gateway)
+ if (nh_grp->type == MLXSW_SP_NEXTHOP_GROUP_TYPE_IPV6 &&
+ !nh_grp->nhgi->gateway)
return 0;
return rhashtable_insert_fast(&mlxsw_sp->router->nexthop_group_ht,
static void mlxsw_sp_nexthop_group_remove(struct mlxsw_sp *mlxsw_sp,
struct mlxsw_sp_nexthop_group *nh_grp)
{
- if (mlxsw_sp_nexthop_group_type(nh_grp) == AF_INET6 &&
- !nh_grp->gateway)
+ if (nh_grp->type == MLXSW_SP_NEXTHOP_GROUP_TYPE_IPV6 &&
+ !nh_grp->nhgi->gateway)
return;
rhashtable_remove_fast(&mlxsw_sp->router->nexthop_group_ht,
{
struct mlxsw_sp_nexthop_group_cmp_arg cmp_arg;
- cmp_arg.proto = MLXSW_SP_L3_PROTO_IPV4;
+ cmp_arg.type = MLXSW_SP_NEXTHOP_GROUP_TYPE_IPV4;
cmp_arg.fi = fi;
return rhashtable_lookup_fast(&mlxsw_sp->router->nexthop_group_ht,
&cmp_arg,
{
struct mlxsw_sp_nexthop_group_cmp_arg cmp_arg;
- cmp_arg.proto = MLXSW_SP_L3_PROTO_IPV6;
+ cmp_arg.type = MLXSW_SP_NEXTHOP_GROUP_TYPE_IPV6;
cmp_arg.fib6_entry = fib6_entry;
return rhashtable_lookup_fast(&mlxsw_sp->router->nexthop_group_ht,
&cmp_arg,
}
static int mlxsw_sp_adj_index_mass_update_vr(struct mlxsw_sp *mlxsw_sp,
- const struct mlxsw_sp_fib *fib,
+ enum mlxsw_sp_l3proto proto,
+ u16 vr_id,
u32 adj_index, u16 ecmp_size,
u32 new_adj_index,
u16 new_ecmp_size)
char raleu_pl[MLXSW_REG_RALEU_LEN];
mlxsw_reg_raleu_pack(raleu_pl,
- (enum mlxsw_reg_ralxx_protocol) fib->proto,
- fib->vr->id, adj_index, ecmp_size, new_adj_index,
+ (enum mlxsw_reg_ralxx_protocol) proto, vr_id,
+ adj_index, ecmp_size, new_adj_index,
new_ecmp_size);
return mlxsw_reg_write(mlxsw_sp->core, MLXSW_REG(raleu), raleu_pl);
}
struct mlxsw_sp_nexthop_group *nh_grp,
u32 old_adj_index, u16 old_ecmp_size)
{
- struct mlxsw_sp_fib_entry *fib_entry;
- struct mlxsw_sp_fib *fib = NULL;
+ struct mlxsw_sp_nexthop_group_info *nhgi = nh_grp->nhgi;
+ struct mlxsw_sp_nexthop_group_vr_entry *vr_entry;
int err;
- list_for_each_entry(fib_entry, &nh_grp->fib_list, nexthop_group_node) {
- if (fib == fib_entry->fib_node->fib)
- continue;
- fib = fib_entry->fib_node->fib;
- err = mlxsw_sp_adj_index_mass_update_vr(mlxsw_sp, fib,
+ list_for_each_entry(vr_entry, &nh_grp->vr_list, list) {
+ err = mlxsw_sp_adj_index_mass_update_vr(mlxsw_sp,
+ vr_entry->key.proto,
+ vr_entry->key.vr_id,
old_adj_index,
old_ecmp_size,
- nh_grp->adj_index,
- nh_grp->ecmp_size);
+ nhgi->adj_index,
+ nhgi->ecmp_size);
if (err)
- return err;
+ goto err_mass_update_vr;
}
return 0;
+
+err_mass_update_vr:
+ list_for_each_entry_continue_reverse(vr_entry, &nh_grp->vr_list, list)
+ mlxsw_sp_adj_index_mass_update_vr(mlxsw_sp, vr_entry->key.proto,
+ vr_entry->key.vr_id,
+ nhgi->adj_index,
+ nhgi->ecmp_size,
+ old_adj_index, old_ecmp_size);
+ return err;
}
static int __mlxsw_sp_nexthop_update(struct mlxsw_sp *mlxsw_sp, u32 adj_index,
mlxsw_reg_ratr_pack(ratr_pl, MLXSW_REG_RATR_OP_WRITE_WRITE_ENTRY,
true, MLXSW_REG_RATR_TYPE_ETHERNET,
- adj_index, neigh_entry->rif);
- mlxsw_reg_ratr_eth_entry_pack(ratr_pl, neigh_entry->ha);
+ adj_index, nh->rif->rif_index);
+ if (nh->discard)
+ mlxsw_reg_ratr_trap_action_set(ratr_pl,
+ MLXSW_REG_RATR_TRAP_ACTION_DISCARD_ERRORS);
+ else
+ mlxsw_reg_ratr_eth_entry_pack(ratr_pl, neigh_entry->ha);
if (nh->counter_valid)
mlxsw_reg_ratr_counter_pack(ratr_pl, nh->counter_index, true);
else
static int
mlxsw_sp_nexthop_group_update(struct mlxsw_sp *mlxsw_sp,
- struct mlxsw_sp_nexthop_group *nh_grp,
+ struct mlxsw_sp_nexthop_group_info *nhgi,
bool reallocate)
{
- u32 adj_index = nh_grp->adj_index; /* base */
+ u32 adj_index = nhgi->adj_index; /* base */
struct mlxsw_sp_nexthop *nh;
int i;
- for (i = 0; i < nh_grp->count; i++) {
- nh = &nh_grp->nexthops[i];
+ for (i = 0; i < nhgi->count; i++) {
+ nh = &nhgi->nexthops[i];
if (!nh->should_offload) {
nh->offloaded = 0;
}
static void
-mlxsw_sp_nexthop_group_normalize(struct mlxsw_sp_nexthop_group *nh_grp)
+mlxsw_sp_nexthop_group_normalize(struct mlxsw_sp_nexthop_group_info *nhgi)
{
int i, g = 0, sum_norm_weight = 0;
struct mlxsw_sp_nexthop *nh;
- for (i = 0; i < nh_grp->count; i++) {
- nh = &nh_grp->nexthops[i];
+ for (i = 0; i < nhgi->count; i++) {
+ nh = &nhgi->nexthops[i];
if (!nh->should_offload)
continue;
g = nh->nh_weight;
}
- for (i = 0; i < nh_grp->count; i++) {
- nh = &nh_grp->nexthops[i];
+ for (i = 0; i < nhgi->count; i++) {
+ nh = &nhgi->nexthops[i];
if (!nh->should_offload)
continue;
sum_norm_weight += nh->norm_nh_weight;
}
- nh_grp->sum_norm_weight = sum_norm_weight;
+ nhgi->sum_norm_weight = sum_norm_weight;
}
static void
-mlxsw_sp_nexthop_group_rebalance(struct mlxsw_sp_nexthop_group *nh_grp)
+mlxsw_sp_nexthop_group_rebalance(struct mlxsw_sp_nexthop_group_info *nhgi)
{
- int total = nh_grp->sum_norm_weight;
- u16 ecmp_size = nh_grp->ecmp_size;
int i, weight = 0, lower_bound = 0;
+ int total = nhgi->sum_norm_weight;
+ u16 ecmp_size = nhgi->ecmp_size;
- for (i = 0; i < nh_grp->count; i++) {
- struct mlxsw_sp_nexthop *nh = &nh_grp->nexthops[i];
+ for (i = 0; i < nhgi->count; i++) {
+ struct mlxsw_sp_nexthop *nh = &nhgi->nexthops[i];
int upper_bound;
if (!nh->should_offload)
{
int i;
- for (i = 0; i < nh_grp->count; i++) {
- struct mlxsw_sp_nexthop *nh = &nh_grp->nexthops[i];
+ for (i = 0; i < nh_grp->nhgi->count; i++) {
+ struct mlxsw_sp_nexthop *nh = &nh_grp->nhgi->nexthops[i];
if (nh->offloaded)
nh->key.fib_nh->fib_nh_flags |= RTNH_F_OFFLOAD;
__mlxsw_sp_nexthop6_group_offload_refresh(nh_grp, fib6_entry);
}
+static void
+mlxsw_sp_nexthop_obj_group_offload_refresh(struct mlxsw_sp *mlxsw_sp,
+ struct mlxsw_sp_nexthop_group *nh_grp)
+{
+ /* Do not update the flags if the nexthop group is being destroyed
+ * since:
+ * 1. The nexthop objects is being deleted, in which case the flags are
+ * irrelevant.
+ * 2. The nexthop group was replaced by a newer group, in which case
+ * the flags of the nexthop object were already updated based on the
+ * new group.
+ */
+ if (nh_grp->can_destroy)
+ return;
+
+ nexthop_set_hw_flags(mlxsw_sp_net(mlxsw_sp), nh_grp->obj.id,
+ nh_grp->nhgi->adj_index_valid, false);
+}
+
static void
mlxsw_sp_nexthop_group_offload_refresh(struct mlxsw_sp *mlxsw_sp,
struct mlxsw_sp_nexthop_group *nh_grp)
{
- switch (mlxsw_sp_nexthop_group_type(nh_grp)) {
- case AF_INET:
+ switch (nh_grp->type) {
+ case MLXSW_SP_NEXTHOP_GROUP_TYPE_IPV4:
mlxsw_sp_nexthop4_group_offload_refresh(mlxsw_sp, nh_grp);
break;
- case AF_INET6:
+ case MLXSW_SP_NEXTHOP_GROUP_TYPE_IPV6:
mlxsw_sp_nexthop6_group_offload_refresh(mlxsw_sp, nh_grp);
break;
+ case MLXSW_SP_NEXTHOP_GROUP_TYPE_OBJ:
+ mlxsw_sp_nexthop_obj_group_offload_refresh(mlxsw_sp, nh_grp);
+ break;
}
}
-static void
+static int
mlxsw_sp_nexthop_group_refresh(struct mlxsw_sp *mlxsw_sp,
struct mlxsw_sp_nexthop_group *nh_grp)
{
+ struct mlxsw_sp_nexthop_group_info *nhgi = nh_grp->nhgi;
u16 ecmp_size, old_ecmp_size;
struct mlxsw_sp_nexthop *nh;
bool offload_change = false;
u32 adj_index;
bool old_adj_index_valid;
+ int i, err2, err = 0;
u32 old_adj_index;
- int i;
- int err;
- if (!nh_grp->gateway) {
- mlxsw_sp_nexthop_fib_entries_update(mlxsw_sp, nh_grp);
- return;
- }
+ if (!nhgi->gateway)
+ return mlxsw_sp_nexthop_fib_entries_update(mlxsw_sp, nh_grp);
- for (i = 0; i < nh_grp->count; i++) {
- nh = &nh_grp->nexthops[i];
+ for (i = 0; i < nhgi->count; i++) {
+ nh = &nhgi->nexthops[i];
if (nh->should_offload != nh->offloaded) {
offload_change = true;
/* Nothing was added or removed, so no need to reallocate. Just
* update MAC on existing adjacency indexes.
*/
- err = mlxsw_sp_nexthop_group_update(mlxsw_sp, nh_grp, false);
+ err = mlxsw_sp_nexthop_group_update(mlxsw_sp, nhgi, false);
if (err) {
dev_warn(mlxsw_sp->bus_info->dev, "Failed to update neigh MAC in adjacency table.\n");
goto set_trap;
}
- return;
+ return 0;
}
- mlxsw_sp_nexthop_group_normalize(nh_grp);
- if (!nh_grp->sum_norm_weight)
+ mlxsw_sp_nexthop_group_normalize(nhgi);
+ if (!nhgi->sum_norm_weight)
/* No neigh of this group is connected so we just set
* the trap and let everthing flow through kernel.
*/
goto set_trap;
- ecmp_size = nh_grp->sum_norm_weight;
+ ecmp_size = nhgi->sum_norm_weight;
err = mlxsw_sp_fix_adj_grp_size(mlxsw_sp, &ecmp_size);
if (err)
/* No valid allocation size available. */
dev_warn(mlxsw_sp->bus_info->dev, "Failed to allocate KVD linear area for nexthop group.\n");
goto set_trap;
}
- old_adj_index_valid = nh_grp->adj_index_valid;
- old_adj_index = nh_grp->adj_index;
- old_ecmp_size = nh_grp->ecmp_size;
- nh_grp->adj_index_valid = 1;
- nh_grp->adj_index = adj_index;
- nh_grp->ecmp_size = ecmp_size;
- mlxsw_sp_nexthop_group_rebalance(nh_grp);
- err = mlxsw_sp_nexthop_group_update(mlxsw_sp, nh_grp, true);
+ old_adj_index_valid = nhgi->adj_index_valid;
+ old_adj_index = nhgi->adj_index;
+ old_ecmp_size = nhgi->ecmp_size;
+ nhgi->adj_index_valid = 1;
+ nhgi->adj_index = adj_index;
+ nhgi->ecmp_size = ecmp_size;
+ mlxsw_sp_nexthop_group_rebalance(nhgi);
+ err = mlxsw_sp_nexthop_group_update(mlxsw_sp, nhgi, true);
if (err) {
dev_warn(mlxsw_sp->bus_info->dev, "Failed to update neigh MAC in adjacency table.\n");
goto set_trap;
dev_warn(mlxsw_sp->bus_info->dev, "Failed to add adjacency index to fib entries.\n");
goto set_trap;
}
- return;
+ return 0;
}
err = mlxsw_sp_adj_index_mass_update(mlxsw_sp, nh_grp,
goto set_trap;
}
- return;
+ return 0;
set_trap:
- old_adj_index_valid = nh_grp->adj_index_valid;
- nh_grp->adj_index_valid = 0;
- for (i = 0; i < nh_grp->count; i++) {
- nh = &nh_grp->nexthops[i];
+ old_adj_index_valid = nhgi->adj_index_valid;
+ nhgi->adj_index_valid = 0;
+ for (i = 0; i < nhgi->count; i++) {
+ nh = &nhgi->nexthops[i];
nh->offloaded = 0;
}
- err = mlxsw_sp_nexthop_fib_entries_update(mlxsw_sp, nh_grp);
- if (err)
+ err2 = mlxsw_sp_nexthop_fib_entries_update(mlxsw_sp, nh_grp);
+ if (err2)
dev_warn(mlxsw_sp->bus_info->dev, "Failed to set traps for fib entries.\n");
mlxsw_sp_nexthop_group_offload_refresh(mlxsw_sp, nh_grp);
if (old_adj_index_valid)
mlxsw_sp_kvdl_free(mlxsw_sp, MLXSW_SP_KVDL_ENTRY_TYPE_ADJ,
- nh_grp->ecmp_size, nh_grp->adj_index);
+ nhgi->ecmp_size, nhgi->adj_index);
+ return err;
}
static void __mlxsw_sp_nexthop_neigh_update(struct mlxsw_sp_nexthop *nh,
nh = list_first_entry(&neigh_entry->nexthop_list,
struct mlxsw_sp_nexthop, neigh_list_node);
- n = neigh_lookup(nh->nh_grp->neigh_tbl, &nh->gw_addr, nh->rif->dev);
+ n = neigh_lookup(nh->neigh_tbl, &nh->gw_addr, nh->rif->dev);
if (!n) {
- n = neigh_create(nh->nh_grp->neigh_tbl, &nh->gw_addr,
- nh->rif->dev);
+ n = neigh_create(nh->neigh_tbl, &nh->gw_addr, nh->rif->dev);
if (IS_ERR(n))
return PTR_ERR(n);
neigh_event_send(n, NULL);
neigh_release(old_n);
neigh_clone(n);
__mlxsw_sp_nexthop_neigh_update(nh, !entry_connected);
- mlxsw_sp_nexthop_group_refresh(mlxsw_sp, nh->nh_grp);
+ mlxsw_sp_nexthop_group_refresh(mlxsw_sp, nh->nhgi->nh_grp);
}
neigh_release(n);
list_for_each_entry(nh, &neigh_entry->nexthop_list,
neigh_list_node) {
__mlxsw_sp_nexthop_neigh_update(nh, removing);
- mlxsw_sp_nexthop_group_refresh(mlxsw_sp, nh->nh_grp);
+ mlxsw_sp_nexthop_group_refresh(mlxsw_sp, nh->nhgi->nh_grp);
}
}
u8 nud_state, dead;
int err;
- if (!nh->nh_grp->gateway || nh->neigh_entry)
+ if (!nh->nhgi->gateway || nh->neigh_entry)
return 0;
/* Take a reference of neigh here ensuring that neigh would
* The reference is taken either in neigh_lookup() or
* in neigh_create() in case n is not found.
*/
- n = neigh_lookup(nh->nh_grp->neigh_tbl, &nh->gw_addr, nh->rif->dev);
+ n = neigh_lookup(nh->neigh_tbl, &nh->gw_addr, nh->rif->dev);
if (!n) {
- n = neigh_create(nh->nh_grp->neigh_tbl, &nh->gw_addr,
- nh->rif->dev);
+ n = neigh_create(nh->neigh_tbl, &nh->gw_addr, nh->rif->dev);
if (IS_ERR(n))
return PTR_ERR(n);
neigh_event_send(n, NULL);
{
bool removing;
- if (!nh->nh_grp->gateway || nh->ipip_entry)
+ if (!nh->nhgi->gateway || nh->ipip_entry)
return;
nh->ipip_entry = ipip_entry;
mlxsw_sp_netdev_ipip_type(mlxsw_sp, dev, p_ipipt);
}
-static void mlxsw_sp_nexthop_type_fini(struct mlxsw_sp *mlxsw_sp,
- struct mlxsw_sp_nexthop *nh)
-{
- switch (nh->type) {
- case MLXSW_SP_NEXTHOP_TYPE_ETH:
- mlxsw_sp_nexthop_neigh_fini(mlxsw_sp, nh);
- mlxsw_sp_nexthop_rif_fini(nh);
- break;
- case MLXSW_SP_NEXTHOP_TYPE_IPIP:
- mlxsw_sp_nexthop_rif_fini(nh);
- mlxsw_sp_nexthop_ipip_fini(mlxsw_sp, nh);
- break;
- }
-}
-
-static int mlxsw_sp_nexthop4_type_init(struct mlxsw_sp *mlxsw_sp,
- struct mlxsw_sp_nexthop *nh,
- struct fib_nh *fib_nh)
+static int mlxsw_sp_nexthop_type_init(struct mlxsw_sp *mlxsw_sp,
+ struct mlxsw_sp_nexthop *nh,
+ const struct net_device *dev)
{
const struct mlxsw_sp_ipip_ops *ipip_ops;
- struct net_device *dev = fib_nh->fib_nh_dev;
struct mlxsw_sp_ipip_entry *ipip_entry;
struct mlxsw_sp_rif *rif;
int err;
ipip_entry = mlxsw_sp_ipip_entry_find_by_ol_dev(mlxsw_sp, dev);
if (ipip_entry) {
ipip_ops = mlxsw_sp->router->ipip_ops_arr[ipip_entry->ipipt];
- if (ipip_ops->can_offload(mlxsw_sp, dev,
- MLXSW_SP_L3_PROTO_IPV4)) {
+ if (ipip_ops->can_offload(mlxsw_sp, dev)) {
nh->type = MLXSW_SP_NEXTHOP_TYPE_IPIP;
mlxsw_sp_nexthop_ipip_init(mlxsw_sp, nh, ipip_entry);
return 0;
return err;
}
-static void mlxsw_sp_nexthop4_type_fini(struct mlxsw_sp *mlxsw_sp,
- struct mlxsw_sp_nexthop *nh)
+static void mlxsw_sp_nexthop_type_fini(struct mlxsw_sp *mlxsw_sp,
+ struct mlxsw_sp_nexthop *nh)
{
- mlxsw_sp_nexthop_type_fini(mlxsw_sp, nh);
+ switch (nh->type) {
+ case MLXSW_SP_NEXTHOP_TYPE_ETH:
+ mlxsw_sp_nexthop_neigh_fini(mlxsw_sp, nh);
+ mlxsw_sp_nexthop_rif_fini(nh);
+ break;
+ case MLXSW_SP_NEXTHOP_TYPE_IPIP:
+ mlxsw_sp_nexthop_rif_fini(nh);
+ mlxsw_sp_nexthop_ipip_fini(mlxsw_sp, nh);
+ break;
+ }
}
static int mlxsw_sp_nexthop4_init(struct mlxsw_sp *mlxsw_sp,
struct in_device *in_dev;
int err;
- nh->nh_grp = nh_grp;
+ nh->nhgi = nh_grp->nhgi;
nh->key.fib_nh = fib_nh;
#ifdef CONFIG_IP_ROUTE_MULTIPATH
nh->nh_weight = fib_nh->fib_nh_weight;
nh->nh_weight = 1;
#endif
memcpy(&nh->gw_addr, &fib_nh->fib_nh_gw4, sizeof(fib_nh->fib_nh_gw4));
+ nh->neigh_tbl = &arp_tbl;
err = mlxsw_sp_nexthop_insert(mlxsw_sp, nh);
if (err)
return err;
if (!dev)
return 0;
+ nh->ifindex = dev->ifindex;
rcu_read_lock();
in_dev = __in_dev_get_rcu(dev);
}
rcu_read_unlock();
- err = mlxsw_sp_nexthop4_type_init(mlxsw_sp, nh, fib_nh);
+ err = mlxsw_sp_nexthop_type_init(mlxsw_sp, nh, dev);
if (err)
goto err_nexthop_neigh_init;
static void mlxsw_sp_nexthop4_fini(struct mlxsw_sp *mlxsw_sp,
struct mlxsw_sp_nexthop *nh)
{
- mlxsw_sp_nexthop4_type_fini(mlxsw_sp, nh);
+ mlxsw_sp_nexthop_type_fini(mlxsw_sp, nh);
list_del(&nh->router_list_node);
mlxsw_sp_nexthop_counter_free(mlxsw_sp, nh);
mlxsw_sp_nexthop_remove(mlxsw_sp, nh);
switch (event) {
case FIB_EVENT_NH_ADD:
- mlxsw_sp_nexthop4_type_init(mlxsw_sp, nh, fib_nh);
+ mlxsw_sp_nexthop_type_init(mlxsw_sp, nh, fib_nh->fib_nh_dev);
break;
case FIB_EVENT_NH_DEL:
- mlxsw_sp_nexthop4_type_fini(mlxsw_sp, nh);
+ mlxsw_sp_nexthop_type_fini(mlxsw_sp, nh);
break;
}
- mlxsw_sp_nexthop_group_refresh(mlxsw_sp, nh->nh_grp);
+ mlxsw_sp_nexthop_group_refresh(mlxsw_sp, nh->nhgi->nh_grp);
}
static void mlxsw_sp_nexthop_rif_update(struct mlxsw_sp *mlxsw_sp,
}
__mlxsw_sp_nexthop_neigh_update(nh, removing);
- mlxsw_sp_nexthop_group_refresh(mlxsw_sp, nh->nh_grp);
+ mlxsw_sp_nexthop_group_refresh(mlxsw_sp, nh->nhgi->nh_grp);
}
}
-static void mlxsw_sp_nexthop_rif_migrate(struct mlxsw_sp *mlxsw_sp,
- struct mlxsw_sp_rif *old_rif,
- struct mlxsw_sp_rif *new_rif)
+static void mlxsw_sp_nexthop_rif_migrate(struct mlxsw_sp *mlxsw_sp,
+ struct mlxsw_sp_rif *old_rif,
+ struct mlxsw_sp_rif *new_rif)
+{
+ struct mlxsw_sp_nexthop *nh;
+
+ list_splice_init(&old_rif->nexthop_list, &new_rif->nexthop_list);
+ list_for_each_entry(nh, &new_rif->nexthop_list, rif_list_node)
+ nh->rif = new_rif;
+ mlxsw_sp_nexthop_rif_update(mlxsw_sp, new_rif);
+}
+
+static void mlxsw_sp_nexthop_rif_gone_sync(struct mlxsw_sp *mlxsw_sp,
+ struct mlxsw_sp_rif *rif)
+{
+ struct mlxsw_sp_nexthop *nh, *tmp;
+
+ list_for_each_entry_safe(nh, tmp, &rif->nexthop_list, rif_list_node) {
+ mlxsw_sp_nexthop_type_fini(mlxsw_sp, nh);
+ mlxsw_sp_nexthop_group_refresh(mlxsw_sp, nh->nhgi->nh_grp);
+ }
+}
+
+static int
+mlxsw_sp_nexthop_obj_single_validate(struct mlxsw_sp *mlxsw_sp,
+ const struct nh_notifier_single_info *nh,
+ struct netlink_ext_ack *extack)
+{
+ int err = -EINVAL;
+
+ if (nh->is_fdb)
+ NL_SET_ERR_MSG_MOD(extack, "FDB nexthops are not supported");
+ else if (nh->has_encap)
+ NL_SET_ERR_MSG_MOD(extack, "Encapsulating nexthops are not supported");
+ else
+ err = 0;
+
+ return err;
+}
+
+static int
+mlxsw_sp_nexthop_obj_group_validate(struct mlxsw_sp *mlxsw_sp,
+ const struct nh_notifier_grp_info *nh_grp,
+ struct netlink_ext_ack *extack)
+{
+ int i;
+
+ if (nh_grp->is_fdb) {
+ NL_SET_ERR_MSG_MOD(extack, "FDB nexthop groups are not supported");
+ return -EINVAL;
+ }
+
+ for (i = 0; i < nh_grp->num_nh; i++) {
+ const struct nh_notifier_single_info *nh;
+ int err;
+
+ nh = &nh_grp->nh_entries[i].nh;
+ err = mlxsw_sp_nexthop_obj_single_validate(mlxsw_sp, nh,
+ extack);
+ if (err)
+ return err;
+
+ /* Device only nexthops with an IPIP device are programmed as
+ * encapsulating adjacency entries.
+ */
+ if (!nh->gw_family && !nh->is_reject &&
+ !mlxsw_sp_netdev_ipip_type(mlxsw_sp, nh->dev, NULL)) {
+ NL_SET_ERR_MSG_MOD(extack, "Nexthop group entry does not have a gateway");
+ return -EINVAL;
+ }
+ }
+
+ return 0;
+}
+
+static int mlxsw_sp_nexthop_obj_validate(struct mlxsw_sp *mlxsw_sp,
+ unsigned long event,
+ struct nh_notifier_info *info)
+{
+ if (event != NEXTHOP_EVENT_REPLACE)
+ return 0;
+
+ if (!info->is_grp)
+ return mlxsw_sp_nexthop_obj_single_validate(mlxsw_sp, info->nh,
+ info->extack);
+ return mlxsw_sp_nexthop_obj_group_validate(mlxsw_sp, info->nh_grp,
+ info->extack);
+}
+
+static bool mlxsw_sp_nexthop_obj_is_gateway(struct mlxsw_sp *mlxsw_sp,
+ const struct nh_notifier_info *info)
+{
+ const struct net_device *dev;
+
+ if (info->is_grp)
+ /* Already validated earlier. */
+ return true;
+
+ dev = info->nh->dev;
+ return info->nh->gw_family || info->nh->is_reject ||
+ mlxsw_sp_netdev_ipip_type(mlxsw_sp, dev, NULL);
+}
+
+static void mlxsw_sp_nexthop_obj_blackhole_init(struct mlxsw_sp *mlxsw_sp,
+ struct mlxsw_sp_nexthop *nh)
+{
+ u16 lb_rif_index = mlxsw_sp->router->lb_rif_index;
+
+ nh->discard = 1;
+ nh->should_offload = 1;
+ /* While nexthops that discard packets do not forward packets
+ * via an egress RIF, they still need to be programmed using a
+ * valid RIF, so use the loopback RIF created during init.
+ */
+ nh->rif = mlxsw_sp->router->rifs[lb_rif_index];
+}
+
+static void mlxsw_sp_nexthop_obj_blackhole_fini(struct mlxsw_sp *mlxsw_sp,
+ struct mlxsw_sp_nexthop *nh)
+{
+ nh->rif = NULL;
+ nh->should_offload = 0;
+}
+
+static int
+mlxsw_sp_nexthop_obj_init(struct mlxsw_sp *mlxsw_sp,
+ struct mlxsw_sp_nexthop_group *nh_grp,
+ struct mlxsw_sp_nexthop *nh,
+ struct nh_notifier_single_info *nh_obj, int weight)
+{
+ struct net_device *dev = nh_obj->dev;
+ int err;
+
+ nh->nhgi = nh_grp->nhgi;
+ nh->nh_weight = weight;
+
+ switch (nh_obj->gw_family) {
+ case AF_INET:
+ memcpy(&nh->gw_addr, &nh_obj->ipv4, sizeof(nh_obj->ipv4));
+ nh->neigh_tbl = &arp_tbl;
+ break;
+ case AF_INET6:
+ memcpy(&nh->gw_addr, &nh_obj->ipv6, sizeof(nh_obj->ipv6));
+#if IS_ENABLED(CONFIG_IPV6)
+ nh->neigh_tbl = &nd_tbl;
+#endif
+ break;
+ }
+
+ mlxsw_sp_nexthop_counter_alloc(mlxsw_sp, nh);
+ list_add_tail(&nh->router_list_node, &mlxsw_sp->router->nexthop_list);
+ nh->ifindex = dev->ifindex;
+
+ err = mlxsw_sp_nexthop_type_init(mlxsw_sp, nh, dev);
+ if (err)
+ goto err_type_init;
+
+ if (nh_obj->is_reject)
+ mlxsw_sp_nexthop_obj_blackhole_init(mlxsw_sp, nh);
+
+ return 0;
+
+err_type_init:
+ list_del(&nh->router_list_node);
+ mlxsw_sp_nexthop_counter_free(mlxsw_sp, nh);
+ return err;
+}
+
+static void mlxsw_sp_nexthop_obj_fini(struct mlxsw_sp *mlxsw_sp,
+ struct mlxsw_sp_nexthop *nh)
+{
+ if (nh->discard)
+ mlxsw_sp_nexthop_obj_blackhole_fini(mlxsw_sp, nh);
+ mlxsw_sp_nexthop_type_fini(mlxsw_sp, nh);
+ list_del(&nh->router_list_node);
+ mlxsw_sp_nexthop_counter_free(mlxsw_sp, nh);
+}
+
+static int
+mlxsw_sp_nexthop_obj_group_info_init(struct mlxsw_sp *mlxsw_sp,
+ struct mlxsw_sp_nexthop_group *nh_grp,
+ struct nh_notifier_info *info)
+{
+ unsigned int nhs = info->is_grp ? info->nh_grp->num_nh : 1;
+ struct mlxsw_sp_nexthop_group_info *nhgi;
+ struct mlxsw_sp_nexthop *nh;
+ int err, i;
+
+ nhgi = kzalloc(struct_size(nhgi, nexthops, nhs), GFP_KERNEL);
+ if (!nhgi)
+ return -ENOMEM;
+ nh_grp->nhgi = nhgi;
+ nhgi->nh_grp = nh_grp;
+ nhgi->gateway = mlxsw_sp_nexthop_obj_is_gateway(mlxsw_sp, info);
+ nhgi->count = nhs;
+ for (i = 0; i < nhgi->count; i++) {
+ struct nh_notifier_single_info *nh_obj;
+ int weight;
+
+ nh = &nhgi->nexthops[i];
+ if (info->is_grp) {
+ nh_obj = &info->nh_grp->nh_entries[i].nh;
+ weight = info->nh_grp->nh_entries[i].weight;
+ } else {
+ nh_obj = info->nh;
+ weight = 1;
+ }
+ err = mlxsw_sp_nexthop_obj_init(mlxsw_sp, nh_grp, nh, nh_obj,
+ weight);
+ if (err)
+ goto err_nexthop_obj_init;
+ }
+ err = mlxsw_sp_nexthop_group_refresh(mlxsw_sp, nh_grp);
+ if (err) {
+ NL_SET_ERR_MSG_MOD(info->extack, "Failed to write adjacency entries to the device");
+ goto err_group_refresh;
+ }
+
+ return 0;
+
+err_group_refresh:
+ i = nhgi->count;
+err_nexthop_obj_init:
+ for (i--; i >= 0; i--) {
+ nh = &nhgi->nexthops[i];
+ mlxsw_sp_nexthop_obj_fini(mlxsw_sp, nh);
+ }
+ kfree(nhgi);
+ return err;
+}
+
+static void
+mlxsw_sp_nexthop_obj_group_info_fini(struct mlxsw_sp *mlxsw_sp,
+ struct mlxsw_sp_nexthop_group *nh_grp)
+{
+ struct mlxsw_sp_nexthop_group_info *nhgi = nh_grp->nhgi;
+ int i;
+
+ for (i = nhgi->count - 1; i >= 0; i--) {
+ struct mlxsw_sp_nexthop *nh = &nhgi->nexthops[i];
+
+ mlxsw_sp_nexthop_obj_fini(mlxsw_sp, nh);
+ }
+ mlxsw_sp_nexthop_group_refresh(mlxsw_sp, nh_grp);
+ WARN_ON_ONCE(nhgi->adj_index_valid);
+ kfree(nhgi);
+}
+
+static struct mlxsw_sp_nexthop_group *
+mlxsw_sp_nexthop_obj_group_create(struct mlxsw_sp *mlxsw_sp,
+ struct nh_notifier_info *info)
+{
+ struct mlxsw_sp_nexthop_group *nh_grp;
+ int err;
+
+ nh_grp = kzalloc(sizeof(*nh_grp), GFP_KERNEL);
+ if (!nh_grp)
+ return ERR_PTR(-ENOMEM);
+ INIT_LIST_HEAD(&nh_grp->vr_list);
+ err = rhashtable_init(&nh_grp->vr_ht,
+ &mlxsw_sp_nexthop_group_vr_ht_params);
+ if (err)
+ goto err_nexthop_group_vr_ht_init;
+ INIT_LIST_HEAD(&nh_grp->fib_list);
+ nh_grp->type = MLXSW_SP_NEXTHOP_GROUP_TYPE_OBJ;
+ nh_grp->obj.id = info->id;
+
+ err = mlxsw_sp_nexthop_obj_group_info_init(mlxsw_sp, nh_grp, info);
+ if (err)
+ goto err_nexthop_group_info_init;
+
+ nh_grp->can_destroy = false;
+
+ return nh_grp;
+
+err_nexthop_group_info_init:
+ rhashtable_destroy(&nh_grp->vr_ht);
+err_nexthop_group_vr_ht_init:
+ kfree(nh_grp);
+ return ERR_PTR(err);
+}
+
+static void
+mlxsw_sp_nexthop_obj_group_destroy(struct mlxsw_sp *mlxsw_sp,
+ struct mlxsw_sp_nexthop_group *nh_grp)
+{
+ if (!nh_grp->can_destroy)
+ return;
+ mlxsw_sp_nexthop_obj_group_info_fini(mlxsw_sp, nh_grp);
+ WARN_ON_ONCE(!list_empty(&nh_grp->fib_list));
+ WARN_ON_ONCE(!list_empty(&nh_grp->vr_list));
+ rhashtable_destroy(&nh_grp->vr_ht);
+ kfree(nh_grp);
+}
+
+static struct mlxsw_sp_nexthop_group *
+mlxsw_sp_nexthop_obj_group_lookup(struct mlxsw_sp *mlxsw_sp, u32 id)
+{
+ struct mlxsw_sp_nexthop_group_cmp_arg cmp_arg;
+
+ cmp_arg.type = MLXSW_SP_NEXTHOP_GROUP_TYPE_OBJ;
+ cmp_arg.id = id;
+ return rhashtable_lookup_fast(&mlxsw_sp->router->nexthop_group_ht,
+ &cmp_arg,
+ mlxsw_sp_nexthop_group_ht_params);
+}
+
+static int mlxsw_sp_nexthop_obj_group_add(struct mlxsw_sp *mlxsw_sp,
+ struct mlxsw_sp_nexthop_group *nh_grp)
+{
+ return mlxsw_sp_nexthop_group_insert(mlxsw_sp, nh_grp);
+}
+
+static int
+mlxsw_sp_nexthop_obj_group_replace(struct mlxsw_sp *mlxsw_sp,
+ struct mlxsw_sp_nexthop_group *nh_grp,
+ struct mlxsw_sp_nexthop_group *old_nh_grp,
+ struct netlink_ext_ack *extack)
+{
+ struct mlxsw_sp_nexthop_group_info *old_nhgi = old_nh_grp->nhgi;
+ struct mlxsw_sp_nexthop_group_info *new_nhgi = nh_grp->nhgi;
+ int err;
+
+ old_nh_grp->nhgi = new_nhgi;
+ new_nhgi->nh_grp = old_nh_grp;
+ nh_grp->nhgi = old_nhgi;
+ old_nhgi->nh_grp = nh_grp;
+
+ if (old_nhgi->adj_index_valid && new_nhgi->adj_index_valid) {
+ /* Both the old adjacency index and the new one are valid.
+ * Routes are currently using the old one. Tell the device to
+ * replace the old adjacency index with the new one.
+ */
+ err = mlxsw_sp_adj_index_mass_update(mlxsw_sp, old_nh_grp,
+ old_nhgi->adj_index,
+ old_nhgi->ecmp_size);
+ if (err) {
+ NL_SET_ERR_MSG_MOD(extack, "Failed to replace old adjacency index with new one");
+ goto err_out;
+ }
+ } else if (old_nhgi->adj_index_valid && !new_nhgi->adj_index_valid) {
+ /* The old adjacency index is valid, while the new one is not.
+ * Iterate over all the routes using the group and change them
+ * to trap packets to the CPU.
+ */
+ err = mlxsw_sp_nexthop_fib_entries_update(mlxsw_sp, old_nh_grp);
+ if (err) {
+ NL_SET_ERR_MSG_MOD(extack, "Failed to update routes to trap packets");
+ goto err_out;
+ }
+ } else if (!old_nhgi->adj_index_valid && new_nhgi->adj_index_valid) {
+ /* The old adjacency index is invalid, while the new one is.
+ * Iterate over all the routes using the group and change them
+ * to forward packets using the new valid index.
+ */
+ err = mlxsw_sp_nexthop_fib_entries_update(mlxsw_sp, old_nh_grp);
+ if (err) {
+ NL_SET_ERR_MSG_MOD(extack, "Failed to update routes to forward packets");
+ goto err_out;
+ }
+ }
+
+ /* Make sure the flags are set / cleared based on the new nexthop group
+ * information.
+ */
+ mlxsw_sp_nexthop_obj_group_offload_refresh(mlxsw_sp, old_nh_grp);
+
+ /* At this point 'nh_grp' is just a shell that is not used by anyone
+ * and its nexthop group info is the old info that was just replaced
+ * with the new one. Remove it.
+ */
+ nh_grp->can_destroy = true;
+ mlxsw_sp_nexthop_obj_group_destroy(mlxsw_sp, nh_grp);
+
+ return 0;
+
+err_out:
+ old_nhgi->nh_grp = old_nh_grp;
+ nh_grp->nhgi = new_nhgi;
+ new_nhgi->nh_grp = nh_grp;
+ old_nh_grp->nhgi = old_nhgi;
+ return err;
+}
+
+static int mlxsw_sp_nexthop_obj_new(struct mlxsw_sp *mlxsw_sp,
+ struct nh_notifier_info *info)
+{
+ struct mlxsw_sp_nexthop_group *nh_grp, *old_nh_grp;
+ struct netlink_ext_ack *extack = info->extack;
+ int err;
+
+ nh_grp = mlxsw_sp_nexthop_obj_group_create(mlxsw_sp, info);
+ if (IS_ERR(nh_grp))
+ return PTR_ERR(nh_grp);
+
+ old_nh_grp = mlxsw_sp_nexthop_obj_group_lookup(mlxsw_sp, info->id);
+ if (!old_nh_grp)
+ err = mlxsw_sp_nexthop_obj_group_add(mlxsw_sp, nh_grp);
+ else
+ err = mlxsw_sp_nexthop_obj_group_replace(mlxsw_sp, nh_grp,
+ old_nh_grp, extack);
+
+ if (err) {
+ nh_grp->can_destroy = true;
+ mlxsw_sp_nexthop_obj_group_destroy(mlxsw_sp, nh_grp);
+ }
+
+ return err;
+}
+
+static void mlxsw_sp_nexthop_obj_del(struct mlxsw_sp *mlxsw_sp,
+ struct nh_notifier_info *info)
+{
+ struct mlxsw_sp_nexthop_group *nh_grp;
+
+ nh_grp = mlxsw_sp_nexthop_obj_group_lookup(mlxsw_sp, info->id);
+ if (!nh_grp)
+ return;
+
+ nh_grp->can_destroy = true;
+ mlxsw_sp_nexthop_group_remove(mlxsw_sp, nh_grp);
+
+ /* If the group still has routes using it, then defer the delete
+ * operation until the last route using it is deleted.
+ */
+ if (!list_empty(&nh_grp->fib_list))
+ return;
+ mlxsw_sp_nexthop_obj_group_destroy(mlxsw_sp, nh_grp);
+}
+
+static int mlxsw_sp_nexthop_obj_event(struct notifier_block *nb,
+ unsigned long event, void *ptr)
+{
+ struct nh_notifier_info *info = ptr;
+ struct mlxsw_sp_router *router;
+ int err = 0;
+
+ router = container_of(nb, struct mlxsw_sp_router, nexthop_nb);
+ err = mlxsw_sp_nexthop_obj_validate(router->mlxsw_sp, event, info);
+ if (err)
+ goto out;
+
+ mutex_lock(&router->lock);
+
+ ASSERT_RTNL();
+
+ switch (event) {
+ case NEXTHOP_EVENT_REPLACE:
+ err = mlxsw_sp_nexthop_obj_new(router->mlxsw_sp, info);
+ break;
+ case NEXTHOP_EVENT_DEL:
+ mlxsw_sp_nexthop_obj_del(router->mlxsw_sp, info);
+ break;
+ default:
+ break;
+ }
+
+ mutex_unlock(&router->lock);
+
+out:
+ return notifier_from_errno(err);
+}
+
+static bool mlxsw_sp_fi_is_gateway(const struct mlxsw_sp *mlxsw_sp,
+ struct fib_info *fi)
{
- struct mlxsw_sp_nexthop *nh;
+ const struct fib_nh *nh = fib_info_nh(fi, 0);
- list_splice_init(&old_rif->nexthop_list, &new_rif->nexthop_list);
- list_for_each_entry(nh, &new_rif->nexthop_list, rif_list_node)
- nh->rif = new_rif;
- mlxsw_sp_nexthop_rif_update(mlxsw_sp, new_rif);
+ return nh->fib_nh_scope == RT_SCOPE_LINK ||
+ mlxsw_sp_nexthop4_ipip_type(mlxsw_sp, nh, NULL);
}
-static void mlxsw_sp_nexthop_rif_gone_sync(struct mlxsw_sp *mlxsw_sp,
- struct mlxsw_sp_rif *rif)
+static int
+mlxsw_sp_nexthop4_group_info_init(struct mlxsw_sp *mlxsw_sp,
+ struct mlxsw_sp_nexthop_group *nh_grp)
{
- struct mlxsw_sp_nexthop *nh, *tmp;
+ unsigned int nhs = fib_info_num_path(nh_grp->ipv4.fi);
+ struct mlxsw_sp_nexthop_group_info *nhgi;
+ struct mlxsw_sp_nexthop *nh;
+ int err, i;
- list_for_each_entry_safe(nh, tmp, &rif->nexthop_list, rif_list_node) {
- mlxsw_sp_nexthop_type_fini(mlxsw_sp, nh);
- mlxsw_sp_nexthop_group_refresh(mlxsw_sp, nh->nh_grp);
+ nhgi = kzalloc(struct_size(nhgi, nexthops, nhs), GFP_KERNEL);
+ if (!nhgi)
+ return -ENOMEM;
+ nh_grp->nhgi = nhgi;
+ nhgi->nh_grp = nh_grp;
+ nhgi->gateway = mlxsw_sp_fi_is_gateway(mlxsw_sp, nh_grp->ipv4.fi);
+ nhgi->count = nhs;
+ for (i = 0; i < nhgi->count; i++) {
+ struct fib_nh *fib_nh;
+
+ nh = &nhgi->nexthops[i];
+ fib_nh = fib_info_nh(nh_grp->ipv4.fi, i);
+ err = mlxsw_sp_nexthop4_init(mlxsw_sp, nh_grp, nh, fib_nh);
+ if (err)
+ goto err_nexthop4_init;
+ }
+ err = mlxsw_sp_nexthop_group_refresh(mlxsw_sp, nh_grp);
+ if (err)
+ goto err_group_refresh;
+
+ return 0;
+
+err_group_refresh:
+ i = nhgi->count;
+err_nexthop4_init:
+ for (i--; i >= 0; i--) {
+ nh = &nhgi->nexthops[i];
+ mlxsw_sp_nexthop4_fini(mlxsw_sp, nh);
}
+ kfree(nhgi);
+ return err;
}
-static bool mlxsw_sp_fi_is_gateway(const struct mlxsw_sp *mlxsw_sp,
- struct fib_info *fi)
+static void
+mlxsw_sp_nexthop4_group_info_fini(struct mlxsw_sp *mlxsw_sp,
+ struct mlxsw_sp_nexthop_group *nh_grp)
{
- const struct fib_nh *nh = fib_info_nh(fi, 0);
+ struct mlxsw_sp_nexthop_group_info *nhgi = nh_grp->nhgi;
+ int i;
- return nh->fib_nh_scope == RT_SCOPE_LINK ||
- mlxsw_sp_nexthop4_ipip_type(mlxsw_sp, nh, NULL);
+ for (i = nhgi->count - 1; i >= 0; i--) {
+ struct mlxsw_sp_nexthop *nh = &nhgi->nexthops[i];
+
+ mlxsw_sp_nexthop4_fini(mlxsw_sp, nh);
+ }
+ mlxsw_sp_nexthop_group_refresh(mlxsw_sp, nh_grp);
+ WARN_ON_ONCE(nhgi->adj_index_valid);
+ kfree(nhgi);
}
static struct mlxsw_sp_nexthop_group *
mlxsw_sp_nexthop4_group_create(struct mlxsw_sp *mlxsw_sp, struct fib_info *fi)
{
- unsigned int nhs = fib_info_num_path(fi);
struct mlxsw_sp_nexthop_group *nh_grp;
- struct mlxsw_sp_nexthop *nh;
- struct fib_nh *fib_nh;
- int i;
int err;
- nh_grp = kzalloc(struct_size(nh_grp, nexthops, nhs), GFP_KERNEL);
+ nh_grp = kzalloc(sizeof(*nh_grp), GFP_KERNEL);
if (!nh_grp)
return ERR_PTR(-ENOMEM);
- nh_grp->priv = fi;
+ INIT_LIST_HEAD(&nh_grp->vr_list);
+ err = rhashtable_init(&nh_grp->vr_ht,
+ &mlxsw_sp_nexthop_group_vr_ht_params);
+ if (err)
+ goto err_nexthop_group_vr_ht_init;
INIT_LIST_HEAD(&nh_grp->fib_list);
- nh_grp->neigh_tbl = &arp_tbl;
-
- nh_grp->gateway = mlxsw_sp_fi_is_gateway(mlxsw_sp, fi);
- nh_grp->count = nhs;
+ nh_grp->type = MLXSW_SP_NEXTHOP_GROUP_TYPE_IPV4;
+ nh_grp->ipv4.fi = fi;
fib_info_hold(fi);
- for (i = 0; i < nh_grp->count; i++) {
- nh = &nh_grp->nexthops[i];
- fib_nh = fib_info_nh(fi, i);
- err = mlxsw_sp_nexthop4_init(mlxsw_sp, nh_grp, nh, fib_nh);
- if (err)
- goto err_nexthop4_init;
- }
+
+ err = mlxsw_sp_nexthop4_group_info_init(mlxsw_sp, nh_grp);
+ if (err)
+ goto err_nexthop_group_info_init;
+
err = mlxsw_sp_nexthop_group_insert(mlxsw_sp, nh_grp);
if (err)
goto err_nexthop_group_insert;
- mlxsw_sp_nexthop_group_refresh(mlxsw_sp, nh_grp);
+
+ nh_grp->can_destroy = true;
+
return nh_grp;
err_nexthop_group_insert:
-err_nexthop4_init:
- for (i--; i >= 0; i--) {
- nh = &nh_grp->nexthops[i];
- mlxsw_sp_nexthop4_fini(mlxsw_sp, nh);
- }
+ mlxsw_sp_nexthop4_group_info_fini(mlxsw_sp, nh_grp);
+err_nexthop_group_info_init:
fib_info_put(fi);
+ rhashtable_destroy(&nh_grp->vr_ht);
+err_nexthop_group_vr_ht_init:
kfree(nh_grp);
return ERR_PTR(err);
}
mlxsw_sp_nexthop4_group_destroy(struct mlxsw_sp *mlxsw_sp,
struct mlxsw_sp_nexthop_group *nh_grp)
{
- struct mlxsw_sp_nexthop *nh;
- int i;
-
+ if (!nh_grp->can_destroy)
+ return;
mlxsw_sp_nexthop_group_remove(mlxsw_sp, nh_grp);
- for (i = 0; i < nh_grp->count; i++) {
- nh = &nh_grp->nexthops[i];
- mlxsw_sp_nexthop4_fini(mlxsw_sp, nh);
- }
- mlxsw_sp_nexthop_group_refresh(mlxsw_sp, nh_grp);
- WARN_ON_ONCE(nh_grp->adj_index_valid);
- fib_info_put(mlxsw_sp_nexthop4_group_fi(nh_grp));
+ mlxsw_sp_nexthop4_group_info_fini(mlxsw_sp, nh_grp);
+ fib_info_put(nh_grp->ipv4.fi);
+ WARN_ON_ONCE(!list_empty(&nh_grp->vr_list));
+ rhashtable_destroy(&nh_grp->vr_ht);
kfree(nh_grp);
}
{
struct mlxsw_sp_nexthop_group *nh_grp;
+ if (fi->nh) {
+ nh_grp = mlxsw_sp_nexthop_obj_group_lookup(mlxsw_sp,
+ fi->nh->id);
+ if (WARN_ON_ONCE(!nh_grp))
+ return -EINVAL;
+ goto out;
+ }
+
nh_grp = mlxsw_sp_nexthop4_group_lookup(mlxsw_sp, fi);
if (!nh_grp) {
nh_grp = mlxsw_sp_nexthop4_group_create(mlxsw_sp, fi);
if (IS_ERR(nh_grp))
return PTR_ERR(nh_grp);
}
+out:
list_add_tail(&fib_entry->nexthop_group_node, &nh_grp->fib_list);
fib_entry->nh_group = nh_grp;
return 0;
list_del(&fib_entry->nexthop_group_node);
if (!list_empty(&nh_grp->fib_list))
return;
+
+ if (nh_grp->type == MLXSW_SP_NEXTHOP_GROUP_TYPE_OBJ) {
+ mlxsw_sp_nexthop_obj_group_destroy(mlxsw_sp, nh_grp);
+ return;
+ }
+
mlxsw_sp_nexthop4_group_destroy(mlxsw_sp, nh_grp);
}
switch (fib_entry->type) {
case MLXSW_SP_FIB_ENTRY_TYPE_REMOTE:
- return !!nh_group->adj_index_valid;
+ return !!nh_group->nhgi->adj_index_valid;
case MLXSW_SP_FIB_ENTRY_TYPE_LOCAL:
- return !!nh_group->nh_rif;
+ return !!nh_group->nhgi->nh_rif;
case MLXSW_SP_FIB_ENTRY_TYPE_BLACKHOLE:
case MLXSW_SP_FIB_ENTRY_TYPE_IPIP_DECAP:
case MLXSW_SP_FIB_ENTRY_TYPE_NVE_DECAP:
{
int i;
- for (i = 0; i < nh_grp->count; i++) {
- struct mlxsw_sp_nexthop *nh = &nh_grp->nexthops[i];
+ for (i = 0; i < nh_grp->nhgi->count; i++) {
+ struct mlxsw_sp_nexthop *nh = &nh_grp->nhgi->nexthops[i];
struct fib6_info *rt = mlxsw_sp_rt6->rt;
if (nh->rif && nh->rif->dev == rt->fib6_nh->fib_nh_dev &&
mlxsw_sp_fib4_entry_hw_flags_set(struct mlxsw_sp *mlxsw_sp,
struct mlxsw_sp_fib_entry *fib_entry)
{
- struct fib_info *fi = mlxsw_sp_nexthop4_group_fi(fib_entry->nh_group);
u32 *p_dst = (u32 *) fib_entry->fib_node->key.addr;
int dst_len = fib_entry->fib_node->key.prefix_len;
struct mlxsw_sp_fib4_entry *fib4_entry;
should_offload = mlxsw_sp_fib_entry_should_offload(fib_entry);
fib4_entry = container_of(fib_entry, struct mlxsw_sp_fib4_entry,
common);
- fri.fi = fi;
+ fri.fi = fib4_entry->fi;
fri.tb_id = fib4_entry->tb_id;
fri.dst = cpu_to_be32(*p_dst);
fri.dst_len = dst_len;
mlxsw_sp_fib4_entry_hw_flags_clear(struct mlxsw_sp *mlxsw_sp,
struct mlxsw_sp_fib_entry *fib_entry)
{
- struct fib_info *fi = mlxsw_sp_nexthop4_group_fi(fib_entry->nh_group);
u32 *p_dst = (u32 *) fib_entry->fib_node->key.addr;
int dst_len = fib_entry->fib_node->key.prefix_len;
struct mlxsw_sp_fib4_entry *fib4_entry;
fib4_entry = container_of(fib_entry, struct mlxsw_sp_fib4_entry,
common);
- fri.fi = fi;
+ fri.fi = fib4_entry->fi;
fri.tb_id = fib4_entry->tb_id;
fri.dst = cpu_to_be32(*p_dst);
fri.dst_len = dst_len;
return err;
}
-static int mlxsw_sp_adj_discard_write(struct mlxsw_sp *mlxsw_sp, u16 rif_index)
+static int mlxsw_sp_adj_discard_write(struct mlxsw_sp *mlxsw_sp)
{
enum mlxsw_reg_ratr_trap_action trap_action;
char ratr_pl[MLXSW_REG_RATR_LEN];
if (err)
return err;
- trap_action = MLXSW_REG_RATR_TRAP_ACTION_DISCARD_ERRORS;
+ trap_action = MLXSW_REG_RATR_TRAP_ACTION_TRAP;
mlxsw_reg_ratr_pack(ratr_pl, MLXSW_REG_RATR_OP_WRITE_WRITE_ENTRY, true,
MLXSW_REG_RATR_TYPE_ETHERNET,
- mlxsw_sp->router->adj_discard_index, rif_index);
+ mlxsw_sp->router->adj_discard_index,
+ mlxsw_sp->router->lb_rif_index);
mlxsw_reg_ratr_trap_action_set(ratr_pl, trap_action);
+ mlxsw_reg_ratr_trap_id_set(ratr_pl, MLXSW_TRAP_ID_RTR_EGRESS0);
err = mlxsw_reg_write(mlxsw_sp->core, MLXSW_REG(ratr), ratr_pl);
if (err)
goto err_ratr_write;
{
const struct mlxsw_sp_router_ll_ops *ll_ops = fib_entry->fib_node->fib->ll_ops;
struct mlxsw_sp_nexthop_group *nh_group = fib_entry->nh_group;
+ struct mlxsw_sp_nexthop_group_info *nhgi = nh_group->nhgi;
enum mlxsw_reg_ralue_trap_action trap_action;
u16 trap_id = 0;
u32 adjacency_index = 0;
*/
if (mlxsw_sp_fib_entry_should_offload(fib_entry)) {
trap_action = MLXSW_REG_RALUE_TRAP_ACTION_NOP;
- adjacency_index = fib_entry->nh_group->adj_index;
- ecmp_size = fib_entry->nh_group->ecmp_size;
- } else if (!nh_group->adj_index_valid && nh_group->count &&
- nh_group->nh_rif) {
- err = mlxsw_sp_adj_discard_write(mlxsw_sp,
- nh_group->nh_rif->rif_index);
+ adjacency_index = nhgi->adj_index;
+ ecmp_size = nhgi->ecmp_size;
+ } else if (!nhgi->adj_index_valid && nhgi->count && nhgi->nh_rif) {
+ err = mlxsw_sp_adj_discard_write(mlxsw_sp);
if (err)
return err;
trap_action = MLXSW_REG_RALUE_TRAP_ACTION_NOP;
enum mlxsw_sp_fib_entry_op op)
{
const struct mlxsw_sp_router_ll_ops *ll_ops = fib_entry->fib_node->fib->ll_ops;
- struct mlxsw_sp_rif *rif = fib_entry->nh_group->nh_rif;
+ struct mlxsw_sp_rif *rif = fib_entry->nh_group->nhgi->nh_rif;
enum mlxsw_reg_ralue_trap_action trap_action;
u16 trap_id = 0;
u16 rif_index = 0;
const struct fib_entry_notifier_info *fen_info,
struct mlxsw_sp_fib_entry *fib_entry)
{
- struct net_device *dev = fib_info_nh(fen_info->fi, 0)->fib_nh_dev;
+ struct mlxsw_sp_nexthop_group_info *nhgi = fib_entry->nh_group->nhgi;
union mlxsw_sp_l3addr dip = { .addr4 = htonl(fen_info->dst) };
struct mlxsw_sp_router *router = mlxsw_sp->router;
u32 tb_id = mlxsw_sp_fix_tb_id(fen_info->tb_id);
+ int ifindex = nhgi->nexthops[0].ifindex;
struct mlxsw_sp_ipip_entry *ipip_entry;
- struct fib_info *fi = fen_info->fi;
switch (fen_info->type) {
case RTN_LOCAL:
- ipip_entry = mlxsw_sp_ipip_entry_find_by_decap(mlxsw_sp, dev,
- MLXSW_SP_L3_PROTO_IPV4, dip);
+ ipip_entry = mlxsw_sp_ipip_entry_find_by_decap(mlxsw_sp, ifindex,
+ MLXSW_SP_L3_PROTO_IPV4, dip);
if (ipip_entry && ipip_entry->ol_dev->flags & IFF_UP) {
fib_entry->type = MLXSW_SP_FIB_ENTRY_TYPE_IPIP_DECAP;
return mlxsw_sp_fib_entry_decap_init(mlxsw_sp,
fib_entry->type = MLXSW_SP_FIB_ENTRY_TYPE_UNREACHABLE;
return 0;
case RTN_UNICAST:
- if (mlxsw_sp_fi_is_gateway(mlxsw_sp, fi))
+ if (nhgi->gateway)
fib_entry->type = MLXSW_SP_FIB_ENTRY_TYPE_REMOTE;
else
fib_entry->type = MLXSW_SP_FIB_ENTRY_TYPE_LOCAL;
goto err_fib_entry_priv_create;
}
- err = mlxsw_sp_fib4_entry_type_set(mlxsw_sp, fen_info, fib_entry);
- if (err)
- goto err_fib4_entry_type_set;
-
err = mlxsw_sp_nexthop4_group_get(mlxsw_sp, fib_entry, fen_info->fi);
if (err)
goto err_nexthop4_group_get;
- fib4_entry->prio = fen_info->fi->fib_priority;
+ err = mlxsw_sp_nexthop_group_vr_link(fib_entry->nh_group,
+ fib_node->fib);
+ if (err)
+ goto err_nexthop_group_vr_link;
+
+ err = mlxsw_sp_fib4_entry_type_set(mlxsw_sp, fen_info, fib_entry);
+ if (err)
+ goto err_fib4_entry_type_set;
+
+ fib4_entry->fi = fen_info->fi;
+ fib_info_hold(fib4_entry->fi);
fib4_entry->tb_id = fen_info->tb_id;
fib4_entry->type = fen_info->type;
fib4_entry->tos = fen_info->tos;
return fib4_entry;
-err_nexthop4_group_get:
- mlxsw_sp_fib4_entry_type_unset(mlxsw_sp, fib_entry);
err_fib4_entry_type_set:
+ mlxsw_sp_nexthop_group_vr_unlink(fib_entry->nh_group, fib_node->fib);
+err_nexthop_group_vr_link:
+ mlxsw_sp_nexthop4_group_put(mlxsw_sp, &fib4_entry->common);
+err_nexthop4_group_get:
mlxsw_sp_fib_entry_priv_put(fib_entry->priv);
err_fib_entry_priv_create:
kfree(fib4_entry);
static void mlxsw_sp_fib4_entry_destroy(struct mlxsw_sp *mlxsw_sp,
struct mlxsw_sp_fib4_entry *fib4_entry)
{
- mlxsw_sp_nexthop4_group_put(mlxsw_sp, &fib4_entry->common);
+ struct mlxsw_sp_fib_node *fib_node = fib4_entry->common.fib_node;
+
+ fib_info_put(fib4_entry->fi);
mlxsw_sp_fib4_entry_type_unset(mlxsw_sp, &fib4_entry->common);
+ mlxsw_sp_nexthop_group_vr_unlink(fib4_entry->common.nh_group,
+ fib_node->fib);
+ mlxsw_sp_nexthop4_group_put(mlxsw_sp, &fib4_entry->common);
mlxsw_sp_fib_entry_priv_put(fib4_entry->common.priv);
kfree(fib4_entry);
}
if (fib4_entry->tb_id == fen_info->tb_id &&
fib4_entry->tos == fen_info->tos &&
fib4_entry->type == fen_info->type &&
- mlxsw_sp_nexthop4_group_fi(fib4_entry->common.nh_group) ==
- fen_info->fi)
+ fib4_entry->fi == fen_info->fi)
return fib4_entry;
return NULL;
{
struct fib6_nh *fib6_nh = mlxsw_sp_rt6->rt->fib6_nh;
- fib6_nh->fib_nh_flags &= ~RTNH_F_OFFLOAD;
+ if (!mlxsw_sp_rt6->rt->nh)
+ fib6_nh->fib_nh_flags &= ~RTNH_F_OFFLOAD;
mlxsw_sp_rt6_release(mlxsw_sp_rt6->rt);
kfree(mlxsw_sp_rt6);
}
mlxsw_sp_netdev_ipip_type(mlxsw_sp, rt->fib6_nh->fib_nh_dev, ret);
}
-static int mlxsw_sp_nexthop6_type_init(struct mlxsw_sp *mlxsw_sp,
- struct mlxsw_sp_nexthop_group *nh_grp,
- struct mlxsw_sp_nexthop *nh,
- const struct fib6_info *rt)
-{
- const struct mlxsw_sp_ipip_ops *ipip_ops;
- struct mlxsw_sp_ipip_entry *ipip_entry;
- struct net_device *dev = rt->fib6_nh->fib_nh_dev;
- struct mlxsw_sp_rif *rif;
- int err;
-
- ipip_entry = mlxsw_sp_ipip_entry_find_by_ol_dev(mlxsw_sp, dev);
- if (ipip_entry) {
- ipip_ops = mlxsw_sp->router->ipip_ops_arr[ipip_entry->ipipt];
- if (ipip_ops->can_offload(mlxsw_sp, dev,
- MLXSW_SP_L3_PROTO_IPV6)) {
- nh->type = MLXSW_SP_NEXTHOP_TYPE_IPIP;
- mlxsw_sp_nexthop_ipip_init(mlxsw_sp, nh, ipip_entry);
- return 0;
- }
- }
-
- nh->type = MLXSW_SP_NEXTHOP_TYPE_ETH;
- rif = mlxsw_sp_rif_find_by_dev(mlxsw_sp, dev);
- if (!rif)
- return 0;
- mlxsw_sp_nexthop_rif_init(nh, rif);
-
- err = mlxsw_sp_nexthop_neigh_init(mlxsw_sp, nh);
- if (err)
- goto err_nexthop_neigh_init;
-
- return 0;
-
-err_nexthop_neigh_init:
- mlxsw_sp_nexthop_rif_fini(nh);
- return err;
-}
-
-static void mlxsw_sp_nexthop6_type_fini(struct mlxsw_sp *mlxsw_sp,
- struct mlxsw_sp_nexthop *nh)
-{
- mlxsw_sp_nexthop_type_fini(mlxsw_sp, nh);
-}
-
static int mlxsw_sp_nexthop6_init(struct mlxsw_sp *mlxsw_sp,
struct mlxsw_sp_nexthop_group *nh_grp,
struct mlxsw_sp_nexthop *nh,
{
struct net_device *dev = rt->fib6_nh->fib_nh_dev;
- nh->nh_grp = nh_grp;
+ nh->nhgi = nh_grp->nhgi;
nh->nh_weight = rt->fib6_nh->fib_nh_weight;
memcpy(&nh->gw_addr, &rt->fib6_nh->fib_nh_gw6, sizeof(nh->gw_addr));
+#if IS_ENABLED(CONFIG_IPV6)
+ nh->neigh_tbl = &nd_tbl;
+#endif
mlxsw_sp_nexthop_counter_alloc(mlxsw_sp, nh);
list_add_tail(&nh->router_list_node, &mlxsw_sp->router->nexthop_list);
return 0;
nh->ifindex = dev->ifindex;
- return mlxsw_sp_nexthop6_type_init(mlxsw_sp, nh_grp, nh, rt);
+ return mlxsw_sp_nexthop_type_init(mlxsw_sp, nh, dev);
}
static void mlxsw_sp_nexthop6_fini(struct mlxsw_sp *mlxsw_sp,
struct mlxsw_sp_nexthop *nh)
{
- mlxsw_sp_nexthop6_type_fini(mlxsw_sp, nh);
+ mlxsw_sp_nexthop_type_fini(mlxsw_sp, nh);
list_del(&nh->router_list_node);
mlxsw_sp_nexthop_counter_free(mlxsw_sp, nh);
}
mlxsw_sp_nexthop6_ipip_type(mlxsw_sp, rt, NULL);
}
-static struct mlxsw_sp_nexthop_group *
-mlxsw_sp_nexthop6_group_create(struct mlxsw_sp *mlxsw_sp,
- struct mlxsw_sp_fib6_entry *fib6_entry)
+static int
+mlxsw_sp_nexthop6_group_info_init(struct mlxsw_sp *mlxsw_sp,
+ struct mlxsw_sp_nexthop_group *nh_grp,
+ struct mlxsw_sp_fib6_entry *fib6_entry)
{
- struct mlxsw_sp_nexthop_group *nh_grp;
+ struct mlxsw_sp_nexthop_group_info *nhgi;
struct mlxsw_sp_rt6 *mlxsw_sp_rt6;
struct mlxsw_sp_nexthop *nh;
- int i = 0;
- int err;
+ int err, i;
- nh_grp = kzalloc(struct_size(nh_grp, nexthops, fib6_entry->nrt6),
- GFP_KERNEL);
- if (!nh_grp)
- return ERR_PTR(-ENOMEM);
- INIT_LIST_HEAD(&nh_grp->fib_list);
-#if IS_ENABLED(CONFIG_IPV6)
- nh_grp->neigh_tbl = &nd_tbl;
-#endif
+ nhgi = kzalloc(struct_size(nhgi, nexthops, fib6_entry->nrt6),
+ GFP_KERNEL);
+ if (!nhgi)
+ return -ENOMEM;
+ nh_grp->nhgi = nhgi;
+ nhgi->nh_grp = nh_grp;
mlxsw_sp_rt6 = list_first_entry(&fib6_entry->rt6_list,
struct mlxsw_sp_rt6, list);
- nh_grp->gateway = mlxsw_sp_rt6_is_gateway(mlxsw_sp, mlxsw_sp_rt6->rt);
- nh_grp->count = fib6_entry->nrt6;
- for (i = 0; i < nh_grp->count; i++) {
+ nhgi->gateway = mlxsw_sp_rt6_is_gateway(mlxsw_sp, mlxsw_sp_rt6->rt);
+ nhgi->count = fib6_entry->nrt6;
+ for (i = 0; i < nhgi->count; i++) {
struct fib6_info *rt = mlxsw_sp_rt6->rt;
- nh = &nh_grp->nexthops[i];
+ nh = &nhgi->nexthops[i];
err = mlxsw_sp_nexthop6_init(mlxsw_sp, nh_grp, nh, rt);
if (err)
goto err_nexthop6_init;
mlxsw_sp_rt6 = list_next_entry(mlxsw_sp_rt6, list);
}
+ nh_grp->nhgi = nhgi;
+ err = mlxsw_sp_nexthop_group_refresh(mlxsw_sp, nh_grp);
+ if (err)
+ goto err_group_refresh;
+
+ return 0;
+
+err_group_refresh:
+ i = nhgi->count;
+err_nexthop6_init:
+ for (i--; i >= 0; i--) {
+ nh = &nhgi->nexthops[i];
+ mlxsw_sp_nexthop6_fini(mlxsw_sp, nh);
+ }
+ kfree(nhgi);
+ return err;
+}
+
+static void
+mlxsw_sp_nexthop6_group_info_fini(struct mlxsw_sp *mlxsw_sp,
+ struct mlxsw_sp_nexthop_group *nh_grp)
+{
+ struct mlxsw_sp_nexthop_group_info *nhgi = nh_grp->nhgi;
+ int i;
+
+ for (i = nhgi->count - 1; i >= 0; i--) {
+ struct mlxsw_sp_nexthop *nh = &nhgi->nexthops[i];
+
+ mlxsw_sp_nexthop6_fini(mlxsw_sp, nh);
+ }
+ mlxsw_sp_nexthop_group_refresh(mlxsw_sp, nh_grp);
+ WARN_ON_ONCE(nhgi->adj_index_valid);
+ kfree(nhgi);
+}
+
+static struct mlxsw_sp_nexthop_group *
+mlxsw_sp_nexthop6_group_create(struct mlxsw_sp *mlxsw_sp,
+ struct mlxsw_sp_fib6_entry *fib6_entry)
+{
+ struct mlxsw_sp_nexthop_group *nh_grp;
+ int err;
+
+ nh_grp = kzalloc(sizeof(*nh_grp), GFP_KERNEL);
+ if (!nh_grp)
+ return ERR_PTR(-ENOMEM);
+ INIT_LIST_HEAD(&nh_grp->vr_list);
+ err = rhashtable_init(&nh_grp->vr_ht,
+ &mlxsw_sp_nexthop_group_vr_ht_params);
+ if (err)
+ goto err_nexthop_group_vr_ht_init;
+ INIT_LIST_HEAD(&nh_grp->fib_list);
+ nh_grp->type = MLXSW_SP_NEXTHOP_GROUP_TYPE_IPV6;
+
+ err = mlxsw_sp_nexthop6_group_info_init(mlxsw_sp, nh_grp, fib6_entry);
+ if (err)
+ goto err_nexthop_group_info_init;
err = mlxsw_sp_nexthop_group_insert(mlxsw_sp, nh_grp);
if (err)
goto err_nexthop_group_insert;
- mlxsw_sp_nexthop_group_refresh(mlxsw_sp, nh_grp);
+ nh_grp->can_destroy = true;
+
return nh_grp;
err_nexthop_group_insert:
-err_nexthop6_init:
- for (i--; i >= 0; i--) {
- nh = &nh_grp->nexthops[i];
- mlxsw_sp_nexthop6_fini(mlxsw_sp, nh);
- }
+ mlxsw_sp_nexthop6_group_info_fini(mlxsw_sp, nh_grp);
+err_nexthop_group_info_init:
+ rhashtable_destroy(&nh_grp->vr_ht);
+err_nexthop_group_vr_ht_init:
kfree(nh_grp);
return ERR_PTR(err);
}
mlxsw_sp_nexthop6_group_destroy(struct mlxsw_sp *mlxsw_sp,
struct mlxsw_sp_nexthop_group *nh_grp)
{
- struct mlxsw_sp_nexthop *nh;
- int i = nh_grp->count;
-
+ if (!nh_grp->can_destroy)
+ return;
mlxsw_sp_nexthop_group_remove(mlxsw_sp, nh_grp);
- for (i--; i >= 0; i--) {
- nh = &nh_grp->nexthops[i];
- mlxsw_sp_nexthop6_fini(mlxsw_sp, nh);
- }
- mlxsw_sp_nexthop_group_refresh(mlxsw_sp, nh_grp);
- WARN_ON(nh_grp->adj_index_valid);
+ mlxsw_sp_nexthop6_group_info_fini(mlxsw_sp, nh_grp);
+ WARN_ON_ONCE(!list_empty(&nh_grp->vr_list));
+ rhashtable_destroy(&nh_grp->vr_ht);
kfree(nh_grp);
}
static int mlxsw_sp_nexthop6_group_get(struct mlxsw_sp *mlxsw_sp,
struct mlxsw_sp_fib6_entry *fib6_entry)
{
+ struct fib6_info *rt = mlxsw_sp_fib6_entry_rt(fib6_entry);
struct mlxsw_sp_nexthop_group *nh_grp;
+ if (rt->nh) {
+ nh_grp = mlxsw_sp_nexthop_obj_group_lookup(mlxsw_sp,
+ rt->nh->id);
+ if (WARN_ON_ONCE(!nh_grp))
+ return -EINVAL;
+ goto out;
+ }
+
nh_grp = mlxsw_sp_nexthop6_group_lookup(mlxsw_sp, fib6_entry);
if (!nh_grp) {
nh_grp = mlxsw_sp_nexthop6_group_create(mlxsw_sp, fib6_entry);
return PTR_ERR(nh_grp);
}
- list_add_tail(&fib6_entry->common.nexthop_group_node,
- &nh_grp->fib_list);
- fib6_entry->common.nh_group = nh_grp;
-
/* The route and the nexthop are described by the same struct, so we
* need to the update the nexthop offload indication for the new route.
*/
__mlxsw_sp_nexthop6_group_offload_refresh(nh_grp, fib6_entry);
+out:
+ list_add_tail(&fib6_entry->common.nexthop_group_node,
+ &nh_grp->fib_list);
+ fib6_entry->common.nh_group = nh_grp;
+
return 0;
}
list_del(&fib_entry->nexthop_group_node);
if (!list_empty(&nh_grp->fib_list))
return;
+
+ if (nh_grp->type == MLXSW_SP_NEXTHOP_GROUP_TYPE_OBJ) {
+ mlxsw_sp_nexthop_obj_group_destroy(mlxsw_sp, nh_grp);
+ return;
+ }
+
mlxsw_sp_nexthop6_group_destroy(mlxsw_sp, nh_grp);
}
struct mlxsw_sp_fib6_entry *fib6_entry)
{
struct mlxsw_sp_nexthop_group *old_nh_grp = fib6_entry->common.nh_group;
+ struct mlxsw_sp_fib_node *fib_node = fib6_entry->common.fib_node;
int err;
+ mlxsw_sp_nexthop_group_vr_unlink(old_nh_grp, fib_node->fib);
fib6_entry->common.nh_group = NULL;
list_del(&fib6_entry->common.nexthop_group_node);
if (err)
goto err_nexthop6_group_get;
+ err = mlxsw_sp_nexthop_group_vr_link(fib6_entry->common.nh_group,
+ fib_node->fib);
+ if (err)
+ goto err_nexthop_group_vr_link;
+
/* In case this entry is offloaded, then the adjacency index
* currently associated with it in the device's table is that
* of the old group. Start using the new one instead.
return 0;
err_fib_entry_update:
+ mlxsw_sp_nexthop_group_vr_unlink(fib6_entry->common.nh_group,
+ fib_node->fib);
+err_nexthop_group_vr_link:
mlxsw_sp_nexthop6_group_put(mlxsw_sp, &fib6_entry->common);
err_nexthop6_group_get:
list_add_tail(&fib6_entry->common.nexthop_group_node,
&old_nh_grp->fib_list);
fib6_entry->common.nh_group = old_nh_grp;
+ mlxsw_sp_nexthop_group_vr_link(old_nh_grp, fib_node->fib);
return err;
}
struct mlxsw_sp_fib_entry *fib_entry,
const struct fib6_info *rt)
{
- /* Packets hitting RTF_REJECT routes need to be discarded by the
- * stack. We can rely on their destination device not having a
- * RIF (it's the loopback device) and can thus use action type
- * local, which will cause them to be trapped with a lower
- * priority than packets that need to be locally received.
- */
if (rt->fib6_flags & (RTF_LOCAL | RTF_ANYCAST))
fib_entry->type = MLXSW_SP_FIB_ENTRY_TYPE_TRAP;
else if (rt->fib6_type == RTN_BLACKHOLE)
fib_entry->type = MLXSW_SP_FIB_ENTRY_TYPE_BLACKHOLE;
else if (rt->fib6_flags & RTF_REJECT)
fib_entry->type = MLXSW_SP_FIB_ENTRY_TYPE_UNREACHABLE;
- else if (mlxsw_sp_rt6_is_gateway(mlxsw_sp, rt))
+ else if (fib_entry->nh_group->nhgi->gateway)
fib_entry->type = MLXSW_SP_FIB_ENTRY_TYPE_REMOTE;
else
fib_entry->type = MLXSW_SP_FIB_ENTRY_TYPE_LOCAL;
fib6_entry->nrt6++;
}
- mlxsw_sp_fib6_entry_type_set(mlxsw_sp, fib_entry, rt_arr[0]);
-
err = mlxsw_sp_nexthop6_group_get(mlxsw_sp, fib6_entry);
if (err)
goto err_nexthop6_group_get;
+ err = mlxsw_sp_nexthop_group_vr_link(fib_entry->nh_group,
+ fib_node->fib);
+ if (err)
+ goto err_nexthop_group_vr_link;
+
+ mlxsw_sp_fib6_entry_type_set(mlxsw_sp, fib_entry, rt_arr[0]);
+
fib_entry->fib_node = fib_node;
return fib6_entry;
+err_nexthop_group_vr_link:
+ mlxsw_sp_nexthop6_group_put(mlxsw_sp, fib_entry);
err_nexthop6_group_get:
i = nrt6;
err_rt6_create:
static void mlxsw_sp_fib6_entry_destroy(struct mlxsw_sp *mlxsw_sp,
struct mlxsw_sp_fib6_entry *fib6_entry)
{
+ struct mlxsw_sp_fib_node *fib_node = fib6_entry->common.fib_node;
+
+ mlxsw_sp_nexthop_group_vr_unlink(fib6_entry->common.nh_group,
+ fib_node->fib);
mlxsw_sp_nexthop6_group_put(mlxsw_sp, &fib6_entry->common);
mlxsw_sp_fib6_entry_rt_destroy_all(fib6_entry);
WARN_ON(fib6_entry->nrt6);
NL_SET_ERR_MSG_MOD(info->extack, "IPv6 gateway with IPv4 route is not supported");
return notifier_from_errno(-EINVAL);
}
- if (fen_info->fi->nh) {
- NL_SET_ERR_MSG_MOD(info->extack, "IPv4 route with nexthop objects is not supported");
- return notifier_from_errno(-EINVAL);
- }
- } else if (info->family == AF_INET6) {
- struct fib6_entry_notifier_info *fen6_info;
-
- fen6_info = container_of(info,
- struct fib6_entry_notifier_info,
- info);
- if (fen6_info->rt->nh) {
- NL_SET_ERR_MSG_MOD(info->extack, "IPv6 route with nexthop objects is not supported");
- return notifier_from_errno(-EINVAL);
- }
}
break;
}
switch (event) {
case NETDEV_UP:
+ if (netif_is_bridge_master(l3_dev) && br_vlan_enabled(l3_dev)) {
+ u16 proto;
+
+ br_vlan_get_proto(l3_dev, &proto);
+ if (proto == ETH_P_8021AD) {
+ NL_SET_ERR_MSG_MOD(extack, "Adding an IP address to 802.1ad bridge is not supported");
+ return -EOPNOTSUPP;
+ }
+ }
rif = mlxsw_sp_rif_create(mlxsw_sp, ¶ms, extack);
if (IS_ERR(rif))
return PTR_ERR(rif);
kfree(router->ll_op_ctx);
}
+static int mlxsw_sp_lb_rif_init(struct mlxsw_sp *mlxsw_sp)
+{
+ u16 lb_rif_index;
+ int err;
+
+ /* Create a generic loopback RIF associated with the main table
+ * (default VRF). Any table can be used, but the main table exists
+ * anyway, so we do not waste resources.
+ */
+ err = mlxsw_sp_router_ul_rif_get(mlxsw_sp, RT_TABLE_MAIN,
+ &lb_rif_index);
+ if (err)
+ return err;
+
+ mlxsw_sp->router->lb_rif_index = lb_rif_index;
+
+ return 0;
+}
+
+static void mlxsw_sp_lb_rif_fini(struct mlxsw_sp *mlxsw_sp)
+{
+ mlxsw_sp_router_ul_rif_put(mlxsw_sp, mlxsw_sp->router->lb_rif_index);
+}
+
int mlxsw_sp_router_init(struct mlxsw_sp *mlxsw_sp,
struct netlink_ext_ack *extack)
{
if (err)
goto err_vrs_init;
+ err = mlxsw_sp_lb_rif_init(mlxsw_sp);
+ if (err)
+ goto err_lb_rif_init;
+
err = mlxsw_sp_neigh_init(mlxsw_sp);
if (err)
goto err_neigh_init;
if (err)
goto err_register_netevent_notifier;
+ mlxsw_sp->router->nexthop_nb.notifier_call =
+ mlxsw_sp_nexthop_obj_event;
+ err = register_nexthop_notifier(mlxsw_sp_net(mlxsw_sp),
+ &mlxsw_sp->router->nexthop_nb,
+ extack);
+ if (err)
+ goto err_register_nexthop_notifier;
+
mlxsw_sp->router->fib_nb.notifier_call = mlxsw_sp_router_fib_event;
err = register_fib_notifier(mlxsw_sp_net(mlxsw_sp),
&mlxsw_sp->router->fib_nb,
return 0;
err_register_fib_notifier:
+ unregister_nexthop_notifier(mlxsw_sp_net(mlxsw_sp),
+ &mlxsw_sp->router->nexthop_nb);
+err_register_nexthop_notifier:
unregister_netevent_notifier(&mlxsw_sp->router->netevent_nb);
err_register_netevent_notifier:
unregister_inet6addr_notifier(&router->inet6addr_nb);
err_mp_hash_init:
mlxsw_sp_neigh_fini(mlxsw_sp);
err_neigh_init:
+ mlxsw_sp_lb_rif_fini(mlxsw_sp);
+err_lb_rif_init:
mlxsw_sp_vrs_fini(mlxsw_sp);
err_vrs_init:
mlxsw_sp_mr_fini(mlxsw_sp);
{
unregister_fib_notifier(mlxsw_sp_net(mlxsw_sp),
&mlxsw_sp->router->fib_nb);
+ unregister_nexthop_notifier(mlxsw_sp_net(mlxsw_sp),
+ &mlxsw_sp->router->nexthop_nb);
unregister_netevent_notifier(&mlxsw_sp->router->netevent_nb);
unregister_inet6addr_notifier(&mlxsw_sp->router->inet6addr_nb);
unregister_inetaddr_notifier(&mlxsw_sp->router->inetaddr_nb);
mlxsw_core_flush_owq();
WARN_ON(!list_empty(&mlxsw_sp->router->fib_event_queue));
mlxsw_sp_neigh_fini(mlxsw_sp);
+ mlxsw_sp_lb_rif_fini(mlxsw_sp);
mlxsw_sp_vrs_fini(mlxsw_sp);
mlxsw_sp_mr_fini(mlxsw_sp);
mlxsw_sp_lpm_fini(mlxsw_sp);
struct list_head nexthop_neighs_list;
struct list_head ipip_list;
bool aborted;
+ struct notifier_block nexthop_nb;
struct notifier_block fib_nb;
struct notifier_block netevent_nb;
struct notifier_block inetaddr_nb;
/* One set of ops for each protocol: IPv4 and IPv6 */
const struct mlxsw_sp_router_ll_ops *proto_ll_ops[MLXSW_SP_L3_PROTO_MAX];
struct mlxsw_sp_fib_entry_op_ctx *ll_op_ctx;
+ u16 lb_rif_index;
};
struct mlxsw_sp_fib_entry_priv {
u32 *p_adj_size, u32 *p_adj_hash_index);
struct mlxsw_sp_rif *mlxsw_sp_nexthop_rif(struct mlxsw_sp_nexthop *nh);
bool mlxsw_sp_nexthop_group_has_ipip(struct mlxsw_sp_nexthop *nh);
+bool mlxsw_sp_nexthop_is_discard(const struct mlxsw_sp_nexthop *nh);
#define mlxsw_sp_nexthop_for_each(nh, router) \
for (nh = mlxsw_sp_nexthop_next(router, NULL); nh; \
nh = mlxsw_sp_nexthop_next(router, nh))
DECLARE_BITMAP(mids_bitmap, MLXSW_SP_MID_MAX);
const struct mlxsw_sp_bridge_ops *bridge_8021q_ops;
const struct mlxsw_sp_bridge_ops *bridge_8021d_ops;
+ const struct mlxsw_sp_bridge_ops *bridge_8021ad_ops;
};
struct mlxsw_sp_bridge_device {
bridge_device->mrouter = br_multicast_router(br_dev);
INIT_LIST_HEAD(&bridge_device->ports_list);
if (vlan_enabled) {
+ u16 proto;
+
bridge->vlan_enabled_exists = true;
- bridge_device->ops = bridge->bridge_8021q_ops;
+ br_vlan_get_proto(br_dev, &proto);
+ if (proto == ETH_P_8021AD)
+ bridge_device->ops = bridge->bridge_8021ad_ops;
+ else
+ bridge_device->ops = bridge->bridge_8021q_ops;
} else {
bridge_device->ops = bridge->bridge_8021d_ops;
}
return -EINVAL;
}
+static int mlxsw_sp_port_attr_br_vlan_proto_set(struct mlxsw_sp_port *mlxsw_sp_port,
+ struct switchdev_trans *trans,
+ struct net_device *orig_dev,
+ u16 vlan_proto)
+{
+ struct mlxsw_sp *mlxsw_sp = mlxsw_sp_port->mlxsw_sp;
+ struct mlxsw_sp_bridge_device *bridge_device;
+
+ if (!switchdev_trans_ph_prepare(trans))
+ return 0;
+
+ bridge_device = mlxsw_sp_bridge_device_find(mlxsw_sp->bridge, orig_dev);
+ if (WARN_ON(!bridge_device))
+ return -EINVAL;
+
+ netdev_err(bridge_device->dev, "VLAN protocol can't be changed on existing bridge\n");
+ return -EINVAL;
+}
+
static int mlxsw_sp_port_attr_mrouter_set(struct mlxsw_sp_port *mlxsw_sp_port,
struct switchdev_trans *trans,
struct net_device *orig_dev,
attr->orig_dev,
attr->u.vlan_filtering);
break;
+ case SWITCHDEV_ATTR_ID_BRIDGE_VLAN_PROTOCOL:
+ err = mlxsw_sp_port_attr_br_vlan_proto_set(mlxsw_sp_port, trans,
+ attr->orig_dev,
+ attr->u.vlan_protocol);
+ break;
case SWITCHDEV_ATTR_ID_PORT_MROUTER:
err = mlxsw_sp_port_attr_mrouter_set(mlxsw_sp_port, trans,
attr->orig_dev,
u16 pvid = mlxsw_sp_port_pvid_determine(mlxsw_sp_port, vid, is_pvid);
struct mlxsw_sp_port_vlan *mlxsw_sp_port_vlan;
u16 old_pvid = mlxsw_sp_port->pvid;
+ u16 proto;
int err;
/* The only valid scenario in which a port-vlan already exists, is if
if (err)
goto err_port_vlan_set;
- err = mlxsw_sp_port_pvid_set(mlxsw_sp_port, pvid);
+ br_vlan_get_proto(bridge_port->bridge_device->dev, &proto);
+ err = mlxsw_sp_port_pvid_set(mlxsw_sp_port, pvid, proto);
if (err)
goto err_port_pvid_set;
return 0;
err_port_vlan_bridge_join:
- mlxsw_sp_port_pvid_set(mlxsw_sp_port, old_pvid);
+ mlxsw_sp_port_pvid_set(mlxsw_sp_port, old_pvid, proto);
err_port_pvid_set:
mlxsw_sp_port_vlan_set(mlxsw_sp_port, vid, vid, false, false);
err_port_vlan_set:
{
u16 pvid = mlxsw_sp_port->pvid == vid ? 0 : mlxsw_sp_port->pvid;
struct mlxsw_sp_port_vlan *mlxsw_sp_port_vlan;
+ u16 proto;
mlxsw_sp_port_vlan = mlxsw_sp_port_vlan_find_by_vid(mlxsw_sp_port, vid);
if (WARN_ON(!mlxsw_sp_port_vlan))
return;
mlxsw_sp_port_vlan_bridge_leave(mlxsw_sp_port_vlan);
- mlxsw_sp_port_pvid_set(mlxsw_sp_port, pvid);
+ br_vlan_get_proto(bridge_port->bridge_device->dev, &proto);
+ mlxsw_sp_port_pvid_set(mlxsw_sp_port, pvid, proto);
mlxsw_sp_port_vlan_set(mlxsw_sp_port, vid, vid, false, false);
mlxsw_sp_port_vlan_destroy(mlxsw_sp_port_vlan);
}
}
static int
-mlxsw_sp_bridge_8021q_port_join(struct mlxsw_sp_bridge_device *bridge_device,
- struct mlxsw_sp_bridge_port *bridge_port,
- struct mlxsw_sp_port *mlxsw_sp_port,
- struct netlink_ext_ack *extack)
+mlxsw_sp_bridge_vlan_aware_port_join(struct mlxsw_sp_bridge_port *bridge_port,
+ struct mlxsw_sp_port *mlxsw_sp_port,
+ struct netlink_ext_ack *extack)
{
if (is_vlan_dev(bridge_port->dev)) {
NL_SET_ERR_MSG_MOD(extack, "Can not enslave a VLAN device to a VLAN-aware bridge");
return 0;
}
+static int
+mlxsw_sp_bridge_8021q_port_join(struct mlxsw_sp_bridge_device *bridge_device,
+ struct mlxsw_sp_bridge_port *bridge_port,
+ struct mlxsw_sp_port *mlxsw_sp_port,
+ struct netlink_ext_ack *extack)
+{
+ return mlxsw_sp_bridge_vlan_aware_port_join(bridge_port, mlxsw_sp_port,
+ extack);
+}
+
+static void
+mlxsw_sp_bridge_vlan_aware_port_leave(struct mlxsw_sp_port *mlxsw_sp_port)
+{
+ /* Make sure untagged frames are allowed to ingress */
+ mlxsw_sp_port_pvid_set(mlxsw_sp_port, MLXSW_SP_DEFAULT_VID,
+ ETH_P_8021Q);
+}
+
static void
mlxsw_sp_bridge_8021q_port_leave(struct mlxsw_sp_bridge_device *bridge_device,
struct mlxsw_sp_bridge_port *bridge_port,
struct mlxsw_sp_port *mlxsw_sp_port)
{
- /* Make sure untagged frames are allowed to ingress */
- mlxsw_sp_port_pvid_set(mlxsw_sp_port, MLXSW_SP_DEFAULT_VID);
+ mlxsw_sp_bridge_vlan_aware_port_leave(mlxsw_sp_port);
}
static int
.fid_vid = mlxsw_sp_bridge_8021d_fid_vid,
};
+static int
+mlxsw_sp_bridge_8021ad_port_join(struct mlxsw_sp_bridge_device *bridge_device,
+ struct mlxsw_sp_bridge_port *bridge_port,
+ struct mlxsw_sp_port *mlxsw_sp_port,
+ struct netlink_ext_ack *extack)
+{
+ int err;
+
+ err = mlxsw_sp_port_vlan_classification_set(mlxsw_sp_port, true, false);
+ if (err)
+ return err;
+
+ err = mlxsw_sp_bridge_vlan_aware_port_join(bridge_port, mlxsw_sp_port,
+ extack);
+ if (err)
+ goto err_bridge_vlan_aware_port_join;
+
+ return 0;
+
+err_bridge_vlan_aware_port_join:
+ mlxsw_sp_port_vlan_classification_set(mlxsw_sp_port, false, true);
+ return err;
+}
+
+static void
+mlxsw_sp_bridge_8021ad_port_leave(struct mlxsw_sp_bridge_device *bridge_device,
+ struct mlxsw_sp_bridge_port *bridge_port,
+ struct mlxsw_sp_port *mlxsw_sp_port)
+{
+ mlxsw_sp_bridge_vlan_aware_port_leave(mlxsw_sp_port);
+ mlxsw_sp_port_vlan_classification_set(mlxsw_sp_port, false, true);
+}
+
+static int
+mlxsw_sp_bridge_8021ad_vxlan_join(struct mlxsw_sp_bridge_device *bridge_device,
+ const struct net_device *vxlan_dev, u16 vid,
+ struct netlink_ext_ack *extack)
+{
+ NL_SET_ERR_MSG_MOD(extack, "VXLAN is not supported with 802.1ad");
+ return -EOPNOTSUPP;
+}
+
+static const struct mlxsw_sp_bridge_ops mlxsw_sp_bridge_8021ad_ops = {
+ .port_join = mlxsw_sp_bridge_8021ad_port_join,
+ .port_leave = mlxsw_sp_bridge_8021ad_port_leave,
+ .vxlan_join = mlxsw_sp_bridge_8021ad_vxlan_join,
+ .fid_get = mlxsw_sp_bridge_8021q_fid_get,
+ .fid_lookup = mlxsw_sp_bridge_8021q_fid_lookup,
+ .fid_vid = mlxsw_sp_bridge_8021q_fid_vid,
+};
+
int mlxsw_sp_port_bridge_join(struct mlxsw_sp_port *mlxsw_sp_port,
struct net_device *brport_dev,
struct net_device *br_dev,
bridge->bridge_8021q_ops = &mlxsw_sp_bridge_8021q_ops;
bridge->bridge_8021d_ops = &mlxsw_sp_bridge_8021d_ops;
+ bridge->bridge_8021ad_ops = &mlxsw_sp_bridge_8021ad_ops;
return mlxsw_sp_fdb_init(mlxsw_sp);
}
TRAP_TO_CPU),
MLXSW_SP_RXL_EXCEPTION(HOST_MISS_IPV6, L3_EXCEPTIONS,
TRAP_TO_CPU),
- MLXSW_SP_RXL_EXCEPTION(DISCARD_ROUTER3, L3_EXCEPTIONS,
+ MLXSW_SP_RXL_EXCEPTION(RTR_EGRESS0, L3_EXCEPTIONS,
TRAP_EXCEPTION_TO_CPU),
},
},
false),
},
},
+ {
+ .trap = MLXSW_SP_TRAP_DROP(BLACKHOLE_NEXTHOP, L3_DROPS),
+ .listeners_arr = {
+ MLXSW_SP_RXL_DISCARD(ROUTER3, L3_DISCARDS),
+ },
+ },
};
static struct mlxsw_sp_trap_policer_item *
#include <linux/types.h>
#include <linux/pci.h>
#include <linux/netdevice.h>
+#include <linux/ethtool.h>
#include <linux/etherdevice.h>
#include <linux/slab.h>
#include <linux/device.h>
MLXSW_TRAP_ID_RTR_INGRESS1 = 0x71,
MLXSW_TRAP_ID_IPV6_PIM = 0x79,
MLXSW_TRAP_ID_IPV6_VRRP = 0x7A,
+ MLXSW_TRAP_ID_RTR_EGRESS0 = 0x80,
MLXSW_TRAP_ID_IPV4_BGP = 0x88,
MLXSW_TRAP_ID_IPV6_BGP = 0x89,
MLXSW_TRAP_ID_L3_IPV6_ROUTER_SOLICITATION = 0x8A,
return result;
}
-static void lan743x_intr_software_isr(void *context)
+static void lan743x_intr_software_isr(struct lan743x_adapter *adapter)
{
- struct lan743x_adapter *adapter = context;
struct lan743x_intr *intr = &adapter->intr;
- u32 int_sts;
- int_sts = lan743x_csr_read(adapter, INT_STS);
- if (int_sts & INT_BIT_SW_GP_) {
- lan743x_csr_write(adapter, INT_STS, INT_BIT_SW_GP_);
- intr->software_isr_flag = 1;
- }
+ /* disable the interrupt to prevent repeated re-triggering */
+ lan743x_csr_write(adapter, INT_EN_CLR, INT_BIT_SW_GP_);
+ intr->software_isr_flag = true;
+ wake_up(&intr->software_isr_wq);
}
static void lan743x_tx_isr(void *context, u32 int_sts, u32 flags)
static int lan743x_intr_test_isr(struct lan743x_adapter *adapter)
{
struct lan743x_intr *intr = &adapter->intr;
- int result = -ENODEV;
- int timeout = 10;
+ int ret;
- intr->software_isr_flag = 0;
+ intr->software_isr_flag = false;
- /* enable interrupt */
+ /* enable and activate test interrupt */
lan743x_csr_write(adapter, INT_EN_SET, INT_BIT_SW_GP_);
-
- /* activate interrupt here */
lan743x_csr_write(adapter, INT_SET, INT_BIT_SW_GP_);
- while ((timeout > 0) && (!(intr->software_isr_flag))) {
- usleep_range(1000, 20000);
- timeout--;
- }
- if (intr->software_isr_flag)
- result = 0;
+ ret = wait_event_timeout(intr->software_isr_wq,
+ intr->software_isr_flag,
+ msecs_to_jiffies(200));
- /* disable interrupts */
+ /* disable test interrupt */
lan743x_csr_write(adapter, INT_EN_CLR, INT_BIT_SW_GP_);
- return result;
+
+ return ret > 0 ? 0 : -ENODEV;
}
static int lan743x_intr_register_isr(struct lan743x_adapter *adapter,
flags |= LAN743X_VECTOR_FLAG_SOURCE_ENABLE_R2C;
}
+ init_waitqueue_head(&intr->software_isr_wq);
+
ret = lan743x_intr_register_isr(adapter, 0, flags,
INT_BIT_ALL_RX_ | INT_BIT_ALL_TX_ |
INT_BIT_ALL_OTHER_,
data = lan743x_csr_read(adapter, MAC_CR);
/* set interface mode */
- if (phy_interface_mode_is_rgmii(adapter->phy_mode))
+ if (phy_interface_is_rgmii(phydev))
/* RGMII */
data &= ~MAC_CR_MII_EN_;
else
static int lan743x_phy_open(struct lan743x_adapter *adapter)
{
+ struct net_device *netdev = adapter->netdev;
struct lan743x_phy *phy = &adapter->phy;
- struct phy_device *phydev = NULL;
- struct device_node *phynode;
- struct net_device *netdev;
+ struct phy_device *phydev;
int ret = -EIO;
- netdev = adapter->netdev;
- phynode = of_node_get(adapter->pdev->dev.of_node);
-
- if (phynode) {
- /* try devicetree phy, or fixed link */
- of_get_phy_mode(phynode, &adapter->phy_mode);
-
- if (of_phy_is_fixed_link(phynode)) {
- ret = of_phy_register_fixed_link(phynode);
- if (ret) {
- netdev_err(netdev,
- "cannot register fixed PHY\n");
- of_node_put(phynode);
- goto return_error;
- }
- }
- phydev = of_phy_connect(netdev, phynode,
- lan743x_phy_link_status_change, 0,
- adapter->phy_mode);
- of_node_put(phynode);
- }
+ /* try devicetree phy, or fixed link */
+ phydev = of_phy_get_and_connect(netdev, adapter->pdev->dev.of_node,
+ lan743x_phy_link_status_change);
if (!phydev) {
/* try internal phy */
if (!phydev)
goto return_error;
- adapter->phy_mode = PHY_INTERFACE_MODE_GMII;
ret = phy_connect_direct(netdev, phydev,
lan743x_phy_link_status_change,
- adapter->phy_mode);
+ PHY_INTERFACE_MODE_GMII);
if (ret)
goto return_error;
}
phy_start(phydev);
phy_start_aneg(phydev);
+ phy_attached_info(phydev);
return 0;
return_error:
goto clear_active;
if (!(buffer_info->flags & TX_BUFFER_INFO_FLAG_TIMESTAMP_REQUESTED)) {
- dev_kfree_skb(buffer_info->skb);
+ dev_kfree_skb_any(buffer_info->skb);
goto clear_skb;
}
if (cleanup) {
lan743x_ptp_unrequest_tx_timestamp(tx->adapter);
- dev_kfree_skb(buffer_info->skb);
+ dev_kfree_skb_any(buffer_info->skb);
} else {
ignore_sync = (buffer_info->flags &
TX_BUFFER_INFO_FLAG_IGNORE_SYNC) != 0;
if (required_number_of_descriptors >
lan743x_tx_get_avail_desc(tx)) {
if (required_number_of_descriptors > (tx->ring_size - 1)) {
- dev_kfree_skb(skb);
+ dev_kfree_skb_irq(skb);
} else {
/* save to overflow buffer */
tx->overflow_skb = skb;
start_frame_length,
do_timestamp,
skb->ip_summed == CHECKSUM_PARTIAL)) {
- dev_kfree_skb(skb);
+ dev_kfree_skb_irq(skb);
goto unlock;
}
* frame assembler clean up was performed inside
* lan743x_tx_frame_add_fragment
*/
- dev_kfree_skb(skb);
+ dev_kfree_skb_irq(skb);
goto unlock;
}
}
int number_of_vectors;
bool using_vectors;
- int software_isr_flag;
+ bool software_isr_flag;
+ wait_queue_head_t software_isr_wq;
};
#define LAN743X_MAX_FRAME_SIZE (9 * 1024)
struct lan743x_adapter {
struct net_device *netdev;
struct mii_bus *mdiobus;
- phy_interface_t phy_mode;
int msg_enable;
#ifdef CONFIG_PM
u32 wolopts;
ipv6 = true;
break;
}
-#endif
fallthrough;
+#endif
case AF_INET:
req_sz = sizeof(struct nfp_crypto_req_add_v4);
ipv6 = false;
struct devlink_flash_update_params *params,
struct netlink_ext_ack *extack)
{
- return nfp_flash_update_common(devlink_priv(devlink), params->file_name, extack);
+ return nfp_flash_update_common(devlink_priv(devlink), params->fw, extack);
}
const struct devlink_ops nfp_devlink_ops = {
return nfp_pcie_sriov_enable(pdev, num_vfs);
}
-int nfp_flash_update_common(struct nfp_pf *pf, const char *path,
+int nfp_flash_update_common(struct nfp_pf *pf, const struct firmware *fw,
struct netlink_ext_ack *extack)
{
struct device *dev = &pf->pdev->dev;
- const struct firmware *fw;
struct nfp_nsp *nsp;
int err;
return err;
}
- err = request_firmware_direct(&fw, path, dev);
- if (err) {
- NL_SET_ERR_MSG_MOD(extack,
- "unable to read flash file from disk");
- goto exit_close_nsp;
- }
-
- dev_info(dev, "Please be patient while writing flash image: %s\n",
- path);
-
err = nfp_nsp_write_flash(nsp, fw);
if (err < 0)
- goto exit_release_fw;
+ goto exit_close_nsp;
dev_info(dev, "Finished writing flash image\n");
err = 0;
-exit_release_fw:
- release_firmware(fw);
exit_close_nsp:
nfp_nsp_close(nsp);
return err;
unsigned int min_size, struct nfp_cpp_area **area);
int nfp_mbox_cmd(struct nfp_pf *pf, u32 cmd, void *in_data, u64 in_length,
void *out_data, u64 out_length);
-int nfp_flash_update_common(struct nfp_pf *pf, const char *path,
+int nfp_flash_update_common(struct nfp_pf *pf, const struct firmware *fw,
struct netlink_ext_ack *extack);
enum nfp_dump_diag {
pch_gbe_clean_tx_ring(adapter, tx_ring);
vfree(tx_ring->buffer_info);
tx_ring->buffer_info = NULL;
- pci_free_consistent(pdev, tx_ring->size, tx_ring->desc, tx_ring->dma);
+ dma_free_coherent(&pdev->dev, tx_ring->size, tx_ring->desc,
+ tx_ring->dma);
tx_ring->desc = NULL;
}
pch_gbe_clean_rx_ring(adapter, rx_ring);
vfree(rx_ring->buffer_info);
rx_ring->buffer_info = NULL;
- pci_free_consistent(pdev, rx_ring->size, rx_ring->desc, rx_ring->dma);
+ dma_free_coherent(&pdev->dev, rx_ring->size, rx_ring->desc,
+ rx_ring->dma);
rx_ring->desc = NULL;
}
pch_gbe_clean_tx_ring(adapter, adapter->tx_ring);
pch_gbe_clean_rx_ring(adapter, adapter->rx_ring);
- pci_free_consistent(adapter->pdev, rx_ring->rx_buff_pool_size,
- rx_ring->rx_buff_pool, rx_ring->rx_buff_pool_logic);
+ dma_free_coherent(&adapter->pdev->dev, rx_ring->rx_buff_pool_size,
+ rx_ring->rx_buff_pool, rx_ring->rx_buff_pool_logic);
rx_ring->rx_buff_pool_logic = 0;
rx_ring->rx_buff_pool_size = 0;
rx_ring->rx_buff_pool = NULL;
if (ret)
return ret;
- if (pci_set_dma_mask(pdev, DMA_BIT_MASK(64))
- || pci_set_consistent_dma_mask(pdev, DMA_BIT_MASK(64))) {
- ret = pci_set_dma_mask(pdev, DMA_BIT_MASK(32));
+ if (dma_set_mask_and_coherent(&pdev->dev, DMA_BIT_MASK(64))) {
+ ret = dma_set_mask_and_coherent(&pdev->dev, DMA_BIT_MASK(32));
if (ret) {
- ret = pci_set_consistent_dma_mask(pdev,
- DMA_BIT_MASK(32));
- if (ret) {
- dev_err(&pdev->dev, "ERR: No usable DMA "
- "configuration, aborting\n");
- return ret;
- }
+ dev_err(&pdev->dev, "ERR: No usable DMA configuration, aborting\n");
+ return ret;
}
}
mac->tx = pasemi_mac_setup_tx_resources(dev);
- if (!mac->tx)
+ if (!mac->tx) {
+ ret = -ENOMEM;
goto out_tx_ring;
+ }
/* We might already have allocated rings in case mtu was changed
* before interface was brought up.
*/
if (dev->mtu > 1500 && !mac->num_cs) {
pasemi_mac_setup_csrings(mac);
- if (!mac->num_cs)
+ if (!mac->num_cs) {
+ ret = -ENOMEM;
goto out_tx_ring;
+ }
}
/* Zero out rmon counters */
work = kzalloc(sizeof(*work), GFP_ATOMIC);
if (!work) {
- dev_err(ionic->dev, "%s OOM\n", __func__);
+ dev_err(ionic->dev, "LIF reset trigger dropped\n");
} else {
work->type = IONIC_DW_TYPE_LIF_RESET;
if (fw_status & IONIC_FW_STS_F_RUNNING &&
{
struct ionic *ionic = devlink_priv(dl);
- return ionic_firmware_update(ionic->lif, params->file_name, extack);
+ return ionic_firmware_update(ionic->lif, params->fw, extack);
}
static int ionic_dl_info_get(struct devlink *dl, struct devlink_info_req *req,
#include <net/devlink.h>
-int ionic_firmware_update(struct ionic_lif *lif, const char *fw_name,
+int ionic_firmware_update(struct ionic_lif *lif, const struct firmware *fw,
struct netlink_ext_ack *extack);
struct ionic *ionic_devlink_alloc(struct device *dev);
return err;
}
-int ionic_firmware_update(struct ionic_lif *lif, const char *fw_name,
+int ionic_firmware_update(struct ionic_lif *lif, const struct firmware *fw,
struct netlink_ext_ack *extack)
{
struct ionic_dev *idev = &lif->ionic->idev;
struct ionic *ionic = lif->ionic;
union ionic_dev_cmd_comp comp;
u32 buf_sz, copy_sz, offset;
- const struct firmware *fw;
struct devlink *dl;
int next_interval;
int err = 0;
u8 fw_slot;
- netdev_info(netdev, "Installing firmware %s\n", fw_name);
+ netdev_info(netdev, "Installing firmware\n");
dl = priv_to_devlink(ionic);
- devlink_flash_update_begin_notify(dl);
devlink_flash_update_status_notify(dl, "Preparing to flash", NULL, 0, 0);
- err = request_firmware(&fw, fw_name, ionic->dev);
- if (err) {
- NL_SET_ERR_MSG_MOD(extack, "Unable to find firmware file");
- goto err_out;
- }
-
buf_sz = sizeof(idev->dev_cmd_regs->data);
netdev_dbg(netdev,
devlink_flash_update_status_notify(dl, "Flash failed", NULL, 0, 0);
else
devlink_flash_update_status_notify(dl, "Flash done", NULL, 0, 0);
- release_firmware(fw);
- devlink_flash_update_end_notify(dl);
return err;
}
// SPDX-License-Identifier: GPL-2.0
/* Copyright(c) 2017 - 2019 Pensando Systems, Inc */
+#include <linux/ethtool.h>
#include <linux/printk.h>
#include <linux/dynamic_debug.h>
#include <linux/netdevice.h>
case IONIC_EVENT_RESET:
work = kzalloc(sizeof(*work), GFP_ATOMIC);
if (!work) {
- netdev_err(lif->netdev, "%s OOM\n", __func__);
+ netdev_err(lif->netdev, "Reset event dropped\n");
} else {
work->type = IONIC_DW_TYPE_LIF_RESET;
ionic_lif_deferred_enqueue(&lif->deferred, work);
if (!can_sleep) {
work = kzalloc(sizeof(*work), GFP_ATOMIC);
- if (!work) {
- netdev_err(lif->netdev, "%s OOM\n", __func__);
+ if (!work)
return -ENOMEM;
- }
work->type = add ? IONIC_DW_TYPE_RX_ADDR_ADD :
IONIC_DW_TYPE_RX_ADDR_DEL;
memcpy(work->addr, addr, ETH_ALEN);
if (!can_sleep) {
work = kzalloc(sizeof(*work), GFP_ATOMIC);
if (!work) {
- netdev_err(lif->netdev, "%s OOM\n", __func__);
+ netdev_err(lif->netdev, "rxmode change dropped\n");
return;
}
work->type = IONIC_DW_TYPE_RX_MODE;
if (err)
return err;
- netdev->mtu = new_mtu;
/* if we're not running, nothing more to do */
- if (!netif_running(netdev))
+ if (!netif_running(netdev)) {
+ netdev->mtu = new_mtu;
return 0;
+ }
ionic_stop_queues_reconfig(lif);
+ netdev->mtu = new_mtu;
return ionic_start_queues_reconfig(lif);
}
idev->port_info_sz,
&idev->port_info_pa,
GFP_KERNEL);
- if (!idev->port_info) {
- dev_err(ionic->dev, "Failed to allocate port info\n");
+ if (!idev->port_info)
return -ENOMEM;
- }
}
sz = min(sizeof(ident->port.config), sizeof(idev->dev_cmd_regs->data));
// SPDX-License-Identifier: GPL-2.0
/* Copyright(c) 2017 - 2019 Pensando Systems, Inc */
+#include <linux/ethtool.h>
#include <linux/kernel.h>
#include <linux/mutex.h>
#include <linux/netdevice.h>
ilog2(rounded_conn_num));
STORE_RT_REG_AGG(p_hwfn, SRC_REG_FIRSTFREE_RT_OFFSET,
- p_hwfn->p_cxt_mngr->first_free);
+ p_hwfn->p_cxt_mngr->src_t2.first_free);
STORE_RT_REG_AGG(p_hwfn, SRC_REG_LASTFREE_RT_OFFSET,
- p_hwfn->p_cxt_mngr->last_free);
+ p_hwfn->p_cxt_mngr->src_t2.last_free);
}
/* Timers PF */
/* SRC T2 */
struct qed_src_t2 src_t2;
- u32 t2_num_pages;
- u64 first_free;
- u64 last_free;
/* total number of SRQ's for this hwfn */
u32 srq_count;
iwarp_info->partial_fpdus = kcalloc((u16)p_hwfn->p_rdma_info->num_qps,
sizeof(*iwarp_info->partial_fpdus),
GFP_KERNEL);
- if (!iwarp_info->partial_fpdus)
+ if (!iwarp_info->partial_fpdus) {
+ rc = -ENOMEM;
goto err;
+ }
iwarp_info->max_num_partial_fpdus = (u16)p_hwfn->p_rdma_info->num_qps;
iwarp_info->mpa_intermediate_buf = kzalloc(buff_size, GFP_KERNEL);
- if (!iwarp_info->mpa_intermediate_buf)
+ if (!iwarp_info->mpa_intermediate_buf) {
+ rc = -ENOMEM;
goto err;
+ }
/* The mpa_bufs array serves for pending RX packets received on the
* mpa ll2 that don't have place on the tx ring and require later
iwarp_info->mpa_bufs = kcalloc(data.input.rx_num_desc,
sizeof(*iwarp_info->mpa_bufs),
GFP_KERNEL);
- if (!iwarp_info->mpa_bufs)
+ if (!iwarp_info->mpa_bufs) {
+ rc = -ENOMEM;
goto err;
+ }
INIT_LIST_HEAD(&iwarp_info->mpa_buf_pending_list);
INIT_LIST_HEAD(&iwarp_info->mpa_buf_list);
/* Boot either flash image or firmware image from host file system */
if (qlcnic_load_fw_file == 1) {
- if (qlcnic_83xx_load_fw_image_from_host(adapter))
+ err = qlcnic_83xx_load_fw_image_from_host(adapter);
+ if (err)
return err;
} else {
QLC_SHARED_REG_WR32(adapter, QLCNIC_FW_IMG_VALID,
dev = skb->dev;
port = rmnet_get_port_rcu(dev);
+ if (unlikely(!port)) {
+ atomic_long_inc(&skb->dev->rx_nohandler);
+ kfree_skb(skb);
+ goto done;
+ }
switch (port->rmnet_mode) {
case RMNET_EPMODE_VND:
*/
#include <linux/etherdevice.h>
+#include <linux/ethtool.h>
#include <linux/if_arp.h>
#include <net/pkt_sched.h>
#include "rmnet_config.h"
rtl_loop_wait_low(tp, &rtl_counters_cond, 10, 1000);
}
-static void rtl8169_reset_counters(struct rtl8169_private *tp)
-{
- /*
- * Versions prior to RTL_GIGA_MAC_VER_19 don't support resetting the
- * tally counters.
- */
- if (tp->mac_version >= RTL_GIGA_MAC_VER_19)
- rtl8169_do_counters(tp, CounterReset);
-}
-
static void rtl8169_update_counters(struct rtl8169_private *tp)
{
u8 val = RTL_R8(tp, ChipCmd);
if (tp->tc_offset.inited)
return;
- rtl8169_reset_counters(tp);
- rtl8169_update_counters(tp);
+ if (tp->mac_version >= RTL_GIGA_MAC_VER_19) {
+ rtl8169_do_counters(tp, CounterReset);
+ } else {
+ rtl8169_update_counters(tp);
+ tp->tc_offset.tx_errors = counters->tx_errors;
+ tp->tc_offset.tx_multi_collision = counters->tx_multi_collision;
+ tp->tc_offset.tx_aborted = counters->tx_aborted;
+ tp->tc_offset.rx_missed = counters->rx_missed;
+ }
- tp->tc_offset.tx_errors = counters->tx_errors;
- tp->tc_offset.tx_multi_collision = counters->tx_multi_collision;
- tp->tc_offset.tx_aborted = counters->tx_aborted;
- tp->tc_offset.rx_missed = counters->rx_missed;
tp->tc_offset.inited = true;
}
return true;
}
-static bool rtl_tx_slots_avail(struct rtl8169_private *tp,
- unsigned int nr_frags)
+static bool rtl_tx_slots_avail(struct rtl8169_private *tp)
{
unsigned int slots_avail = READ_ONCE(tp->dirty_tx) + NUM_TX_DESC
- READ_ONCE(tp->cur_tx);
/* A skbuff with nr_frags needs nr_frags+1 entries in the tx queue */
- return slots_avail > nr_frags;
+ return slots_avail > MAX_SKB_FRAGS;
}
/* Versions RTL8102e and from RTL8168c onwards support csum_v2 */
bool stop_queue, door_bell;
u32 opts[2];
- txd_first = tp->TxDescArray + entry;
-
- if (unlikely(!rtl_tx_slots_avail(tp, frags))) {
+ if (unlikely(!rtl_tx_slots_avail(tp))) {
if (net_ratelimit())
netdev_err(dev, "BUG! Tx Ring full when queue awake!\n");
goto err_stop_0;
}
- if (unlikely(le32_to_cpu(txd_first->opts1) & DescOwn))
- goto err_stop_0;
-
opts[1] = rtl8169_tx_vlan_tag(skb);
opts[0] = 0;
entry, false)))
goto err_dma_0;
+ txd_first = tp->TxDescArray + entry;
+
if (frags) {
if (rtl8169_xmit_frags(tp, skb, opts, entry))
goto err_dma_1;
/* rtl_tx needs to see descriptor changes before updated tp->cur_tx */
smp_wmb();
- tp->cur_tx += frags + 1;
+ WRITE_ONCE(tp->cur_tx, tp->cur_tx + frags + 1);
- stop_queue = !rtl_tx_slots_avail(tp, MAX_SKB_FRAGS);
+ stop_queue = !rtl_tx_slots_avail(tp);
if (unlikely(stop_queue)) {
/* Avoid wrongly optimistic queue wake-up: rtl_tx thread must
* not miss a ring update when it notices a stopped queue.
*/
smp_wmb();
netif_stop_queue(dev);
- door_bell = true;
- }
-
- if (door_bell)
- rtl8169_doorbell(tp);
-
- if (unlikely(stop_queue)) {
/* Sync with rtl_tx:
* - publish queue status and cur_tx ring index (write barrier)
* - refresh dirty_tx ring index (read barrier).
* status and forget to wake up queue, a racing rtl_tx thread
* can't.
*/
- smp_mb();
- if (rtl_tx_slots_avail(tp, MAX_SKB_FRAGS))
+ smp_mb__after_atomic();
+ if (rtl_tx_slots_avail(tp))
netif_start_queue(dev);
+ door_bell = true;
}
+ if (door_bell)
+ rtl8169_doorbell(tp);
+
return NETDEV_TX_OK;
err_dma_1:
int budget)
{
unsigned int dirty_tx, bytes_compl = 0, pkts_compl = 0;
+ struct sk_buff *skb;
dirty_tx = tp->dirty_tx;
while (READ_ONCE(tp->cur_tx) != dirty_tx) {
unsigned int entry = dirty_tx % NUM_TX_DESC;
- struct sk_buff *skb = tp->tx_skb[entry].skb;
u32 status;
status = le32_to_cpu(tp->TxDescArray[entry].opts1);
if (status & DescOwn)
break;
+ skb = tp->tx_skb[entry].skb;
rtl8169_unmap_tx_skb(tp, entry);
if (skb) {
* ring status.
*/
smp_store_mb(tp->dirty_tx, dirty_tx);
- if (netif_queue_stopped(dev) &&
- rtl_tx_slots_avail(tp, MAX_SKB_FRAGS)) {
+ if (netif_queue_stopped(dev) && rtl_tx_slots_avail(tp))
netif_wake_queue(dev);
- }
/*
* 8168 hack: TxPoll requests are lost when the Tx packets are
* too close. Let's kick an extra TxPoll request when a burst
* of start_xmit activity is detected (if it is not detected,
* it is slow enough). -- FR
+ * If skb is NULL then we come here again once a tx irq is
+ * triggered after the last fragment is marked transmitted.
*/
- if (tp->cur_tx != dirty_tx)
+ if (tp->cur_tx != dirty_tx && skb)
rtl8169_doorbell(tp);
}
}
if (rc == -ENOENT)
/* clk-core allows NULL (for suspend / resume) */
rc = 0;
- else if (rc != -EPROBE_DEFER)
- dev_err(d, "failed to get clk: %d\n", rc);
+ else
+ dev_err_probe(d, rc, "failed to get clk\n");
} else {
tp->clk = clk;
rc = clk_prepare_enable(clk);
static bool netsec_clean_tx_dring(struct netsec_priv *priv)
{
struct netsec_desc_ring *dring = &priv->desc_ring[NETSEC_RING_TX];
+ struct xdp_frame_bulk bq;
struct netsec_de *entry;
int tail = dring->tail;
unsigned int bytes;
spin_lock(&dring->lock);
bytes = 0;
+ xdp_frame_bulk_init(&bq);
entry = dring->vaddr + DESC_SZ * tail;
+ rcu_read_lock(); /* need for xdp_return_frame_bulk */
+
while (!(entry->attr & (1U << NETSEC_TX_SHIFT_OWN_FIELD)) &&
cnt < DESC_NUM) {
struct netsec_desc *desc;
dev_kfree_skb(desc->skb);
} else {
bytes += desc->xdpf->len;
- xdp_return_frame(desc->xdpf);
+ if (desc->buf_type == TYPE_NETSEC_XDP_TX)
+ xdp_return_frame_rx_napi(desc->xdpf);
+ else
+ xdp_return_frame_bulk(desc->xdpf, &bq);
}
next:
/* clean up so netsec_uninit_pkt_dring() won't free the skb
entry = dring->vaddr + DESC_SZ * tail;
cnt++;
}
+ xdp_flush_frame_bulk(&bq);
+
+ rcu_read_unlock();
spin_unlock(&dring->lock);
/* Enable TX clock */
if (dwmac->data->tx_clk_en) {
dwmac->tx_clk = devm_clk_get(&pdev->dev, "tx_clk");
- if (IS_ERR(dwmac->tx_clk))
+ if (IS_ERR(dwmac->tx_clk)) {
+ ret = PTR_ERR(dwmac->tx_clk);
goto err_remove_config_dt;
+ }
clk_prepare_enable(dwmac->tx_clk);
.pcs_get_adv_lp = dwmac4_get_adv_lp,
.debug = dwmac4_debug,
.set_filter = dwmac4_set_filter,
- .flex_pps_config = dwmac5_flex_pps_config,
.set_mac_loopback = dwmac4_set_mac_loopback,
.update_vlan_hash = dwmac4_update_vlan_hash,
.sarc_configure = dwmac4_sarc_configure,
.pcs_get_adv_lp = dwmac4_get_adv_lp,
.debug = dwmac4_debug,
.set_filter = dwmac4_set_filter,
+ .flex_pps_config = dwmac5_flex_pps_config,
.set_mac_loopback = dwmac4_set_mac_loopback,
.update_vlan_hash = dwmac4_update_vlan_hash,
.sarc_configure = dwmac4_sarc_configure,
return readl_poll_timeout(ioaddr + DMA_BUS_MODE, value,
!(value & DMA_BUS_MODE_SFT_RESET),
- 10000, 100000);
+ 10000, 200000);
}
/* CSR1 enables the transmit DMA to check for new descriptor */
#define DRV_MODULE_VERSION "Jan_2016"
#include <linux/clk.h>
+#include <linux/hrtimer.h>
#include <linux/if_vlan.h>
#include <linux/stmmac.h>
#include <linux/phylink.h>
struct stmmac_tx_queue {
u32 tx_count_frames;
int tbs;
- struct timer_list txtimer;
+ struct hrtimer txtimer;
u32 queue_index;
struct stmmac_priv *priv_data;
struct dma_extended_desc *dma_etx ____cacheline_aligned_in_smp;
static void stmmac_exit_fs(struct net_device *dev);
#endif
-#define STMMAC_COAL_TIMER(x) (jiffies + usecs_to_jiffies(x))
+#define STMMAC_COAL_TIMER(x) (ns_to_ktime((x) * NSEC_PER_USEC))
/**
* stmmac_verify_args - verify the driver parameters.
/* We still have pending packets, let's call for a new scheduling */
if (tx_q->dirty_tx != tx_q->cur_tx)
- mod_timer(&tx_q->txtimer, STMMAC_COAL_TIMER(priv->tx_coal_timer));
+ hrtimer_start(&tx_q->txtimer, STMMAC_COAL_TIMER(priv->tx_coal_timer),
+ HRTIMER_MODE_REL);
__netif_tx_unlock_bh(netdev_get_tx_queue(priv->dev, queue));
{
struct stmmac_tx_queue *tx_q = &priv->tx_queue[queue];
- mod_timer(&tx_q->txtimer, STMMAC_COAL_TIMER(priv->tx_coal_timer));
+ hrtimer_start(&tx_q->txtimer, STMMAC_COAL_TIMER(priv->tx_coal_timer),
+ HRTIMER_MODE_REL);
}
/**
* Description:
* This is the timer handler to directly invoke the stmmac_tx_clean.
*/
-static void stmmac_tx_timer(struct timer_list *t)
+static enum hrtimer_restart stmmac_tx_timer(struct hrtimer *t)
{
- struct stmmac_tx_queue *tx_q = from_timer(tx_q, t, txtimer);
+ struct stmmac_tx_queue *tx_q = container_of(t, struct stmmac_tx_queue, txtimer);
struct stmmac_priv *priv = tx_q->priv_data;
struct stmmac_channel *ch;
spin_unlock_irqrestore(&ch->lock, flags);
__napi_schedule(&ch->tx_napi);
}
+
+ return HRTIMER_NORESTART;
}
/**
for (chan = 0; chan < tx_channel_count; chan++) {
struct stmmac_tx_queue *tx_q = &priv->tx_queue[chan];
- timer_setup(&tx_q->txtimer, stmmac_tx_timer, 0);
+ hrtimer_init(&tx_q->txtimer, CLOCK_MONOTONIC, HRTIMER_MODE_REL);
+ tx_q->txtimer.function = stmmac_tx_timer;
}
}
phylink_stop(priv->phylink);
for (chan = 0; chan < priv->plat->tx_queues_to_use; chan++)
- del_timer_sync(&priv->tx_queue[chan].txtimer);
+ hrtimer_cancel(&priv->tx_queue[chan].txtimer);
stmmac_hw_teardown(dev);
init_error:
stmmac_disable_all_queues(priv);
for (chan = 0; chan < priv->plat->tx_queues_to_use; chan++)
- del_timer_sync(&priv->tx_queue[chan].txtimer);
+ hrtimer_cancel(&priv->tx_queue[chan].txtimer);
/* Free the IRQ lines */
free_irq(dev->irq, dev);
stmmac_disable_all_queues(priv);
for (chan = 0; chan < priv->plat->tx_queues_to_use; chan++)
- del_timer_sync(&priv->tx_queue[chan].txtimer);
+ hrtimer_cancel(&priv->tx_queue[chan].txtimer);
/* Stop TX/RX DMA */
stmmac_stop_all_dma(priv);
return ret;
}
+ rtnl_lock();
mutex_lock(&priv->lock);
stmmac_reset_queues_param(priv);
stmmac_enable_all_queues(priv);
mutex_unlock(&priv->lock);
+ rtnl_unlock();
if (!device_may_wakeup(priv->device) || !priv->plat->pmt) {
rtnl_lock();
if (IS_ERR_OR_NULL(cpts->ptp_clock)) {
dev_err(dev, "Failed to register ptp clk %ld\n",
PTR_ERR(cpts->ptp_clock));
- if (!cpts->ptp_clock)
- ret = -ENODEV;
+ ret = cpts->ptp_clock ? PTR_ERR(cpts->ptp_clock) : -ENODEV;
goto refclk_disable;
}
cpts->phc_index = ptp_clock_index(cpts->ptp_clock);
if (ret < 0)
goto err_cleanup;
- if (cpts_register(cpsw->cpts))
- dev_err(priv->dev, "error registering cpts device\n");
-
+ if (cpsw->cpts) {
+ if (cpts_register(cpsw->cpts))
+ dev_err(priv->dev, "error registering cpts device\n");
+ else
+ writel(0x10, &cpsw->wr_regs->misc_en);
+ }
}
cpsw_restore(priv);
CPSW_MAX_QUEUES, CPSW_MAX_QUEUES);
if (!ndev) {
dev_err(dev, "error allocating net_device\n");
+ ret = -ENOMEM;
goto clean_cpts;
}
/* Enable misc CPTS evnt_pend IRQ */
cpts_set_irqpoll(cpsw->cpts, false);
- writel(0x10, &cpsw->wr_regs->misc_en);
skip_cpts:
cpsw_notice(priv, probe,
if (ret < 0)
goto err_cleanup;
- if (cpts_register(cpsw->cpts))
- dev_err(priv->dev, "error registering cpts device\n");
+ if (cpsw->cpts) {
+ if (cpts_register(cpsw->cpts))
+ dev_err(priv->dev, "error registering cpts device\n");
+ else
+ writel(0x10, &cpsw->wr_regs->misc_en);
+ }
napi_enable(&cpsw->napi_rx);
napi_enable(&cpsw->napi_tx);
/* Enable misc CPTS evnt_pend IRQ */
cpts_set_irqpoll(cpsw->cpts, false);
- writel(0x10, &cpsw->wr_regs->misc_en);
skip_cpts:
ret = cpsw_register_notifiers(cpsw);
#define pr_fmt(fmt) KBUILD_MODNAME ": " fmt
+#include <linux/ethtool.h>
#include <linux/kernel.h>
#include <linux/module.h>
#include <linux/etherdevice.h>
if (ip_tunnel_collect_metadata() || gs->collect_md) {
__be16 flags;
- flags = TUNNEL_KEY | TUNNEL_GENEVE_OPT |
- (gnvh->oam ? TUNNEL_OAM : 0) |
+ flags = TUNNEL_KEY | (gnvh->oam ? TUNNEL_OAM : 0) |
(gnvh->critical ? TUNNEL_CRIT_OPT : 0);
tun_dst = udp_tun_rx_dst(skb, geneve_get_sk_family(gs), flags,
skb_dst_set(skb, &tun_dst->dst);
/* Ignore packet loops (and multicast echo) */
- if (ether_addr_equal(eth_hdr(skb)->h_source, geneve->dev->dev_addr)) {
- geneve->dev->stats.rx_errors++;
- goto drop;
- }
+ if (ether_addr_equal(eth_hdr(skb)->h_source, geneve->dev->dev_addr))
+ goto rx_error;
+ switch (skb_protocol(skb, true)) {
+ case htons(ETH_P_IP):
+ if (pskb_may_pull(skb, sizeof(struct iphdr)))
+ goto rx_error;
+ break;
+ case htons(ETH_P_IPV6):
+ if (pskb_may_pull(skb, sizeof(struct ipv6hdr)))
+ goto rx_error;
+ break;
+ default:
+ goto rx_error;
+ }
oiph = skb_network_header(skb);
skb_reset_network_header(skb);
dev_sw_netstats_rx_add(geneve->dev, len);
return;
+rx_error:
+ geneve->dev->stats.rx_errors++;
drop:
/* Consume bad packet */
kfree_skb(skb);
#include <linux/init.h>
#include <linux/atomic.h>
+#include <linux/ethtool.h>
#include <linux/module.h>
#include <linux/highmem.h>
#include <linux/device.h>
* Haiyang Zhang <haiyangz@microsoft.com>
* Hank Janssen <hjanssen@microsoft.com>
*/
+#include <linux/ethtool.h>
#include <linux/kernel.h>
#include <linux/sched.h>
#include <linux/wait.h>
#define GSI_CMD_TIMEOUT 5 /* seconds */
#define GSI_CHANNEL_STOP_RX_RETRIES 10
+#define GSI_CHANNEL_MODEM_HALT_RETRIES 10
#define GSI_MHI_EVENT_ID_START 10 /* 1st reserved event id */
#define GSI_MHI_EVENT_ID_END 16 /* Last reserved event id */
/* Turn off all GSI interrupts initially */
static void gsi_irq_setup(struct gsi *gsi)
{
+ u32 adjust;
+
/* Disable all interrupt types */
gsi_irq_type_update(gsi, 0);
iowrite32(0, gsi->virt + GSI_CNTXT_SRC_EV_CH_IRQ_MSK_OFFSET);
iowrite32(0, gsi->virt + GSI_CNTXT_GLOB_IRQ_EN_OFFSET);
iowrite32(0, gsi->virt + GSI_CNTXT_SRC_IEOB_IRQ_MSK_OFFSET);
- iowrite32(0, gsi->virt + GSI_INTER_EE_SRC_CH_IRQ_OFFSET);
- iowrite32(0, gsi->virt + GSI_INTER_EE_SRC_EV_CH_IRQ_OFFSET);
+
+ /* Reverse the offset adjustment for inter-EE register offsets */
+ adjust = gsi->version < IPA_VERSION_4_5 ? 0 : GSI_EE_REG_ADJUST;
+ iowrite32(0, gsi->virt + adjust + GSI_INTER_EE_SRC_CH_IRQ_OFFSET);
+ iowrite32(0, gsi->virt + adjust + GSI_INTER_EE_SRC_EV_CH_IRQ_OFFSET);
+
iowrite32(0, gsi->virt + GSI_CNTXT_GSI_IRQ_EN_OFFSET);
}
/* Get initial event ring state */
evt_ring->state = gsi_evt_ring_state(gsi, evt_ring_id);
if (evt_ring->state != GSI_EVT_RING_STATE_NOT_ALLOCATED) {
- dev_err(gsi->dev, "bad event ring state %u before alloc\n",
- evt_ring->state);
+ dev_err(gsi->dev, "event ring %u bad state %u before alloc\n",
+ evt_ring_id, evt_ring->state);
return -EINVAL;
}
ret = evt_ring_command(gsi, evt_ring_id, GSI_EVT_ALLOCATE);
if (!ret && evt_ring->state != GSI_EVT_RING_STATE_ALLOCATED) {
- dev_err(gsi->dev, "bad event ring state %u after alloc\n",
- evt_ring->state);
+ dev_err(gsi->dev, "event ring %u bad state %u after alloc\n",
+ evt_ring_id, evt_ring->state);
ret = -EIO;
}
if (state != GSI_EVT_RING_STATE_ALLOCATED &&
state != GSI_EVT_RING_STATE_ERROR) {
- dev_err(gsi->dev, "bad event ring state %u before reset\n",
- evt_ring->state);
+ dev_err(gsi->dev, "event ring %u bad state %u before reset\n",
+ evt_ring_id, evt_ring->state);
return;
}
ret = evt_ring_command(gsi, evt_ring_id, GSI_EVT_RESET);
if (!ret && evt_ring->state != GSI_EVT_RING_STATE_ALLOCATED)
- dev_err(gsi->dev, "bad event ring state %u after reset\n",
- evt_ring->state);
+ dev_err(gsi->dev, "event ring %u bad state %u after reset\n",
+ evt_ring_id, evt_ring->state);
}
/* Issue a hardware de-allocation request for an allocated event ring */
int ret;
if (evt_ring->state != GSI_EVT_RING_STATE_ALLOCATED) {
- dev_err(gsi->dev, "bad event ring state %u before dealloc\n",
- evt_ring->state);
+ dev_err(gsi->dev, "event ring %u state %u before dealloc\n",
+ evt_ring_id, evt_ring->state);
return;
}
ret = evt_ring_command(gsi, evt_ring_id, GSI_EVT_DE_ALLOC);
if (!ret && evt_ring->state != GSI_EVT_RING_STATE_NOT_ALLOCATED)
- dev_err(gsi->dev, "bad event ring state %u after dealloc\n",
- evt_ring->state);
+ dev_err(gsi->dev, "event ring %u bad state %u after dealloc\n",
+ evt_ring_id, evt_ring->state);
}
/* Fetch the current state of a channel from hardware */
/* Get initial channel state */
state = gsi_channel_state(channel);
if (state != GSI_CHANNEL_STATE_NOT_ALLOCATED) {
- dev_err(dev, "bad channel state %u before alloc\n", state);
+ dev_err(dev, "channel %u bad state %u before alloc\n",
+ channel_id, state);
return -EINVAL;
}
/* Channel state will normally have been updated */
state = gsi_channel_state(channel);
if (!ret && state != GSI_CHANNEL_STATE_ALLOCATED) {
- dev_err(dev, "bad channel state %u after alloc\n", state);
+ dev_err(dev, "channel %u bad state %u after alloc\n",
+ channel_id, state);
ret = -EIO;
}
state = gsi_channel_state(channel);
if (state != GSI_CHANNEL_STATE_ALLOCATED &&
state != GSI_CHANNEL_STATE_STOPPED) {
- dev_err(dev, "bad channel state %u before start\n", state);
+ dev_err(dev, "channel %u bad state %u before start\n",
+ gsi_channel_id(channel), state);
return -EINVAL;
}
/* Channel state will normally have been updated */
state = gsi_channel_state(channel);
if (!ret && state != GSI_CHANNEL_STATE_STARTED) {
- dev_err(dev, "bad channel state %u after start\n", state);
+ dev_err(dev, "channel %u bad state %u after start\n",
+ gsi_channel_id(channel), state);
ret = -EIO;
}
if (state != GSI_CHANNEL_STATE_STARTED &&
state != GSI_CHANNEL_STATE_STOP_IN_PROC) {
- dev_err(dev, "bad channel state %u before stop\n", state);
+ dev_err(dev, "channel %u bad state %u before stop\n",
+ gsi_channel_id(channel), state);
return -EINVAL;
}
if (state == GSI_CHANNEL_STATE_STOP_IN_PROC)
return -EAGAIN;
- dev_err(dev, "bad channel state %u after stop\n", state);
+ dev_err(dev, "channel %u bad state %u after stop\n",
+ gsi_channel_id(channel), state);
return -EIO;
}
state = gsi_channel_state(channel);
if (state != GSI_CHANNEL_STATE_STOPPED &&
state != GSI_CHANNEL_STATE_ERROR) {
- dev_err(dev, "bad channel state %u before reset\n", state);
+ /* No need to reset a channel already in ALLOCATED state */
+ if (state != GSI_CHANNEL_STATE_ALLOCATED)
+ dev_err(dev, "channel %u bad state %u before reset\n",
+ gsi_channel_id(channel), state);
return;
}
/* Channel state will normally have been updated */
state = gsi_channel_state(channel);
if (!ret && state != GSI_CHANNEL_STATE_ALLOCATED)
- dev_err(dev, "bad channel state %u after reset\n", state);
+ dev_err(dev, "channel %u bad state %u after reset\n",
+ gsi_channel_id(channel), state);
}
/* Deallocate an ALLOCATED GSI channel */
state = gsi_channel_state(channel);
if (state != GSI_CHANNEL_STATE_ALLOCATED) {
- dev_err(dev, "bad channel state %u before dealloc\n", state);
+ dev_err(dev, "channel %u bad state %u before dealloc\n",
+ channel_id, state);
return;
}
/* Channel state will normally have been updated */
state = gsi_channel_state(channel);
if (!ret && state != GSI_CHANNEL_STATE_NOT_ALLOCATED)
- dev_err(dev, "bad channel state %u after dealloc\n", state);
+ dev_err(dev, "channel %u bad state %u after dealloc\n",
+ channel_id, state);
}
/* Ring an event ring doorbell, reporting the last entry processed by the AP.
if (gsi->version == IPA_VERSION_3_5_1 && doorbell)
val |= USE_DB_ENG_FMASK;
- /* Starting with IPA v4.0 the command channel uses the escape buffer */
- if (gsi->version != IPA_VERSION_3_5_1 && channel->command)
- val |= USE_ESCAPE_BUF_ONLY_FMASK;
+ /* v4.0 introduces an escape buffer for prefetch. We use it
+ * on all but the AP command channel.
+ */
+ if (gsi->version != IPA_VERSION_3_5_1 && !channel->command) {
+ /* If not otherwise set, prefetch buffers are used */
+ if (gsi->version < IPA_VERSION_4_5)
+ val |= USE_ESCAPE_BUF_ONLY_FMASK;
+ else
+ val |= u32_encode_bits(GSI_ESCAPE_BUF_ONLY,
+ PREFETCH_MODE_FMASK);
+ }
iowrite32(val, gsi->virt + GSI_CH_C_QOS_OFFSET(channel_id));
u32 result;
u32 val;
+ /* This interrupt is used to handle completions of the two GENERIC
+ * GSI commands. We use these to allocate and halt channels on
+ * the modem's behalf due to a hardware quirk on IPA v4.2. Once
+ * allocated, the modem "owns" these channels, and as a result we
+ * have no way of knowing the channel's state at any given time.
+ *
+ * It is recommended that we halt the modem channels we allocated
+ * when shutting down, but it's possible the channel isn't running
+ * at the time we issue the HALT command. We'll get an error in
+ * that case, but it's harmless (the channel is already halted).
+ *
+ * For this reason, we silently ignore a CHANNEL_NOT_RUNNING error
+ * if we receive it.
+ */
val = ioread32(gsi->virt + GSI_CNTXT_SCRATCH_0_OFFSET);
result = u32_get_bits(val, GENERIC_EE_RESULT_FMASK);
- if (result != GENERIC_EE_SUCCESS)
+
+ switch (result) {
+ case GENERIC_EE_SUCCESS:
+ case GENERIC_EE_CHANNEL_NOT_RUNNING:
+ gsi->result = 0;
+ break;
+
+ case GENERIC_EE_RETRY:
+ gsi->result = -EAGAIN;
+ break;
+
+ default:
dev_err(gsi->dev, "global INT1 generic result %u\n", result);
+ gsi->result = -EIO;
+ break;
+ }
complete(&gsi->completion);
}
iowrite32(BIT(ERROR_INT), gsi->virt + GSI_CNTXT_GLOB_IRQ_EN_OFFSET);
if (success)
- return 0;
+ return gsi->result;
dev_err(gsi->dev, "GSI generic command %u to channel %u timed out\n",
opcode, channel_id);
static void gsi_modem_channel_halt(struct gsi *gsi, u32 channel_id)
{
- (void)gsi_generic_command(gsi, channel_id, GSI_GENERIC_HALT_CHANNEL);
+ u32 retries = GSI_CHANNEL_MODEM_HALT_RETRIES;
+ int ret;
+
+ do
+ ret = gsi_generic_command(gsi, channel_id,
+ GSI_GENERIC_HALT_CHANNEL);
+ while (ret == -EAGAIN && retries--);
+
+ if (ret)
+ dev_err(gsi->dev, "error %d halting modem channel %u\n",
+ ret, channel_id);
}
/* Setup function for channels */
struct device *dev = &pdev->dev;
struct resource *res;
resource_size_t size;
+ u32 adjust;
int ret;
gsi_validate_build();
return -EINVAL;
}
+ /* Make sure we can make our pointer adjustment if necessary */
+ adjust = gsi->version < IPA_VERSION_4_5 ? 0 : GSI_EE_REG_ADJUST;
+ if (res->start < adjust) {
+ dev_err(dev, "DT memory resource \"gsi\" too low (< %u)\n",
+ adjust);
+ return -EINVAL;
+ }
+
gsi->virt = ioremap(res->start, size);
if (!gsi->virt) {
dev_err(dev, "unable to remap \"gsi\" memory\n");
return -ENOMEM;
}
+ /* Adjust register range pointer downward for newer IPA versions */
+ gsi->virt -= adjust;
init_completion(&gsi->completion);
/* Execution environment IDs */
enum gsi_ee_id {
- GSI_EE_AP = 0,
- GSI_EE_MODEM = 1,
- GSI_EE_UC = 2,
- GSI_EE_TZ = 3,
+ GSI_EE_AP = 0x0,
+ GSI_EE_MODEM = 0x1,
+ GSI_EE_UC = 0x2,
+ GSI_EE_TZ = 0x3,
};
struct gsi_ring {
/* Hardware values signifying the state of a channel */
enum gsi_channel_state {
- GSI_CHANNEL_STATE_NOT_ALLOCATED = 0x0,
- GSI_CHANNEL_STATE_ALLOCATED = 0x1,
- GSI_CHANNEL_STATE_STARTED = 0x2,
- GSI_CHANNEL_STATE_STOPPED = 0x3,
- GSI_CHANNEL_STATE_STOP_IN_PROC = 0x4,
- GSI_CHANNEL_STATE_ERROR = 0xf,
+ GSI_CHANNEL_STATE_NOT_ALLOCATED = 0x0,
+ GSI_CHANNEL_STATE_ALLOCATED = 0x1,
+ GSI_CHANNEL_STATE_STARTED = 0x2,
+ GSI_CHANNEL_STATE_STOPPED = 0x3,
+ GSI_CHANNEL_STATE_STOP_IN_PROC = 0x4,
+ GSI_CHANNEL_STATE_ERROR = 0xf,
};
/* We only care about channels between IPA and AP */
u32 type_enabled_bitmap; /* GSI IRQ types enabled */
u32 ieob_enabled_bitmap; /* IEOB IRQ enabled (event rings) */
struct completion completion; /* for global EE commands */
+ int result; /* Negative errno (generic commands) */
struct mutex mutex; /* protects commands, programming */
};
* (though the actual limit is hardware-dependent).
*/
+/* GSI EE registers as a group are shifted downward by a fixed
+ * constant amount for IPA versions 4.5 and beyond. This applies
+ * to all GSI registers we use *except* the ones that disable
+ * inter-EE interrupts for channels and event channels.
+ *
+ * We handle this by adjusting the pointer to the mapped GSI memory
+ * region downward. Then in the one place we use them (gsi_irq_setup())
+ * we undo that adjustment for the inter-EE interrupt registers.
+ */
+#define GSI_EE_REG_ADJUST 0x0000d000 /* IPA v4.5+ */
+
#define GSI_INTER_EE_SRC_CH_IRQ_OFFSET \
GSI_INTER_EE_N_SRC_CH_IRQ_OFFSET(GSI_EE_AP)
#define GSI_INTER_EE_N_SRC_CH_IRQ_OFFSET(ee) \
#define ERINDEX_FMASK GENMASK(18, 14)
#define CHSTATE_FMASK GENMASK(23, 20)
#define ELEMENT_SIZE_FMASK GENMASK(31, 24)
+
/** enum gsi_channel_type - CHTYPE_PROTOCOL field values in CH_C_CNTXT_0 */
enum gsi_channel_type {
GSI_CHANNEL_TYPE_MHI = 0x0,
#define USE_DB_ENG_FMASK GENMASK(9, 9)
/* The next field is only present for IPA v4.0, v4.1, and v4.2 */
#define USE_ESCAPE_BUF_ONLY_FMASK GENMASK(10, 10)
+/* The next two fields are present for IPA v4.5 and above */
+#define PREFETCH_MODE_FMASK GENMASK(13, 10)
+#define EMPTY_LVL_THRSHOLD_FMASK GENMASK(23, 16)
+/** enum gsi_prefetch_mode - PREFETCH_MODE field in CH_C_QOS */
+enum gsi_prefetch_mode {
+ GSI_USE_PREFETCH_BUFS = 0x0,
+ GSI_ESCAPE_BUF_ONLY = 0x1,
+ GSI_SMART_PREFETCH = 0x2,
+ GSI_FREE_PREFETCH = 0x3,
+};
#define GSI_CH_C_SCRATCH_0_OFFSET(ch) \
GSI_EE_N_CH_C_SCRATCH_0_OFFSET((ch), GSI_EE_AP)
(0x0001f008 + 0x4000 * (ee))
#define CH_CHID_FMASK GENMASK(7, 0)
#define CH_OPCODE_FMASK GENMASK(31, 24)
+
/** enum gsi_ch_cmd_opcode - CH_OPCODE field values in CH_CMD */
enum gsi_ch_cmd_opcode {
GSI_CH_ALLOCATE = 0x0,
(0x0001f010 + 0x4000 * (ee))
#define EV_CHID_FMASK GENMASK(7, 0)
#define EV_OPCODE_FMASK GENMASK(31, 24)
+
/** enum gsi_evt_cmd_opcode - EV_OPCODE field values in EV_CH_CMD */
enum gsi_evt_cmd_opcode {
GSI_EVT_ALLOCATE = 0x0,
#define GENERIC_OPCODE_FMASK GENMASK(4, 0)
#define GENERIC_CHID_FMASK GENMASK(9, 5)
#define GENERIC_EE_FMASK GENMASK(13, 10)
+
/** enum gsi_generic_cmd_opcode - GENERIC_OPCODE field values in GENERIC_CMD */
enum gsi_generic_cmd_opcode {
GSI_GENERIC_HALT_CHANNEL = 0x1,
/* Fields below are present for IPA v4.2 and above */
#define GSI_USE_RD_WR_ENG_FMASK GENMASK(30, 30)
#define GSI_USE_INTER_EE_FMASK GENMASK(31, 31)
+
/** enum gsi_iram_size - IRAM_SIZE field values in HW_PARAM_2 */
enum gsi_iram_size {
IRAM_SIZE_ONE_KB = 0x0,
/* The next two values are available for IPA v4.0 and above */
IRAM_SIZE_TWO_N_HALF_KB = 0x2,
IRAM_SIZE_THREE_KB = 0x3,
+ /* The next two values are available for IPA v4.5 and above */
+ IRAM_SIZE_THREE_N_HALF_KB = 0x4,
+ IRAM_SIZE_FOUR_KB = 0x5,
};
/* IRQ condition for each type is cleared by writing type-specific register */
GSI_EE_N_CNTXT_TYPE_IRQ_MSK_OFFSET(GSI_EE_AP)
#define GSI_EE_N_CNTXT_TYPE_IRQ_MSK_OFFSET(ee) \
(0x0001f088 + 0x4000 * (ee))
+
/* Values here are bit positions in the TYPE_IRQ and TYPE_IRQ_MSK registers */
enum gsi_irq_type_id {
- GSI_CH_CTRL = 0, /* channel allocation, etc. */
- GSI_EV_CTRL = 1, /* event ring allocation, etc. */
- GSI_GLOB_EE = 2, /* global/general event */
- GSI_IEOB = 3, /* TRE completion */
- GSI_INTER_EE_CH_CTRL = 4, /* remote-issued stop/reset (unused) */
- GSI_INTER_EE_EV_CTRL = 5, /* remote-issued event reset (unused) */
- GSI_GENERAL = 6, /* general-purpose event */
+ GSI_CH_CTRL = 0x0, /* channel allocation, etc. */
+ GSI_EV_CTRL = 0x1, /* event ring allocation, etc. */
+ GSI_GLOB_EE = 0x2, /* global/general event */
+ GSI_IEOB = 0x3, /* TRE completion */
+ GSI_INTER_EE_CH_CTRL = 0x4, /* remote-issued stop/reset (unused) */
+ GSI_INTER_EE_EV_CTRL = 0x5, /* remote-issued event reset (unused) */
+ GSI_GENERAL = 0x6, /* general-purpose event */
};
#define GSI_CNTXT_SRC_CH_IRQ_OFFSET \
#define ERR_VIRT_IDX_FMASK GENMASK(23, 19)
#define ERR_TYPE_FMASK GENMASK(27, 24)
#define ERR_EE_FMASK GENMASK(31, 28)
+
/** enum gsi_err_code - ERR_CODE field values in EE_ERR_LOG */
enum gsi_err_code {
GSI_INVALID_TRE = 0x1,
/* 7 is not assigned */
GSI_HWO_1 = 0x8,
};
+
/** enum gsi_err_type - ERR_TYPE field values in EE_ERR_LOG */
enum gsi_err_type {
GSI_ERR_TYPE_GLOB = 0x1,
(0x0001f400 + 0x4000 * (ee))
#define INTER_EE_RESULT_FMASK GENMASK(2, 0)
#define GENERIC_EE_RESULT_FMASK GENMASK(7, 5)
+
+/** enum gsi_generic_ee_result - GENERIC_EE_RESULT field values in SCRATCH_0 */
enum gsi_generic_ee_result {
GENERIC_EE_SUCCESS = 0x1,
GENERIC_EE_CHANNEL_NOT_RUNNING = 0x2,
GENERIC_EE_RETRY = 0x6,
GENERIC_EE_NO_RESOURCES = 0x7,
};
+
#define USB_MAX_PACKET_FMASK GENMASK(15, 15) /* 0: HS; 1: SS */
#define MHI_BASE_CHANNEL_FMASK GENMASK(31, 24)
return trans;
}
-/* Free a previously-allocated transaction (used only in case of error) */
+/* Free a previously-allocated transaction */
void gsi_trans_free(struct gsi_trans *trans)
{
+ refcount_t *refcount = &trans->refcount;
struct gsi_trans_info *trans_info;
+ bool last;
- if (!refcount_dec_and_test(&trans->refcount))
+ /* We must hold the lock to release the last reference */
+ if (refcount_dec_not_one(refcount))
return;
trans_info = &trans->gsi->channel[trans->channel_id].trans_info;
spin_lock_bh(&trans_info->spinlock);
- list_del(&trans->links);
+ /* Reference might have been added before we got the lock */
+ last = refcount_dec_and_test(refcount);
+ if (last)
+ list_del(&trans->links);
spin_unlock_bh(&trans_info->spinlock);
+ if (!last)
+ return;
+
ipa_gsi_trans_release(trans);
/* Releasing the reserved TREs implicitly frees the sgl[] and
#include "ipa.h"
#include "ipa_clock.h"
#include "ipa_modem.h"
+#include "ipa_data.h"
/**
* DOC: IPA Clocking
* An IPA clock reference must be held for any access to IPA hardware.
*/
-#define IPA_CORE_CLOCK_RATE (75UL * 1000 * 1000) /* Hz */
-
-/* Interconnect path bandwidths (each times 1000 bytes per second) */
-#define IPA_MEMORY_AVG (80 * 1000) /* 80 MBps */
-#define IPA_MEMORY_PEAK (600 * 1000)
-
-#define IPA_IMEM_AVG (80 * 1000)
-#define IPA_IMEM_PEAK (350 * 1000)
-
-#define IPA_CONFIG_AVG (40 * 1000)
-#define IPA_CONFIG_PEAK (40 * 1000)
-
/**
* struct ipa_clock - IPA clocking information
* @count: Clocking reference count
* @memory_path: Memory interconnect
* @imem_path: Internal memory interconnect
* @config_path: Configuration space interconnect
+ * @interconnect_data: Interconnect configuration data
*/
struct ipa_clock {
refcount_t count;
struct icc_path *memory_path;
struct icc_path *imem_path;
struct icc_path *config_path;
+ const struct ipa_interconnect_data *interconnect_data;
};
static struct icc_path *
/* Currently we only use one bandwidth level, so just "enable" interconnects */
static int ipa_interconnect_enable(struct ipa *ipa)
{
+ const struct ipa_interconnect_data *data;
struct ipa_clock *clock = ipa->clock;
int ret;
- ret = icc_set_bw(clock->memory_path, IPA_MEMORY_AVG, IPA_MEMORY_PEAK);
+ data = &clock->interconnect_data[IPA_INTERCONNECT_MEMORY];
+ ret = icc_set_bw(clock->memory_path, data->average_rate,
+ data->peak_rate);
if (ret)
return ret;
- ret = icc_set_bw(clock->imem_path, IPA_IMEM_AVG, IPA_IMEM_PEAK);
+ data = &clock->interconnect_data[IPA_INTERCONNECT_IMEM];
+ ret = icc_set_bw(clock->memory_path, data->average_rate,
+ data->peak_rate);
if (ret)
goto err_memory_path_disable;
- ret = icc_set_bw(clock->config_path, IPA_CONFIG_AVG, IPA_CONFIG_PEAK);
+ data = &clock->interconnect_data[IPA_INTERCONNECT_CONFIG];
+ ret = icc_set_bw(clock->memory_path, data->average_rate,
+ data->peak_rate);
if (ret)
goto err_imem_path_disable;
/* To disable an interconnect, we just its bandwidth to 0 */
static int ipa_interconnect_disable(struct ipa *ipa)
{
+ const struct ipa_interconnect_data *data;
struct ipa_clock *clock = ipa->clock;
int ret;
return 0;
err_imem_path_reenable:
- (void)icc_set_bw(clock->imem_path, IPA_IMEM_AVG, IPA_IMEM_PEAK);
+ data = &clock->interconnect_data[IPA_INTERCONNECT_IMEM];
+ (void)icc_set_bw(clock->imem_path, data->average_rate,
+ data->peak_rate);
err_memory_path_reenable:
- (void)icc_set_bw(clock->memory_path, IPA_MEMORY_AVG, IPA_MEMORY_PEAK);
+ data = &clock->interconnect_data[IPA_INTERCONNECT_MEMORY];
+ (void)icc_set_bw(clock->memory_path, data->average_rate,
+ data->peak_rate);
return ret;
}
}
/* Initialize IPA clocking */
-struct ipa_clock *ipa_clock_init(struct device *dev)
+struct ipa_clock *
+ipa_clock_init(struct device *dev, const struct ipa_clock_data *data)
{
struct ipa_clock *clock;
struct clk *clk;
return ERR_CAST(clk);
}
- ret = clk_set_rate(clk, IPA_CORE_CLOCK_RATE);
+ ret = clk_set_rate(clk, data->core_clock_rate);
if (ret) {
- dev_err(dev, "error %d setting core clock rate to %lu\n",
- ret, IPA_CORE_CLOCK_RATE);
+ dev_err(dev, "error %d setting core clock rate to %u\n",
+ ret, data->core_clock_rate);
goto err_clk_put;
}
goto err_clk_put;
}
clock->core = clk;
+ clock->interconnect_data = data->interconnect;
ret = ipa_interconnect_init(clock, dev);
if (ret)
struct device;
struct ipa;
+struct ipa_clock_data;
/**
* ipa_clock_rate() - Return the current IPA core clock rate
/**
* ipa_clock_init() - Initialize IPA clocking
* @dev: IPA device
+ * @data: Clock configuration data
*
* Return: A pointer to an ipa_clock structure, or a pointer-coded error
*/
-struct ipa_clock *ipa_clock_init(struct device *dev);
+struct ipa_clock *ipa_clock_init(struct device *dev,
+ const struct ipa_clock_data *data);
/**
* ipa_clock_exit() - Inverse of ipa_clock_init()
/* Some commands can wait until indicated pipeline stages are clear */
enum pipeline_clear_options {
- pipeline_clear_hps = 0,
- pipeline_clear_src_grp = 1,
- pipeline_clear_full = 2,
+ pipeline_clear_hps = 0x0,
+ pipeline_clear_src_grp = 0x1,
+ pipeline_clear_full = 0x2,
};
/* IPA_CMD_IP_V{4,6}_{FILTER,ROUTING}_INIT */
* a request is *not* an immediate command.
*/
enum ipa_cmd_opcode {
- IPA_CMD_NONE = 0,
- IPA_CMD_IP_V4_FILTER_INIT = 3,
- IPA_CMD_IP_V6_FILTER_INIT = 4,
- IPA_CMD_IP_V4_ROUTING_INIT = 7,
- IPA_CMD_IP_V6_ROUTING_INIT = 8,
- IPA_CMD_HDR_INIT_LOCAL = 9,
- IPA_CMD_REGISTER_WRITE = 12,
- IPA_CMD_IP_PACKET_INIT = 16,
- IPA_CMD_DMA_SHARED_MEM = 19,
- IPA_CMD_IP_PACKET_TAG_STATUS = 20,
+ IPA_CMD_NONE = 0x0,
+ IPA_CMD_IP_V4_FILTER_INIT = 0x3,
+ IPA_CMD_IP_V6_FILTER_INIT = 0x4,
+ IPA_CMD_IP_V4_ROUTING_INIT = 0x7,
+ IPA_CMD_IP_V6_ROUTING_INIT = 0x8,
+ IPA_CMD_HDR_INIT_LOCAL = 0x9,
+ IPA_CMD_REGISTER_WRITE = 0xc,
+ IPA_CMD_IP_PACKET_INIT = 0x10,
+ IPA_CMD_DMA_SHARED_MEM = 0x13,
+ IPA_CMD_IP_PACKET_TAG_STATUS = 0x14,
};
/**
enum dma_data_direction direction;
};
-
#ifdef IPA_VALIDATE
/**
.smem_size = 0x00002000,
};
+static struct ipa_clock_data ipa_clock_data = {
+ .core_clock_rate = 100 * 1000 * 1000, /* Hz */
+ /* Interconnect rates are in 1000 byte/second units */
+ .interconnect = {
+ [IPA_INTERCONNECT_MEMORY] = {
+ .peak_rate = 465000, /* 465 MBps */
+ .average_rate = 80000, /* 80 MBps */
+ },
+ /* Average rate is unused for the next two interconnects */
+ [IPA_INTERCONNECT_IMEM] = {
+ .peak_rate = 68570, /* 68.570 MBps */
+ .average_rate = 0, /* unused */
+ },
+ [IPA_INTERCONNECT_CONFIG] = {
+ .peak_rate = 30000, /* 30 MBps */
+ .average_rate = 0, /* unused */
+ },
+ },
+};
+
/* Configuration data for the SC7180 SoC. */
const struct ipa_data ipa_data_sc7180 = {
.version = IPA_VERSION_4_2,
.endpoint_data = ipa_gsi_endpoint_data,
.resource_data = &ipa_resource_data,
.mem_data = &ipa_mem_data,
+ .clock_data = &ipa_clock_data,
};
.smem_size = 0x00002000,
};
+static struct ipa_clock_data ipa_clock_data = {
+ .core_clock_rate = 75 * 1000 * 1000, /* Hz */
+ /* Interconnect rates are in 1000 byte/second units */
+ .interconnect = {
+ [IPA_INTERCONNECT_MEMORY] = {
+ .peak_rate = 600000, /* 600 MBps */
+ .average_rate = 80000, /* 80 MBps */
+ },
+ /* Average rate is unused for the next two interconnects */
+ [IPA_INTERCONNECT_IMEM] = {
+ .peak_rate = 350000, /* 350 MBps */
+ .average_rate = 0, /* unused */
+ },
+ [IPA_INTERCONNECT_CONFIG] = {
+ .peak_rate = 40000, /* 40 MBps */
+ .average_rate = 0, /* unused */
+ },
+ },
+};
+
/* Configuration data for the SDM845 SoC. */
const struct ipa_data ipa_data_sdm845 = {
.version = IPA_VERSION_3_5_1,
.endpoint_data = ipa_gsi_endpoint_data,
.resource_data = &ipa_resource_data,
.mem_data = &ipa_mem_data,
+ .clock_data = &ipa_clock_data,
};
};
/**
- * struct ipa_mem - description of IPA memory regions
+ * struct ipa_mem_data - description of IPA memory regions
* @local_count: number of regions defined in the local[] array
* @local: array of IPA-local memory region descriptors
* @imem_addr: physical address of IPA region within IMEM
u32 smem_size;
};
+/** enum ipa_interconnect_id - IPA interconnect identifier */
+enum ipa_interconnect_id {
+ IPA_INTERCONNECT_MEMORY,
+ IPA_INTERCONNECT_IMEM,
+ IPA_INTERCONNECT_CONFIG,
+ IPA_INTERCONNECT_COUNT, /* Last; not an interconnect */
+};
+
+/**
+ * struct ipa_interconnect_data - description of IPA interconnect rates
+ * @peak_rate: Peak interconnect bandwidth (in 1000 byte/sec units)
+ * @average_rate: Average interconnect bandwidth (in 1000 byte/sec units)
+ */
+struct ipa_interconnect_data {
+ u32 peak_rate;
+ u32 average_rate;
+};
+
+/**
+ * struct ipa_clock_data - description of IPA clock and interconnect rates
+ * @core_clock_rate: Core clock rate (Hz)
+ * @interconnect: Array of interconnect bandwidth parameters
+ */
+struct ipa_clock_data {
+ u32 core_clock_rate;
+ struct ipa_interconnect_data interconnect[IPA_INTERCONNECT_COUNT];
+};
+
/**
* struct ipa_data - combined IPA/GSI configuration data
* @version: IPA hardware version
const struct ipa_gsi_endpoint_data *endpoint_data;
const struct ipa_resource_data *resource_data;
const struct ipa_mem_data *mem_data;
+ const struct ipa_clock_data *clock_data;
};
extern const struct ipa_data ipa_data_sdm845;
#define IPA_ENDPOINT_QMAP_METADATA_MASK 0x000000ff /* host byte order */
#define IPA_ENDPOINT_RESET_AGGR_RETRY_MAX 3
-#define IPA_AGGR_TIME_LIMIT_DEFAULT 500 /* microseconds */
+#define IPA_AGGR_TIME_LIMIT 500 /* microseconds */
/** enum ipa_status_opcode - status element opcode hardware values */
enum ipa_status_opcode {
#ifdef IPA_VALIDATE
-static void ipa_endpoint_validate_build(void)
-{
- /* The aggregation byte limit defines the point at which an
- * aggregation window will close. It is programmed into the
- * IPA hardware as a number of KB. We don't use "hard byte
- * limit" aggregation, which means that we need to supply
- * enough space in a receive buffer to hold a complete MTU
- * plus normal skb overhead *after* that aggregation byte
- * limit has been crossed.
- *
- * This check just ensures we don't define a receive buffer
- * size that would exceed what we can represent in the field
- * that is used to program its size.
- */
- BUILD_BUG_ON(IPA_RX_BUFFER_SIZE >
- field_max(AGGR_BYTE_LIMIT_FMASK) * SZ_1K +
- IPA_MTU + IPA_RX_BUFFER_OVERHEAD);
-
- /* I honestly don't know where this requirement comes from. But
- * it holds, and if we someday need to loosen the constraint we
- * can try to track it down.
- */
- BUILD_BUG_ON(sizeof(struct ipa_status) % 4);
-}
-
static bool ipa_endpoint_data_valid_one(struct ipa *ipa, u32 count,
const struct ipa_gsi_endpoint_data *all_data,
const struct ipa_gsi_endpoint_data *data)
return true;
}
+static u32 aggr_byte_limit_max(enum ipa_version version)
+{
+ if (version < IPA_VERSION_4_5)
+ return field_max(aggr_byte_limit_fmask(true));
+
+ return field_max(aggr_byte_limit_fmask(false));
+}
+
static bool ipa_endpoint_data_valid(struct ipa *ipa, u32 count,
const struct ipa_gsi_endpoint_data *data)
{
const struct ipa_gsi_endpoint_data *dp = data;
struct device *dev = &ipa->pdev->dev;
enum ipa_endpoint_name name;
+ u32 limit;
- ipa_endpoint_validate_build();
+ /* Not sure where this constraint come from... */
+ BUILD_BUG_ON(sizeof(struct ipa_status) % 4);
if (count > IPA_ENDPOINT_COUNT) {
dev_err(dev, "too many endpoints specified (%u > %u)\n",
return false;
}
+ /* The aggregation byte limit defines the point at which an
+ * aggregation window will close. It is programmed into the
+ * IPA hardware as a number of KB. We don't use "hard byte
+ * limit" aggregation, which means that we need to supply
+ * enough space in a receive buffer to hold a complete MTU
+ * plus normal skb overhead *after* that aggregation byte
+ * limit has been crossed.
+ *
+ * This check ensures we don't define a receive buffer size
+ * that would exceed what we can represent in the field that
+ * is used to program its size.
+ */
+ limit = aggr_byte_limit_max(ipa->version) * SZ_1K;
+ limit += IPA_MTU + IPA_RX_BUFFER_OVERHEAD;
+ if (limit < IPA_RX_BUFFER_SIZE) {
+ dev_err(dev, "buffer size too big for aggregation (%u > %u)\n",
+ IPA_RX_BUFFER_SIZE, limit);
+ return false;
+ }
+
/* Make sure needed endpoints have defined data */
if (ipa_gsi_endpoint_data_empty(&data[IPA_ENDPOINT_AP_COMMAND_TX])) {
dev_err(dev, "command TX endpoint not defined\n");
static void ipa_endpoint_init_hdr(struct ipa_endpoint *endpoint)
{
u32 offset = IPA_REG_ENDP_INIT_HDR_N_OFFSET(endpoint->endpoint_id);
+ struct ipa *ipa = endpoint->ipa;
u32 val = 0;
if (endpoint->data->qmap) {
size_t header_size = sizeof(struct rmnet_map_header);
+ enum ipa_version version = ipa->version;
/* We might supply a checksum header after the QMAP header */
if (endpoint->toward_ipa && endpoint->data->checksum)
header_size += sizeof(struct rmnet_map_ul_csum_header);
- val |= u32_encode_bits(header_size, HDR_LEN_FMASK);
+ val |= ipa_header_size_encoded(version, header_size);
/* Define how to fill fields in a received QMAP header */
if (!endpoint->toward_ipa) {
- u32 off; /* Field offset within header */
+ u32 offset; /* Field offset within header */
/* Where IPA will write the metadata value */
- off = offsetof(struct rmnet_map_header, mux_id);
- val |= u32_encode_bits(off, HDR_OFST_METADATA_FMASK);
+ offset = offsetof(struct rmnet_map_header, mux_id);
+ val |= ipa_metadata_offset_encoded(version, offset);
/* Where IPA will write the length */
- off = offsetof(struct rmnet_map_header, pkt_len);
+ offset = offsetof(struct rmnet_map_header, pkt_len);
+ /* Upper bits are stored in HDR_EXT with IPA v4.5 */
+ if (version == IPA_VERSION_4_5)
+ offset &= field_mask(HDR_OFST_PKT_SIZE_FMASK);
+
val |= HDR_OFST_PKT_SIZE_VALID_FMASK;
- val |= u32_encode_bits(off, HDR_OFST_PKT_SIZE_FMASK);
+ val |= u32_encode_bits(offset, HDR_OFST_PKT_SIZE_FMASK);
}
/* For QMAP TX, metadata offset is 0 (modem assumes this) */
val |= HDR_OFST_METADATA_VALID_FMASK;
/* HDR_ADDITIONAL_CONST_LEN is 0; (RX only) */
/* HDR_A5_MUX is 0 */
/* HDR_LEN_INC_DEAGG_HDR is 0 */
- /* HDR_METADATA_REG_VALID is 0 (TX only) */
+ /* HDR_METADATA_REG_VALID is 0 (TX only, version < v4.5) */
}
- iowrite32(val, endpoint->ipa->reg_virt + offset);
+ iowrite32(val, ipa->reg_virt + offset);
}
static void ipa_endpoint_init_hdr_ext(struct ipa_endpoint *endpoint)
{
u32 offset = IPA_REG_ENDP_INIT_HDR_EXT_N_OFFSET(endpoint->endpoint_id);
u32 pad_align = endpoint->data->rx.pad_align;
+ struct ipa *ipa = endpoint->ipa;
u32 val = 0;
val |= HDR_ENDIANNESS_FMASK; /* big endian */
if (!endpoint->toward_ipa)
val |= u32_encode_bits(pad_align, HDR_PAD_TO_ALIGNMENT_FMASK);
- iowrite32(val, endpoint->ipa->reg_virt + offset);
+ /* IPA v4.5 adds some most-significant bits to a few fields,
+ * two of which are defined in the HDR (not HDR_EXT) register.
+ */
+ if (ipa->version == IPA_VERSION_4_5) {
+ /* HDR_TOTAL_LEN_OR_PAD_OFFSET is 0, so MSB is 0 */
+ if (endpoint->data->qmap && !endpoint->toward_ipa) {
+ u32 offset;
+
+ offset = offsetof(struct rmnet_map_header, pkt_len);
+ offset >>= hweight32(HDR_OFST_PKT_SIZE_FMASK);
+ val |= u32_encode_bits(offset,
+ HDR_OFST_PKT_SIZE_MSB_FMASK);
+ /* HDR_ADDITIONAL_CONST_LEN is 0 so MSB is 0 */
+ }
+ }
+ iowrite32(val, ipa->reg_virt + offset);
}
-
static void ipa_endpoint_init_hdr_metadata_mask(struct ipa_endpoint *endpoint)
{
u32 endpoint_id = endpoint->endpoint_id;
return rx_buffer_size / SZ_1K;
}
+/* Encoded values for AGGR endpoint register fields */
+static u32 aggr_byte_limit_encoded(enum ipa_version version, u32 limit)
+{
+ if (version < IPA_VERSION_4_5)
+ return u32_encode_bits(limit, aggr_byte_limit_fmask(true));
+
+ return u32_encode_bits(limit, aggr_byte_limit_fmask(false));
+}
+
+/* Encode the aggregation timer limit (microseconds) based on IPA version */
+static u32 aggr_time_limit_encoded(enum ipa_version version, u32 limit)
+{
+ u32 gran_sel;
+ u32 fmask;
+ u32 val;
+
+ if (version < IPA_VERSION_4_5) {
+ /* We set aggregation granularity in ipa_hardware_config() */
+ limit = DIV_ROUND_CLOSEST(limit, IPA_AGGR_GRANULARITY);
+
+ return u32_encode_bits(limit, aggr_time_limit_fmask(true));
+ }
+
+ /* IPA v4.5 expresses the time limit using Qtime. The AP has
+ * pulse generators 0 and 1 available, which were configured
+ * in ipa_qtime_config() to have granularity 100 usec and
+ * 1 msec, respectively. Use pulse generator 0 if possible,
+ * otherwise fall back to pulse generator 1.
+ */
+ fmask = aggr_time_limit_fmask(false);
+ val = DIV_ROUND_CLOSEST(limit, 100);
+ if (val > field_max(fmask)) {
+ /* Have to use pulse generator 1 (millisecond granularity) */
+ gran_sel = AGGR_GRAN_SEL_FMASK;
+ val = DIV_ROUND_CLOSEST(limit, 1000);
+ } else {
+ /* We can use pulse generator 0 (100 usec granularity) */
+ gran_sel = 0;
+ }
+
+ return gran_sel | u32_encode_bits(val, fmask);
+}
+
+static u32 aggr_sw_eof_active_encoded(enum ipa_version version, bool enabled)
+{
+ u32 val = enabled ? 1 : 0;
+
+ if (version < IPA_VERSION_4_5)
+ return u32_encode_bits(val, aggr_sw_eof_active_fmask(true));
+
+ return u32_encode_bits(val, aggr_sw_eof_active_fmask(false));
+}
+
static void ipa_endpoint_init_aggr(struct ipa_endpoint *endpoint)
{
u32 offset = IPA_REG_ENDP_INIT_AGGR_N_OFFSET(endpoint->endpoint_id);
+ enum ipa_version version = endpoint->ipa->version;
u32 val = 0;
if (endpoint->data->aggregation) {
if (!endpoint->toward_ipa) {
+ bool close_eof;
u32 limit;
val |= u32_encode_bits(IPA_ENABLE_AGGR, AGGR_EN_FMASK);
val |= u32_encode_bits(IPA_GENERIC, AGGR_TYPE_FMASK);
limit = ipa_aggr_size_kb(IPA_RX_BUFFER_SIZE);
- val |= u32_encode_bits(limit, AGGR_BYTE_LIMIT_FMASK);
+ val |= aggr_byte_limit_encoded(version, limit);
- limit = IPA_AGGR_TIME_LIMIT_DEFAULT;
- limit = DIV_ROUND_CLOSEST(limit, IPA_AGGR_GRANULARITY);
- val |= u32_encode_bits(limit, AGGR_TIME_LIMIT_FMASK);
+ limit = IPA_AGGR_TIME_LIMIT;
+ val |= aggr_time_limit_encoded(version, limit);
/* AGGR_PKT_LIMIT is 0 (unlimited) */
- if (endpoint->data->rx.aggr_close_eof)
- val |= AGGR_SW_EOF_ACTIVE_FMASK;
+ close_eof = endpoint->data->rx.aggr_close_eof;
+ val |= aggr_sw_eof_active_encoded(version, close_eof);
+
/* AGGR_HARD_BYTE_LIMIT_ENABLE is 0 */
} else {
val |= u32_encode_bits(IPA_ENABLE_DEAGGR,
/* other fields ignored */
}
/* AGGR_FORCE_CLOSE is 0 */
+ /* AGGR_GRAN_SEL is 0 for IPA v4.5 */
} else {
val |= u32_encode_bits(IPA_BYPASS_AGGR, AGGR_EN_FMASK);
/* other fields ignored */
iowrite32(val, endpoint->ipa->reg_virt + offset);
}
-/* The head-of-line blocking timer is defined as a tick count, where each
- * tick represents 128 cycles of the IPA core clock. Return the value
- * that should be written to that register that represents the timeout
- * period provided.
+/* Return the Qtime-based head-of-line blocking timer value that
+ * represents the given number of microseconds. The result
+ * includes both the timer value and the selected timer granularity.
*/
-static u32 ipa_reg_init_hol_block_timer_val(struct ipa *ipa, u32 microseconds)
+static u32 hol_block_timer_qtime_val(struct ipa *ipa, u32 microseconds)
+{
+ u32 gran_sel;
+ u32 val;
+
+ /* IPA v4.5 expresses time limits using Qtime. The AP has
+ * pulse generators 0 and 1 available, which were configured
+ * in ipa_qtime_config() to have granularity 100 usec and
+ * 1 msec, respectively. Use pulse generator 0 if possible,
+ * otherwise fall back to pulse generator 1.
+ */
+ val = DIV_ROUND_CLOSEST(microseconds, 100);
+ if (val > field_max(TIME_LIMIT_FMASK)) {
+ /* Have to use pulse generator 1 (millisecond granularity) */
+ gran_sel = GRAN_SEL_FMASK;
+ val = DIV_ROUND_CLOSEST(microseconds, 1000);
+ } else {
+ /* We can use pulse generator 0 (100 usec granularity) */
+ gran_sel = 0;
+ }
+
+ return gran_sel | u32_encode_bits(val, TIME_LIMIT_FMASK);
+}
+
+/* The head-of-line blocking timer is defined as a tick count. For
+ * IPA version 4.5 the tick count is based on the Qtimer, which is
+ * derived from the 19.2 MHz SoC XO clock. For older IPA versions
+ * each tick represents 128 cycles of the IPA core clock.
+ *
+ * Return the encoded value that should be written to that register
+ * that represents the timeout period provided. For IPA v4.2 this
+ * encodes a base and scale value, while for earlier versions the
+ * value is a simple tick count.
+ */
+static u32 hol_block_timer_val(struct ipa *ipa, u32 microseconds)
{
u32 width;
u32 scale;
if (!microseconds)
return 0; /* Nothing to compute if timer period is 0 */
+ if (ipa->version == IPA_VERSION_4_5)
+ return hol_block_timer_qtime_val(ipa, microseconds);
+
/* Use 64 bit arithmetic to avoid overflow... */
rate = ipa_clock_rate(ipa);
ticks = DIV_ROUND_CLOSEST(microseconds * rate, 128 * USEC_PER_SEC);
/* ...but we still need to fit into a 32-bit register */
WARN_ON(ticks > U32_MAX);
- /* IPA v3.5.1 just records the tick count */
- if (ipa->version == IPA_VERSION_3_5_1)
+ /* IPA v3.5.1 through v4.1 just record the tick count */
+ if (ipa->version < IPA_VERSION_4_2)
return (u32)ticks;
/* For IPA v4.2, the tick count is represented by base and
u32 val;
offset = IPA_REG_ENDP_INIT_HOL_BLOCK_TIMER_N_OFFSET(endpoint_id);
- val = ipa_reg_init_hol_block_timer_val(ipa, microseconds);
+ val = hol_block_timer_val(ipa, microseconds);
iowrite32(val, ipa->reg_virt + offset);
}
val |= u32_encode_bits(status_endpoint_id,
STATUS_ENDP_FMASK);
}
- /* STATUS_LOCATION is 0 (status element precedes packet) */
- /* The next field is present for IPA v4.0 and above */
- /* STATUS_PKT_SUPPRESS_FMASK is 0 */
+ /* STATUS_LOCATION is 0, meaning status element precedes
+ * packet (not present for IPA v4.5)
+ */
+ /* STATUS_PKT_SUPPRESS_FMASK is 0 (not present for v3.5.1) */
}
iowrite32(val, ipa->reg_virt + offset);
val = ioread32(ipa->reg_virt + IPA_REG_FLAVOR_0_OFFSET);
/* Our RX is an IPA producer */
- rx_base = u32_get_bits(val, BAM_PROD_LOWEST_FMASK);
- max = rx_base + u32_get_bits(val, BAM_MAX_PROD_PIPES_FMASK);
+ rx_base = u32_get_bits(val, IPA_PROD_LOWEST_FMASK);
+ max = rx_base + u32_get_bits(val, IPA_MAX_PROD_PIPES_FMASK);
if (max > IPA_ENDPOINT_MAX) {
dev_err(dev, "too many endpoints (%u > %u)\n",
max, IPA_ENDPOINT_MAX);
rx_mask = GENMASK(max - 1, rx_base);
/* Our TX is an IPA consumer */
- max = u32_get_bits(val, BAM_MAX_CONS_PIPES_FMASK);
+ max = u32_get_bits(val, IPA_MAX_CONS_PIPES_FMASK);
tx_mask = GENMASK(max - 1, 0);
ipa->available = rx_mask | tx_mask;
#define IPA_MTU ETH_DATA_LEN
enum ipa_endpoint_name {
- IPA_ENDPOINT_AP_MODEM_TX = 0,
+ IPA_ENDPOINT_AP_MODEM_TX,
IPA_ENDPOINT_MODEM_LAN_TX,
IPA_ENDPOINT_MODEM_COMMAND_TX,
IPA_ENDPOINT_AP_COMMAND_TX,
u32 val;
/* assert(mask & ipa->available); */
- val = ioread32(ipa->reg_virt + IPA_REG_SUSPEND_IRQ_EN_OFFSET);
+ val = ioread32(ipa->reg_virt + IPA_REG_IRQ_SUSPEND_EN_OFFSET);
if (enable)
val |= mask;
else
val &= ~mask;
- iowrite32(val, ipa->reg_virt + IPA_REG_SUSPEND_IRQ_EN_OFFSET);
+ iowrite32(val, ipa->reg_virt + IPA_REG_IRQ_SUSPEND_EN_OFFSET);
}
/* Enable TX_SUSPEND for an endpoint */
u32 val;
val = ioread32(ipa->reg_virt + IPA_REG_IRQ_SUSPEND_INFO_OFFSET);
- iowrite32(val, ipa->reg_virt + IPA_REG_SUSPEND_IRQ_CLR_OFFSET);
+ iowrite32(val, ipa->reg_virt + IPA_REG_IRQ_SUSPEND_CLR_OFFSET);
}
/* Simulate arrival of an IPA TX_SUSPEND interrupt */
struct ipa;
struct ipa_interrupt;
-/**
- * enum ipa_irq_id - IPA interrupt type
- * @IPA_IRQ_UC_0: Microcontroller event interrupt
- * @IPA_IRQ_UC_1: Microcontroller response interrupt
- * @IPA_IRQ_TX_SUSPEND: Data ready interrupt
- *
- * The data ready interrupt is signaled if data has arrived that is destined
- * for an AP RX endpoint whose underlying GSI channel is suspended/stopped.
- */
-enum ipa_irq_id {
- IPA_IRQ_UC_0 = 2,
- IPA_IRQ_UC_1 = 3,
- IPA_IRQ_TX_SUSPEND = 14,
- IPA_IRQ_COUNT, /* Number of interrupt types (not an index) */
-};
-
/**
* typedef ipa_irq_handler_t - IPA interrupt handler function type
* @ipa: IPA pointer
#define IPA_FWS_PATH "ipa_fws.mdt"
#define IPA_PAS_ID 15
+/* Shift of 19.2 MHz timestamp to achieve lower resolution timestamps */
+#define DPL_TIMESTAMP_SHIFT 14 /* ~1.172 kHz, ~853 usec per tick */
+#define TAG_TIMESTAMP_SHIFT 14
+#define NAT_TIMESTAMP_SHIFT 24 /* ~1.144 Hz, ~874 msec per tick */
+
+/* Divider for 19.2 MHz crystal oscillator clock to get common timer clock */
+#define IPA_XO_CLOCK_DIVIDER 192 /* 1 is subtracted where used */
+
/**
* ipa_suspend_handler() - Handle the suspend IPA interrupt
* @ipa: IPA pointer
val &= ~IPA_QMB_SELECT_CONS_EN_FMASK;
val &= ~IPA_QMB_SELECT_PROD_EN_FMASK;
val &= ~IPA_QMB_SELECT_GLOBAL_EN_FMASK;
- } else {
+ } else if (ipa->version < IPA_VERSION_4_5) {
val |= GSI_MULTI_AXI_MASTERS_DIS_FMASK;
+ } else {
+ /* For IPA v4.5 IPA_FULL_FLUSH_WAIT_RSC_CLOSE_EN is 0 */
}
val |= GSI_MULTI_INORDER_RD_DIS_FMASK;
/* Configure DDR and PCIe max read/write QSB values */
static void ipa_hardware_config_qsb(struct ipa *ipa)
{
+ enum ipa_version version = ipa->version;
+ u32 max0;
+ u32 max1;
u32 val;
- /* QMB_0 represents DDR; QMB_1 represents PCIe (not present in 4.2) */
+ /* QMB_0 represents DDR; QMB_1 represents PCIe */
val = u32_encode_bits(8, GEN_QMB_0_MAX_WRITES_FMASK);
- if (ipa->version == IPA_VERSION_4_2)
- val |= u32_encode_bits(0, GEN_QMB_1_MAX_WRITES_FMASK);
- else
- val |= u32_encode_bits(4, GEN_QMB_1_MAX_WRITES_FMASK);
+ switch (version) {
+ case IPA_VERSION_4_2:
+ max1 = 0; /* PCIe not present */
+ break;
+ case IPA_VERSION_4_5:
+ max1 = 8;
+ break;
+ default:
+ max1 = 4;
+ break;
+ }
+ val |= u32_encode_bits(max1, GEN_QMB_1_MAX_WRITES_FMASK);
iowrite32(val, ipa->reg_virt + IPA_REG_QSB_MAX_WRITES_OFFSET);
- if (ipa->version == IPA_VERSION_3_5_1) {
- val = u32_encode_bits(8, GEN_QMB_0_MAX_READS_FMASK);
- val |= u32_encode_bits(12, GEN_QMB_1_MAX_READS_FMASK);
- } else {
- val = u32_encode_bits(12, GEN_QMB_0_MAX_READS_FMASK);
- if (ipa->version == IPA_VERSION_4_2)
- val |= u32_encode_bits(0, GEN_QMB_1_MAX_READS_FMASK);
- else
- val |= u32_encode_bits(12, GEN_QMB_1_MAX_READS_FMASK);
+ max1 = 12;
+ switch (version) {
+ case IPA_VERSION_3_5_1:
+ max0 = 8;
+ break;
+ case IPA_VERSION_4_0:
+ case IPA_VERSION_4_1:
+ max0 = 12;
+ break;
+ case IPA_VERSION_4_2:
+ max0 = 12;
+ max1 = 0; /* PCIe not present */
+ break;
+ case IPA_VERSION_4_5:
+ max0 = 0; /* No limit (hardware maximum) */
+ break;
+ }
+ val = u32_encode_bits(max0, GEN_QMB_0_MAX_READS_FMASK);
+ val |= u32_encode_bits(max1, GEN_QMB_1_MAX_READS_FMASK);
+ if (version != IPA_VERSION_3_5_1) {
/* GEN_QMB_0_MAX_READS_BEATS is 0 */
/* GEN_QMB_1_MAX_READS_BEATS is 0 */
}
iowrite32(val, ipa->reg_virt + IPA_REG_QSB_MAX_READS_OFFSET);
}
+/* IPA uses unified Qtime starting at IPA v4.5, implementing various
+ * timestamps and timers independent of the IPA core clock rate. The
+ * Qtimer is based on a 56-bit timestamp incremented at each tick of
+ * a 19.2 MHz SoC crystal oscillator (XO clock).
+ *
+ * For IPA timestamps (tag, NAT, data path logging) a lower resolution
+ * timestamp is achieved by shifting the Qtimer timestamp value right
+ * some number of bits to produce the low-order bits of the coarser
+ * granularity timestamp.
+ *
+ * For timers, a common timer clock is derived from the XO clock using
+ * a divider (we use 192, to produce a 100kHz timer clock). From
+ * this common clock, three "pulse generators" are used to produce
+ * timer ticks at a configurable frequency. IPA timers (such as
+ * those used for aggregation or head-of-line block handling) now
+ * define their period based on one of these pulse generators.
+ */
+static void ipa_qtime_config(struct ipa *ipa)
+{
+ u32 val;
+
+ /* Timer clock divider must be disabled when we change the rate */
+ iowrite32(0, ipa->reg_virt + IPA_REG_TIMERS_XO_CLK_DIV_CFG_OFFSET);
+
+ /* Set DPL time stamp resolution to use Qtime (instead of 1 msec) */
+ val = u32_encode_bits(DPL_TIMESTAMP_SHIFT, DPL_TIMESTAMP_LSB_FMASK);
+ val |= u32_encode_bits(1, DPL_TIMESTAMP_SEL_FMASK);
+ /* Configure tag and NAT Qtime timestamp resolution as well */
+ val |= u32_encode_bits(TAG_TIMESTAMP_SHIFT, TAG_TIMESTAMP_LSB_FMASK);
+ val |= u32_encode_bits(NAT_TIMESTAMP_SHIFT, NAT_TIMESTAMP_LSB_FMASK);
+ iowrite32(val, ipa->reg_virt + IPA_REG_QTIME_TIMESTAMP_CFG_OFFSET);
+
+ /* Set granularity of pulse generators used for other timers */
+ val = u32_encode_bits(IPA_GRAN_100_US, GRAN_0_FMASK);
+ val |= u32_encode_bits(IPA_GRAN_1_MS, GRAN_1_FMASK);
+ val |= u32_encode_bits(IPA_GRAN_1_MS, GRAN_2_FMASK);
+ iowrite32(val, ipa->reg_virt + IPA_REG_TIMERS_PULSE_GRAN_CFG_OFFSET);
+
+ /* Actual divider is 1 more than value supplied here */
+ val = u32_encode_bits(IPA_XO_CLOCK_DIVIDER - 1, DIV_VALUE_FMASK);
+ iowrite32(val, ipa->reg_virt + IPA_REG_TIMERS_XO_CLK_DIV_CFG_OFFSET);
+
+ /* Divider value is set; re-enable the common timer clock divider */
+ val |= u32_encode_bits(1, DIV_ENABLE_FMASK);
+ iowrite32(val, ipa->reg_virt + IPA_REG_TIMERS_XO_CLK_DIV_CFG_OFFSET);
+}
+
static void ipa_idle_indication_cfg(struct ipa *ipa,
u32 enter_idle_debounce_thresh,
bool const_non_idle_enable)
*/
static void ipa_hardware_dcd_config(struct ipa *ipa)
{
- /* Recommended values for IPA 3.5 according to IPA HPG */
+ /* Recommended values for IPA 3.5 and later according to IPA HPG */
ipa_idle_indication_cfg(ipa, 256, false);
}
*/
static void ipa_hardware_config(struct ipa *ipa)
{
+ enum ipa_version version = ipa->version;
u32 granularity;
u32 val;
- /* Fill in backward-compatibility register, based on version */
- val = ipa_reg_bcr_val(ipa->version);
- iowrite32(val, ipa->reg_virt + IPA_REG_BCR_OFFSET);
+ /* IPA v4.5 has no backward compatibility register */
+ if (version < IPA_VERSION_4_5) {
+ val = ipa_reg_bcr_val(version);
+ iowrite32(val, ipa->reg_virt + IPA_REG_BCR_OFFSET);
+ }
- if (ipa->version != IPA_VERSION_3_5_1) {
- /* Enable open global clocks (hardware workaround) */
+ /* Implement some hardware workarounds */
+ if (version != IPA_VERSION_3_5_1 && version < IPA_VERSION_4_5) {
+ /* Enable open global clocks (not needed for IPA v4.5) */
val = GLOBAL_FMASK;
val |= GLOBAL_2X_CLK_FMASK;
iowrite32(val, ipa->reg_virt + IPA_REG_CLKON_CFG_OFFSET);
- /* Disable PA mask to allow HOLB drop (hardware workaround) */
+ /* Disable PA mask to allow HOLB drop */
val = ioread32(ipa->reg_virt + IPA_REG_TX_CFG_OFFSET);
- val &= ~PA_MASK_EN;
+ val &= ~PA_MASK_EN_FMASK;
iowrite32(val, ipa->reg_virt + IPA_REG_TX_CFG_OFFSET);
}
/* Configure system bus limits */
ipa_hardware_config_qsb(ipa);
- /* Configure aggregation granularity */
- granularity = ipa_aggr_granularity_val(IPA_AGGR_GRANULARITY);
- val = u32_encode_bits(granularity, AGGR_GRANULARITY);
- iowrite32(val, ipa->reg_virt + IPA_REG_COUNTER_CFG_OFFSET);
+ if (version < IPA_VERSION_4_5) {
+ /* Configure aggregation timer granularity */
+ granularity = ipa_aggr_granularity_val(IPA_AGGR_GRANULARITY);
+ val = u32_encode_bits(granularity, AGGR_GRANULARITY_FMASK);
+ iowrite32(val, ipa->reg_virt + IPA_REG_COUNTER_CFG_OFFSET);
+ } else {
+ ipa_qtime_config(ipa);
+ }
- /* Disable hashed IPv4 and IPv6 routing and filtering for IPA v4.2 */
- if (ipa->version == IPA_VERSION_4_2)
- iowrite32(0, ipa->reg_virt + IPA_REG_FILT_ROUT_HASH_EN_OFFSET);
+ /* IPA v4.2 does not support hashed tables, so disable them */
+ if (version == IPA_VERSION_4_2) {
+ u32 offset = ipa_reg_filt_rout_hash_en_offset(version);
+
+ iowrite32(0, ipa->reg_virt + offset);
+ }
/* Enable dynamic clock division */
ipa_hardware_dcd_config(ipa);
/* Aggregation granularity value can't be 0, and must fit */
BUILD_BUG_ON(!ipa_aggr_granularity_val(IPA_AGGR_GRANULARITY));
BUILD_BUG_ON(ipa_aggr_granularity_val(IPA_AGGR_GRANULARITY) >
- field_max(AGGR_GRANULARITY));
+ field_max(AGGR_GRANULARITY_FMASK));
#endif /* IPA_VALIDATE */
}
ipa_validate_build();
+ /* Get configuration data early; needed for clock initialization */
+ data = of_device_get_match_data(dev);
+ if (!data) {
+ /* This is really IPA_VALIDATE (should never happen) */
+ dev_err(dev, "matched hardware not supported\n");
+ return -ENODEV;
+ }
+
/* If we need Trust Zone, make sure it's available */
modem_init = of_property_read_bool(dev->of_node, "modem-init");
if (!modem_init)
/* The clock and interconnects might not be ready when we're
* probed, so might return -EPROBE_DEFER.
*/
- clock = ipa_clock_init(dev);
+ clock = ipa_clock_init(dev, data->clock_data);
if (IS_ERR(clock)) {
ret = PTR_ERR(clock);
goto err_rproc_put;
}
- /* No more EPROBE_DEFER. Get our configuration data */
- data = of_device_get_match_data(dev);
- if (!data) {
- /* This is really IPA_VALIDATE (should never happen) */
- dev_err(dev, "matched hardware not supported\n");
- ret = -ENOTSUPP;
- goto err_clock_exit;
- }
-
- /* Allocate and initialize the IPA structure */
+ /* No more EPROBE_DEFER. Allocate and initialize the IPA structure */
ipa = kzalloc(sizeof(*ipa), GFP_KERNEL);
if (!ipa) {
ret = -ENOMEM;
if (ipa->setup_complete) {
ret = ipa_modem_stop(ipa);
+ /* If starting or stopping is in progress, try once more */
+ if (ret == -EBUSY) {
+ usleep_range(USEC_PER_MSEC, 2 * USEC_PER_MSEC);
+ ret = ipa_modem_stop(ipa);
+ }
if (ret)
return ret;
return 0;
}
+static void ipa_shutdown(struct platform_device *pdev)
+{
+ int ret;
+
+ ret = ipa_remove(pdev);
+ if (ret)
+ dev_err(&pdev->dev, "shutdown: remove returned %d\n", ret);
+}
+
/**
* ipa_suspend() - Power management system suspend callback
* @dev: IPA device structure
};
static struct platform_driver ipa_driver = {
- .probe = ipa_probe,
- .remove = ipa_remove,
+ .probe = ipa_probe,
+ .remove = ipa_remove,
+ .shutdown = ipa_shutdown,
.driver = {
.name = "ipa",
.pm = &ipa_pm_ops,
ipa_qmi->indication_sent = false;
}
-static struct qmi_ops ipa_server_ops = {
+static const struct qmi_ops ipa_server_ops = {
.bye = ipa_server_bye,
};
}
/* The server handles two request message types sent by the modem. */
-static struct qmi_msg_handler ipa_server_msg_handlers[] = {
+static const struct qmi_msg_handler ipa_server_msg_handlers[] = {
{
.type = QMI_REQUEST,
.msg_id = IPA_QMI_INDICATION_REGISTER,
}
/* The client handles one response message type sent by the modem. */
-static struct qmi_msg_handler ipa_client_msg_handlers[] = {
+static const struct qmi_msg_handler ipa_client_msg_handlers[] = {
{
.type = QMI_RESPONSE,
.msg_id = IPA_QMI_INIT_DRIVER,
return 0;
}
-static struct qmi_ops ipa_client_ops = {
+static const struct qmi_ops ipa_client_ops = {
.new_server = ipa_client_new_server,
};
/* The AP tells the modem its platform type. We assume Android. */
enum ipa_platform_type {
- IPA_QMI_PLATFORM_TYPE_INVALID = 0, /* Invalid */
- IPA_QMI_PLATFORM_TYPE_TN = 1, /* Data card */
- IPA_QMI_PLATFORM_TYPE_LE = 2, /* Data router */
- IPA_QMI_PLATFORM_TYPE_MSM_ANDROID = 3, /* Android MSM */
- IPA_QMI_PLATFORM_TYPE_MSM_WINDOWS = 4, /* Windows MSM */
- IPA_QMI_PLATFORM_TYPE_MSM_QNX_V01 = 5, /* QNX MSM */
+ IPA_QMI_PLATFORM_TYPE_INVALID = 0x0, /* Invalid */
+ IPA_QMI_PLATFORM_TYPE_TN = 0x1, /* Data card */
+ IPA_QMI_PLATFORM_TYPE_LE = 0x2, /* Data router */
+ IPA_QMI_PLATFORM_TYPE_MSM_ANDROID = 0x3, /* Android MSM */
+ IPA_QMI_PLATFORM_TYPE_MSM_WINDOWS = 0x4, /* Windows MSM */
+ IPA_QMI_PLATFORM_TYPE_MSM_QNX_V01 = 0x5, /* QNX MSM */
};
/* This defines the start and end offset of a range of memory. Both
* of valid bits for the register.
*/
-#define IPA_REG_ENABLED_PIPES_OFFSET 0x00000038
-
#define IPA_REG_COMP_CFG_OFFSET 0x0000003c
+/* The next field is not supported for IPA v4.1 */
#define ENABLE_FMASK GENMASK(0, 0)
#define GSI_SNOC_BYPASS_DIS_FMASK GENMASK(1, 1)
#define GEN_QMB_0_SNOC_BYPASS_DIS_FMASK GENMASK(2, 2)
#define GEN_QMB_1_SNOC_BYPASS_DIS_FMASK GENMASK(3, 3)
+/* The next field is not present for IPA v4.5 */
#define IPA_DCMP_FAST_CLK_EN_FMASK GENMASK(4, 4)
+/* The remaining fields are not present for IPA v3.5.1 */
#define IPA_QMB_SELECT_CONS_EN_FMASK GENMASK(5, 5)
#define IPA_QMB_SELECT_PROD_EN_FMASK GENMASK(6, 6)
#define GSI_MULTI_INORDER_RD_DIS_FMASK GENMASK(7, 7)
#define GSI_MULTI_AXI_MASTERS_DIS_FMASK GENMASK(15, 15)
#define IPA_QMB_SELECT_GLOBAL_EN_FMASK GENMASK(16, 16)
#define IPA_ATOMIC_FETCHER_ARB_LOCK_DIS_FMASK GENMASK(20, 17)
+/* The next field is present for IPA v4.5 */
+#define IPA_FULL_FLUSH_WAIT_RSC_CLOSE_EN_FMASK GENMASK(21, 21)
#define IPA_REG_CLKON_CFG_OFFSET 0x00000044
#define RX_FMASK GENMASK(0, 0)
#define ACK_MNGR_FMASK GENMASK(14, 14)
#define D_DCPH_FMASK GENMASK(15, 15)
#define H_DCPH_FMASK GENMASK(16, 16)
+/* The next field is not present for IPA v4.5 */
#define DCMP_FMASK GENMASK(17, 17)
#define NTF_TX_CMDQS_FMASK GENMASK(18, 18)
#define TX_0_FMASK GENMASK(19, 19)
#define TX_1_FMASK GENMASK(20, 20)
#define FNR_FMASK GENMASK(21, 21)
+/* The remaining fields are not present for IPA v3.5.1 */
#define QSB2AXI_CMDQ_L_FMASK GENMASK(22, 22)
#define AGGR_WRAPPER_FMASK GENMASK(23, 23)
#define RAM_SLAVEWAY_FMASK GENMASK(24, 24)
#define GSI_IF_FMASK GENMASK(27, 27)
#define GLOBAL_FMASK GENMASK(28, 28)
#define GLOBAL_2X_CLK_FMASK GENMASK(29, 29)
+/* The next field is present for IPA v4.5 */
+#define DPL_FIFO_FMASK GENMASK(30, 30)
#define IPA_REG_ROUTE_OFFSET 0x00000048
#define ROUTE_DIS_FMASK GENMASK(0, 0)
#define IPA_REG_QSB_MAX_READS_OFFSET 0x00000078
#define GEN_QMB_0_MAX_READS_FMASK GENMASK(3, 0)
#define GEN_QMB_1_MAX_READS_FMASK GENMASK(7, 4)
-/* The next two fields are present for IPA v4.0 and above */
+/* The next two fields are not present for IPA v3.5.1 */
#define GEN_QMB_0_MAX_READS_BEATS_FMASK GENMASK(23, 16)
#define GEN_QMB_1_MAX_READS_BEATS_FMASK GENMASK(31, 24)
-static inline u32 ipa_reg_state_aggr_active_offset(enum ipa_version version)
+static inline u32 ipa_reg_filt_rout_hash_en_offset(enum ipa_version version)
{
if (version == IPA_VERSION_3_5_1)
- return 0x0000010c;
+ return 0x000008c;
- return 0x000000b4;
+ return 0x0000148;
}
-/* ipa->available defines the valid bits in the STATE_AGGR_ACTIVE register */
-
-/* The next register is present for IPA v4.2 and above */
-#define IPA_REG_FILT_ROUT_HASH_EN_OFFSET 0x00000148
-#define IPV6_ROUTER_HASH_EN GENMASK(0, 0)
-#define IPV6_FILTER_HASH_EN GENMASK(4, 4)
-#define IPV4_ROUTER_HASH_EN GENMASK(8, 8)
-#define IPV4_FILTER_HASH_EN GENMASK(12, 12)
static inline u32 ipa_reg_filt_rout_hash_flush_offset(enum ipa_version version)
{
return 0x000014c;
}
-#define IPV6_ROUTER_HASH_FLUSH GENMASK(0, 0)
-#define IPV6_FILTER_HASH_FLUSH GENMASK(4, 4)
-#define IPV4_ROUTER_HASH_FLUSH GENMASK(8, 8)
-#define IPV4_FILTER_HASH_FLUSH GENMASK(12, 12)
+/* The next four fields are used for the hash enable and flush registers */
+#define IPV6_ROUTER_HASH_FMASK GENMASK(0, 0)
+#define IPV6_FILTER_HASH_FMASK GENMASK(4, 4)
+#define IPV4_ROUTER_HASH_FMASK GENMASK(8, 8)
+#define IPV4_FILTER_HASH_FMASK GENMASK(12, 12)
+
+/* ipa->available defines the valid bits in the STATE_AGGR_ACTIVE register */
+static inline u32 ipa_reg_state_aggr_active_offset(enum ipa_version version)
+{
+ if (version == IPA_VERSION_3_5_1)
+ return 0x0000010c;
+
+ return 0x000000b4;
+}
+/* The next register is not present for IPA v4.5 */
#define IPA_REG_BCR_OFFSET 0x000001d0
-#define BCR_CMDQ_L_LACK_ONE_ENTRY BIT(0)
-#define BCR_TX_NOT_USING_BRESP BIT(1)
-#define BCR_SUSPEND_L2_IRQ BIT(3)
-#define BCR_HOLB_DROP_L2_IRQ BIT(4)
-#define BCR_DUAL_TX BIT(5)
+/* The next two fields are not present for IPA v4.2 */
+#define BCR_CMDQ_L_LACK_ONE_ENTRY_FMASK GENMASK(0, 0)
+#define BCR_TX_NOT_USING_BRESP_FMASK GENMASK(1, 1)
+/* The next field is invalid for IPA v4.1 */
+#define BCR_TX_SUSPEND_IRQ_ASSERT_ONCE_FMASK GENMASK(2, 2)
+/* The next two fields are not present for IPA v4.2 */
+#define BCR_SUSPEND_L2_IRQ_FMASK GENMASK(3, 3)
+#define BCR_HOLB_DROP_L2_IRQ_FMASK GENMASK(4, 4)
+#define BCR_DUAL_TX_FMASK GENMASK(5, 5)
+#define BCR_ENABLE_FILTER_DATA_CACHE_FMASK GENMASK(6, 6)
+#define BCR_NOTIF_PRIORITY_OVER_ZLT_FMASK GENMASK(7, 7)
+#define BCR_FILTER_PREFETCH_EN_FMASK GENMASK(8, 8)
+#define BCR_ROUTER_PREFETCH_EN_FMASK GENMASK(9, 9)
/* Backward compatibility register value to use for each version */
static inline u32 ipa_reg_bcr_val(enum ipa_version version)
{
if (version == IPA_VERSION_3_5_1)
- return BCR_CMDQ_L_LACK_ONE_ENTRY | BCR_TX_NOT_USING_BRESP |
- BCR_SUSPEND_L2_IRQ | BCR_HOLB_DROP_L2_IRQ | BCR_DUAL_TX;
+ return BCR_CMDQ_L_LACK_ONE_ENTRY_FMASK |
+ BCR_TX_NOT_USING_BRESP_FMASK |
+ BCR_SUSPEND_L2_IRQ_FMASK |
+ BCR_HOLB_DROP_L2_IRQ_FMASK |
+ BCR_DUAL_TX_FMASK;
if (version == IPA_VERSION_4_0 || version == IPA_VERSION_4_1)
- return BCR_CMDQ_L_LACK_ONE_ENTRY | BCR_SUSPEND_L2_IRQ |
- BCR_HOLB_DROP_L2_IRQ | BCR_DUAL_TX;
+ return BCR_CMDQ_L_LACK_ONE_ENTRY_FMASK |
+ BCR_SUSPEND_L2_IRQ_FMASK |
+ BCR_HOLB_DROP_L2_IRQ_FMASK |
+ BCR_DUAL_TX_FMASK;
+
+ /* assert(version != IPA_VERSION_4_5); */
return 0x00000000;
}
+/* The value of the next register must be a multiple of 8 */
#define IPA_REG_LOCAL_PKT_PROC_CNTXT_BASE_OFFSET 0x000001e8
-#define IPA_REG_AGGR_FORCE_CLOSE_OFFSET 0x000001ec
/* ipa->available defines the valid bits in the AGGR_FORCE_CLOSE register */
+#define IPA_REG_AGGR_FORCE_CLOSE_OFFSET 0x000001ec
+
+/* The next register is not present for IPA v4.5 */
+#define IPA_REG_COUNTER_CFG_OFFSET 0x000001f0
+#define AGGR_GRANULARITY_FMASK GENMASK(8, 4)
/* The internal inactivity timer clock is used for the aggregation timer */
-#define TIMER_FREQUENCY 32000 /* 32 KHz inactivity timer clock */
+#define TIMER_FREQUENCY 32000 /* 32 KHz inactivity timer clock */
-#define IPA_REG_COUNTER_CFG_OFFSET 0x000001f0
-#define AGGR_GRANULARITY GENMASK(8, 4)
/* Compute the value to use in the AGGR_GRANULARITY field representing the
* given number of microseconds. The value is one less than the number of
- * timer ticks in the requested period. Zero not a valid granularity value.
+ * timer ticks in the requested period. 0 not a valid granularity value.
*/
static inline u32 ipa_aggr_granularity_val(u32 usec)
{
return DIV_ROUND_CLOSEST(usec * TIMER_FREQUENCY, USEC_PER_SEC) - 1;
}
+/* The next register is not present for IPA v4.5 */
#define IPA_REG_TX_CFG_OFFSET 0x000001fc
/* The first three fields are present for IPA v3.5.1 only */
-#define TX0_PREFETCH_DISABLE GENMASK(0, 0)
-#define TX1_PREFETCH_DISABLE GENMASK(1, 1)
-#define PREFETCH_ALMOST_EMPTY_SIZE GENMASK(4, 2)
-/* The next fields are present for IPA v4.0 and above */
-#define PREFETCH_ALMOST_EMPTY_SIZE_TX0 GENMASK(5, 2)
-#define DMAW_SCND_OUTSD_PRED_THRESHOLD GENMASK(9, 6)
-#define DMAW_SCND_OUTSD_PRED_EN GENMASK(10, 10)
-#define DMAW_MAX_BEATS_256_DIS GENMASK(11, 11)
-#define PA_MASK_EN GENMASK(12, 12)
-#define PREFETCH_ALMOST_EMPTY_SIZE_TX1 GENMASK(16, 13)
-/* The last two fields are present for IPA v4.2 and above */
-#define SSPND_PA_NO_START_STATE GENMASK(18, 18)
-#define SSPND_PA_NO_BQ_STATE GENMASK(19, 19)
+#define TX0_PREFETCH_DISABLE_FMASK GENMASK(0, 0)
+#define TX1_PREFETCH_DISABLE_FMASK GENMASK(1, 1)
+#define PREFETCH_ALMOST_EMPTY_SIZE_FMASK GENMASK(4, 2)
+/* The next six fields are present for IPA v4.0 and above */
+#define PREFETCH_ALMOST_EMPTY_SIZE_TX0_FMASK GENMASK(5, 2)
+#define DMAW_SCND_OUTSD_PRED_THRESHOLD_FMASK GENMASK(9, 6)
+#define DMAW_SCND_OUTSD_PRED_EN_FMASK GENMASK(10, 10)
+#define DMAW_MAX_BEATS_256_DIS_FMASK GENMASK(11, 11)
+#define PA_MASK_EN_FMASK GENMASK(12, 12)
+#define PREFETCH_ALMOST_EMPTY_SIZE_TX1_FMASK GENMASK(16, 13)
+/* The next field is present for IPA v4.5 */
+#define DUAL_TX_ENABLE_FMASK GENMASK(17, 17)
+/* The next two fields are present for IPA v4.2 only */
+#define SSPND_PA_NO_START_STATE_FMASK GENMASK(18, 18)
+#define SSPND_PA_NO_BQ_STATE_FMASK GENMASK(19, 19)
#define IPA_REG_FLAVOR_0_OFFSET 0x00000210
-#define BAM_MAX_PIPES_FMASK GENMASK(4, 0)
-#define BAM_MAX_CONS_PIPES_FMASK GENMASK(12, 8)
-#define BAM_MAX_PROD_PIPES_FMASK GENMASK(20, 16)
-#define BAM_PROD_LOWEST_FMASK GENMASK(27, 24)
+#define IPA_MAX_PIPES_FMASK GENMASK(3, 0)
+#define IPA_MAX_CONS_PIPES_FMASK GENMASK(12, 8)
+#define IPA_MAX_PROD_PIPES_FMASK GENMASK(20, 16)
+#define IPA_PROD_LOWEST_FMASK GENMASK(27, 24)
static inline u32 ipa_reg_idle_indication_cfg_offset(enum ipa_version version)
{
- if (version == IPA_VERSION_4_2)
+ if (version >= IPA_VERSION_4_2)
return 0x00000240;
return 0x00000220;
#define ENTER_IDLE_DEBOUNCE_THRESH_FMASK GENMASK(15, 0)
#define CONST_NON_IDLE_ENABLE_FMASK GENMASK(16, 16)
+/* The next register is present for IPA v4.5 */
+#define IPA_REG_QTIME_TIMESTAMP_CFG_OFFSET 0x0000024c
+#define DPL_TIMESTAMP_LSB_FMASK GENMASK(4, 0)
+#define DPL_TIMESTAMP_SEL_FMASK GENMASK(7, 7)
+#define TAG_TIMESTAMP_LSB_FMASK GENMASK(12, 8)
+#define NAT_TIMESTAMP_LSB_FMASK GENMASK(20, 16)
+
+/* The next register is present for IPA v4.5 */
+#define IPA_REG_TIMERS_XO_CLK_DIV_CFG_OFFSET 0x00000250
+#define DIV_VALUE_FMASK GENMASK(8, 0)
+#define DIV_ENABLE_FMASK GENMASK(31, 31)
+
+/* The next register is present for IPA v4.5 */
+#define IPA_REG_TIMERS_PULSE_GRAN_CFG_OFFSET 0x00000254
+#define GRAN_0_FMASK GENMASK(2, 0)
+#define GRAN_1_FMASK GENMASK(5, 3)
+#define GRAN_2_FMASK GENMASK(8, 6)
+/* Values for GRAN_x fields of TIMERS_PULSE_GRAN_CFG */
+enum ipa_pulse_gran {
+ IPA_GRAN_10_US = 0x0,
+ IPA_GRAN_20_US = 0x1,
+ IPA_GRAN_50_US = 0x2,
+ IPA_GRAN_100_US = 0x3,
+ IPA_GRAN_1_MS = 0x4,
+ IPA_GRAN_10_MS = 0x5,
+ IPA_GRAN_100_MS = 0x6,
+ IPA_GRAN_655350_US = 0x7,
+};
+
/* # IPA source resource groups available based on version */
static inline u32 ipa_resource_group_src_count(enum ipa_version version)
{
case IPA_VERSION_4_2:
return 1;
+ case IPA_VERSION_4_5:
+ return 5;
+
default:
return 0;
}
case IPA_VERSION_4_2:
return 1;
+ case IPA_VERSION_4_5:
+ return 5;
+
default:
return 0;
}
(0x00000400 + 0x0020 * (rt))
#define IPA_REG_SRC_RSRC_GRP_23_RSRC_TYPE_N_OFFSET(rt) \
(0x00000404 + 0x0020 * (rt))
+/* The next register is only present for IPA v4.5 */
#define IPA_REG_SRC_RSRC_GRP_45_RSRC_TYPE_N_OFFSET(rt) \
(0x00000408 + 0x0020 * (rt))
#define IPA_REG_DST_RSRC_GRP_01_RSRC_TYPE_N_OFFSET(rt) \
(0x00000500 + 0x0020 * (rt))
#define IPA_REG_DST_RSRC_GRP_23_RSRC_TYPE_N_OFFSET(rt) \
(0x00000504 + 0x0020 * (rt))
+/* The next register is only present for IPA v4.5 */
#define IPA_REG_DST_RSRC_GRP_45_RSRC_TYPE_N_OFFSET(rt) \
(0x00000508 + 0x0020 * (rt))
+/* The next four fields are used for all resource group registers */
#define X_MIN_LIM_FMASK GENMASK(5, 0)
#define X_MAX_LIM_FMASK GENMASK(13, 8)
+/* The next two fields are not always present (if resource count is odd) */
#define Y_MIN_LIM_FMASK GENMASK(21, 16)
#define Y_MAX_LIM_FMASK GENMASK(29, 24)
#define IPA_REG_ENDP_INIT_CTRL_N_OFFSET(ep) \
(0x00000800 + 0x0070 * (ep))
+/* The next field should only used for IPA v3.5.1 */
#define ENDP_SUSPEND_FMASK GENMASK(0, 0)
#define ENDP_DELAY_FMASK GENMASK(1, 1)
#define CS_METADATA_HDR_OFFSET_FMASK GENMASK(6, 3)
#define CS_GEN_QMB_MASTER_SEL_FMASK GENMASK(8, 8)
+/** enum ipa_cs_offload_en - checksum offload field in ENDP_INIT_CFG_N */
+enum ipa_cs_offload_en {
+ IPA_CS_OFFLOAD_NONE = 0x0,
+ IPA_CS_OFFLOAD_UL = 0x1,
+ IPA_CS_OFFLOAD_DL = 0x2,
+};
+
#define IPA_REG_ENDP_INIT_HDR_N_OFFSET(ep) \
(0x00000810 + 0x0070 * (ep))
#define HDR_LEN_FMASK GENMASK(5, 0)
#define HDR_OFST_PKT_SIZE_FMASK GENMASK(25, 20)
#define HDR_A5_MUX_FMASK GENMASK(26, 26)
#define HDR_LEN_INC_DEAGG_HDR_FMASK GENMASK(27, 27)
+/* The next field is not present for IPA v4.5 */
#define HDR_METADATA_REG_VALID_FMASK GENMASK(28, 28)
+/* The next two fields are present for IPA v4.5 */
+#define HDR_LEN_MSB_FMASK GENMASK(29, 28)
+#define HDR_OFST_METADATA_MSB_FMASK GENMASK(31, 30)
+
+/* Encoded value for ENDP_INIT_HDR register HDR_LEN* field(s) */
+static inline u32 ipa_header_size_encoded(enum ipa_version version,
+ u32 header_size)
+{
+ u32 val;
+
+ val = u32_encode_bits(header_size, HDR_LEN_FMASK);
+ if (version < IPA_VERSION_4_5)
+ return val;
+
+ /* IPA v4.5 adds a few more most-significant bits */
+ header_size >>= hweight32(HDR_LEN_FMASK);
+ val |= u32_encode_bits(header_size, HDR_LEN_MSB_FMASK);
+
+ return val;
+}
+
+/* Encoded value for ENDP_INIT_HDR register OFST_METADATA* field(s) */
+static inline u32 ipa_metadata_offset_encoded(enum ipa_version version,
+ u32 offset)
+{
+ u32 val;
+
+ val = u32_encode_bits(offset, HDR_OFST_METADATA_FMASK);
+ if (version < IPA_VERSION_4_5)
+ return val;
+
+ /* IPA v4.5 adds a few more most-significant bits */
+ offset >>= hweight32(HDR_OFST_METADATA_FMASK);
+ val |= u32_encode_bits(offset, HDR_OFST_METADATA_MSB_FMASK);
+
+ return val;
+}
#define IPA_REG_ENDP_INIT_HDR_EXT_N_OFFSET(ep) \
(0x00000814 + 0x0070 * (ep))
#define HDR_PAYLOAD_LEN_INC_PADDING_FMASK GENMASK(3, 3)
#define HDR_TOTAL_LEN_OR_PAD_OFFSET_FMASK GENMASK(9, 4)
#define HDR_PAD_TO_ALIGNMENT_FMASK GENMASK(13, 10)
+/* The next three fields are present for IPA v4.5 */
+#define HDR_TOTAL_LEN_OR_PAD_OFFSET_MSB_FMASK GENMASK(17, 16)
+#define HDR_OFST_PKT_SIZE_MSB_FMASK GENMASK(19, 18)
+#define HDR_ADDITIONAL_CONST_LEN_MSB_FMASK GENMASK(21, 20)
/* Valid only for RX (IPA producer) endpoints */
#define IPA_REG_ENDP_INIT_HDR_METADATA_MASK_N_OFFSET(rxep) \
#define IPA_REG_ENDP_INIT_MODE_N_OFFSET(txep) \
(0x00000820 + 0x0070 * (txep))
#define MODE_FMASK GENMASK(2, 0)
+/* The next field is present for IPA v4.5 */
+#define DCPH_ENABLE_FMASK GENMASK(3, 3)
#define DEST_PIPE_INDEX_FMASK GENMASK(8, 4)
#define BYTE_THRESHOLD_FMASK GENMASK(27, 12)
#define PIPE_REPLICATION_EN_FMASK GENMASK(28, 28)
#define PAD_EN_FMASK GENMASK(29, 29)
+/* The next register is not present for IPA v4.5 */
#define HDR_FTCH_DISABLE_FMASK GENMASK(30, 30)
+/** enum ipa_mode - mode field in ENDP_INIT_MODE_N */
+enum ipa_mode {
+ IPA_BASIC = 0x0,
+ IPA_ENABLE_FRAMING_HDLC = 0x1,
+ IPA_ENABLE_DEFRAMING_HDLC = 0x2,
+ IPA_DMA = 0x3,
+};
+
#define IPA_REG_ENDP_INIT_AGGR_N_OFFSET(ep) \
(0x00000824 + 0x0070 * (ep))
#define AGGR_EN_FMASK GENMASK(1, 0)
#define AGGR_TYPE_FMASK GENMASK(4, 2)
-#define AGGR_BYTE_LIMIT_FMASK GENMASK(9, 5)
-#define AGGR_TIME_LIMIT_FMASK GENMASK(14, 10)
-#define AGGR_PKT_LIMIT_FMASK GENMASK(20, 15)
-#define AGGR_SW_EOF_ACTIVE_FMASK GENMASK(21, 21)
-#define AGGR_FORCE_CLOSE_FMASK GENMASK(22, 22)
-#define AGGR_HARD_BYTE_LIMIT_ENABLE_FMASK GENMASK(24, 24)
+static inline u32 aggr_byte_limit_fmask(bool legacy)
+{
+ return legacy ? GENMASK(9, 5) : GENMASK(10, 5);
+}
+
+static inline u32 aggr_time_limit_fmask(bool legacy)
+{
+ return legacy ? GENMASK(14, 10) : GENMASK(16, 12);
+}
+
+static inline u32 aggr_pkt_limit_fmask(bool legacy)
+{
+ return legacy ? GENMASK(20, 15) : GENMASK(22, 17);
+}
+
+static inline u32 aggr_sw_eof_active_fmask(bool legacy)
+{
+ return legacy ? GENMASK(21, 21) : GENMASK(23, 23);
+}
+
+static inline u32 aggr_force_close_fmask(bool legacy)
+{
+ return legacy ? GENMASK(22, 22) : GENMASK(24, 24);
+}
+
+static inline u32 aggr_hard_byte_limit_enable_fmask(bool legacy)
+{
+ return legacy ? GENMASK(24, 24) : GENMASK(26, 26);
+}
+
+/* The next field is present for IPA v4.5 */
+#define AGGR_GRAN_SEL_FMASK GENMASK(27, 27)
+
+/** enum ipa_aggr_en - aggregation enable field in ENDP_INIT_AGGR_N */
+enum ipa_aggr_en {
+ IPA_BYPASS_AGGR = 0x0,
+ IPA_ENABLE_AGGR = 0x1,
+ IPA_ENABLE_DEAGGR = 0x2,
+};
+
+/** enum ipa_aggr_type - aggregation type field in ENDP_INIT_AGGR_N */
+enum ipa_aggr_type {
+ IPA_MBIM_16 = 0x0,
+ IPA_HDLC = 0x1,
+ IPA_TLP = 0x2,
+ IPA_RNDIS = 0x3,
+ IPA_GENERIC = 0x4,
+ IPA_COALESCE = 0x5,
+ IPA_QCMAP = 0x6,
+};
/* Valid only for RX (IPA producer) endpoints */
#define IPA_REG_ENDP_INIT_HOL_BLOCK_EN_N_OFFSET(rxep) \
/* Valid only for RX (IPA producer) endpoints */
#define IPA_REG_ENDP_INIT_HOL_BLOCK_TIMER_N_OFFSET(rxep) \
(0x00000830 + 0x0070 * (rxep))
-/* The next fields are present for IPA v4.2 only */
+/* The next two fields are present for IPA v4.2 only */
#define BASE_VALUE_FMASK GENMASK(4, 0)
#define SCALE_FMASK GENMASK(12, 8)
+/* The next two fields are present for IPA v4.5 */
+#define TIME_LIMIT_FMASK GENMASK(4, 0)
+#define GRAN_SEL_FMASK GENMASK(8, 8)
/* Valid only for TX (IPA consumer) endpoints */
#define IPA_REG_ENDP_INIT_DEAGGR_N_OFFSET(txep) \
(0x00000834 + 0x0070 * (txep))
#define DEAGGR_HDR_LEN_FMASK GENMASK(5, 0)
+#define SYSPIPE_ERR_DETECTION_FMASK GENMASK(6, 6)
#define PACKET_OFFSET_VALID_FMASK GENMASK(7, 7)
#define PACKET_OFFSET_LOCATION_FMASK GENMASK(13, 8)
+#define IGNORE_MIN_PKT_ERR_FMASK GENMASK(14, 14)
#define MAX_PACKET_LEN_FMASK GENMASK(31, 16)
#define IPA_REG_ENDP_INIT_RSRC_GRP_N_OFFSET(ep) \
(0x00000838 + 0x0070 * (ep))
-/* Encoded value for RSRC_GRP endpoint register RSRC_GRP field */
+/* Encoded value for ENDP_INIT_RSRC_GRP register RSRC_GRP field */
static inline u32 rsrc_grp_encoded(enum ipa_version version, u32 rsrc_grp)
{
switch (version) {
case IPA_VERSION_4_2:
return u32_encode_bits(rsrc_grp, GENMASK(0, 0));
+ case IPA_VERSION_4_5:
+ return u32_encode_bits(rsrc_grp, GENMASK(2, 0));
default:
return u32_encode_bits(rsrc_grp, GENMASK(1, 0));
}
#define HPS_REP_SEQ_TYPE_FMASK GENMASK(11, 8)
#define DPS_REP_SEQ_TYPE_FMASK GENMASK(15, 12)
+/**
+ * enum ipa_seq_type - HPS and DPS sequencer type fields in ENDP_INIT_SEQ_N
+ * @IPA_SEQ_DMA_ONLY: only DMA is performed
+ * @IPA_SEQ_2ND_PKT_PROCESS_PASS_NO_DEC_UCP:
+ * second packet processing pass + no decipher + microcontroller
+ * @IPA_SEQ_PKT_PROCESS_NO_DEC_NO_UCP_DMAP:
+ * packet processing + no decipher + no uCP + HPS REP DMA parser
+ * @IPA_SEQ_INVALID: invalid sequencer type
+ *
+ * The values defined here are broken into 4-bit nibbles that are written
+ * into fields of the ENDP_INIT_SEQ registers.
+ */
+enum ipa_seq_type {
+ IPA_SEQ_DMA_ONLY = 0x0000,
+ IPA_SEQ_2ND_PKT_PROCESS_PASS_NO_DEC_UCP = 0x0004,
+ IPA_SEQ_PKT_PROCESS_NO_DEC_NO_UCP_DMAP = 0x0806,
+ IPA_SEQ_INVALID = 0xffff,
+};
+
#define IPA_REG_ENDP_STATUS_N_OFFSET(ep) \
(0x00000840 + 0x0070 * (ep))
#define STATUS_EN_FMASK GENMASK(0, 0)
#define STATUS_ENDP_FMASK GENMASK(5, 1)
+/* The next field is not present for IPA v4.5 */
#define STATUS_LOCATION_FMASK GENMASK(8, 8)
-/* The next field is present for IPA v4.0 and above */
+/* The next field is not present for IPA v3.5.1 */
#define STATUS_PKT_SUPPRESS_FMASK GENMASK(9, 9)
-/* "er" is either an endpoint ID (for filters) or a route ID (for routes) */
+/* The next register is only present for IPA versions that support hashing */
#define IPA_REG_ENDP_FILTER_ROUTER_HSH_CFG_N_OFFSET(er) \
(0x0000085c + 0x0070 * (er))
#define FILTER_HASH_MSK_SRC_ID_FMASK GENMASK(0, 0)
IPA_REG_IRQ_CLR_EE_N_OFFSET(GSI_EE_AP)
#define IPA_REG_IRQ_CLR_EE_N_OFFSET(ee) \
(0x00003010 + 0x1000 * (ee))
+/**
+ * enum ipa_irq_id - Bit positions representing type of IPA IRQ
+ * @IPA_IRQ_UC_0: Microcontroller event interrupt
+ * @IPA_IRQ_UC_1: Microcontroller response interrupt
+ * @IPA_IRQ_TX_SUSPEND: Data ready interrupt
+ *
+ * IRQ types not described above are not currently used.
+ */
+enum ipa_irq_id {
+ IPA_IRQ_BAD_SNOC_ACCESS = 0x0,
+ /* Type (bit) 0x1 is not defined */
+ IPA_IRQ_UC_0 = 0x2,
+ IPA_IRQ_UC_1 = 0x3,
+ IPA_IRQ_UC_2 = 0x4,
+ IPA_IRQ_UC_3 = 0x5,
+ IPA_IRQ_UC_IN_Q_NOT_EMPTY = 0x6,
+ IPA_IRQ_UC_RX_CMD_Q_NOT_FULL = 0x7,
+ IPA_IRQ_PROC_UC_ACK_Q_NOT_EMPTY = 0x8,
+ IPA_IRQ_RX_ERR = 0x9,
+ IPA_IRQ_DEAGGR_ERR = 0xa,
+ IPA_IRQ_TX_ERR = 0xb,
+ IPA_IRQ_STEP_MODE = 0xc,
+ IPA_IRQ_PROC_ERR = 0xd,
+ IPA_IRQ_TX_SUSPEND = 0xe,
+ IPA_IRQ_TX_HOLB_DROP = 0xf,
+ IPA_IRQ_BAM_GSI_IDLE = 0x10,
+ IPA_IRQ_PIPE_YELLOW_BELOW = 0x11,
+ IPA_IRQ_PIPE_RED_BELOW = 0x12,
+ IPA_IRQ_PIPE_YELLOW_ABOVE = 0x13,
+ IPA_IRQ_PIPE_RED_ABOVE = 0x14,
+ IPA_IRQ_UCP = 0x15,
+ IPA_IRQ_DCMP = 0x16,
+ IPA_IRQ_GSI_EE = 0x17,
+ IPA_IRQ_GSI_IPA_IF_TLV_RCVD = 0x18,
+ IPA_IRQ_GSI_UC = 0x19,
+ /* The next bit is present for IPA v4.5 */
+ IPA_IRQ_TLV_LEN_MIN_DSM = 0x1a,
+ IPA_IRQ_COUNT, /* Last; not an id */
+};
#define IPA_REG_IRQ_UC_OFFSET \
IPA_REG_IRQ_UC_EE_N_OFFSET(GSI_EE_AP)
#define IPA_REG_IRQ_UC_EE_N_OFFSET(ee) \
(0x0000301c + 0x1000 * (ee))
+#define UC_INTR_FMASK GENMASK(0, 0)
+/* ipa->available defines the valid bits in the SUSPEND_INFO register */
#define IPA_REG_IRQ_SUSPEND_INFO_OFFSET \
IPA_REG_IRQ_SUSPEND_INFO_EE_N_OFFSET(GSI_EE_AP)
#define IPA_REG_IRQ_SUSPEND_INFO_EE_N_OFFSET(ee) \
(0x00003030 + 0x1000 * (ee))
-/* ipa->available defines the valid bits in the SUSPEND_INFO register */
-#define IPA_REG_SUSPEND_IRQ_EN_OFFSET \
- IPA_REG_SUSPEND_IRQ_EN_EE_N_OFFSET(GSI_EE_AP)
-#define IPA_REG_SUSPEND_IRQ_EN_EE_N_OFFSET(ee) \
+/* ipa->available defines the valid bits in the IRQ_SUSPEND_EN register */
+#define IPA_REG_IRQ_SUSPEND_EN_OFFSET \
+ IPA_REG_IRQ_SUSPEND_EN_EE_N_OFFSET(GSI_EE_AP)
+#define IPA_REG_IRQ_SUSPEND_EN_EE_N_OFFSET(ee) \
(0x00003034 + 0x1000 * (ee))
-/* ipa->available defines the valid bits in the SUSPEND_IRQ_EN register */
-#define IPA_REG_SUSPEND_IRQ_CLR_OFFSET \
- IPA_REG_SUSPEND_IRQ_CLR_EE_N_OFFSET(GSI_EE_AP)
-#define IPA_REG_SUSPEND_IRQ_CLR_EE_N_OFFSET(ee) \
+/* ipa->available defines the valid bits in the IRQ_SUSPEND_CLR register */
+#define IPA_REG_IRQ_SUSPEND_CLR_OFFSET \
+ IPA_REG_IRQ_SUSPEND_CLR_EE_N_OFFSET(GSI_EE_AP)
+#define IPA_REG_IRQ_SUSPEND_CLR_EE_N_OFFSET(ee) \
(0x00003038 + 0x1000 * (ee))
-/* ipa->available defines the valid bits in the SUSPEND_IRQ_CLR register */
-
-/** enum ipa_cs_offload_en - checksum offload field in ENDP_INIT_CFG_N */
-enum ipa_cs_offload_en {
- IPA_CS_OFFLOAD_NONE = 0,
- IPA_CS_OFFLOAD_UL = 1,
- IPA_CS_OFFLOAD_DL = 2,
- IPA_CS_RSVD
-};
-
-/** enum ipa_aggr_en - aggregation enable field in ENDP_INIT_AGGR_N */
-enum ipa_aggr_en {
- IPA_BYPASS_AGGR = 0,
- IPA_ENABLE_AGGR = 1,
- IPA_ENABLE_DEAGGR = 2,
-};
-
-/** enum ipa_aggr_type - aggregation type field in in_ENDP_INIT_AGGR_N */
-enum ipa_aggr_type {
- IPA_MBIM_16 = 0,
- IPA_HDLC = 1,
- IPA_TLP = 2,
- IPA_RNDIS = 3,
- IPA_GENERIC = 4,
- IPA_COALESCE = 5,
- IPA_QCMAP = 6,
-};
-
-/** enum ipa_mode - mode field in ENDP_INIT_MODE_N */
-enum ipa_mode {
- IPA_BASIC = 0,
- IPA_ENABLE_FRAMING_HDLC = 1,
- IPA_ENABLE_DEFRAMING_HDLC = 2,
- IPA_DMA = 3,
-};
-
-/**
- * enum ipa_seq_type - HPS and DPS sequencer type fields in in ENDP_INIT_SEQ_N
- * @IPA_SEQ_DMA_ONLY: only DMA is performed
- * @IPA_SEQ_PKT_PROCESS_NO_DEC_UCP:
- * packet processing + no decipher + microcontroller (Ethernet Bridging)
- * @IPA_SEQ_2ND_PKT_PROCESS_PASS_NO_DEC_UCP:
- * second packet processing pass + no decipher + microcontroller
- * @IPA_SEQ_DMA_DEC: DMA + cipher/decipher
- * @IPA_SEQ_DMA_COMP_DECOMP: DMA + compression/decompression
- * @IPA_SEQ_PKT_PROCESS_NO_DEC_NO_UCP_DMAP:
- * packet processing + no decipher + no uCP + HPS REP DMA parser
- * @IPA_SEQ_INVALID: invalid sequencer type
- *
- * The values defined here are broken into 4-bit nibbles that are written
- * into fields of the INIT_SEQ_N endpoint registers.
- */
-enum ipa_seq_type {
- IPA_SEQ_DMA_ONLY = 0x0000,
- IPA_SEQ_PKT_PROCESS_NO_DEC_UCP = 0x0002,
- IPA_SEQ_2ND_PKT_PROCESS_PASS_NO_DEC_UCP = 0x0004,
- IPA_SEQ_DMA_DEC = 0x0011,
- IPA_SEQ_DMA_COMP_DECOMP = 0x0020,
- IPA_SEQ_PKT_PROCESS_NO_DEC_NO_UCP_DMAP = 0x0806,
- IPA_SEQ_INVALID = 0xffff,
-};
int ipa_reg_init(struct ipa *ipa);
void ipa_reg_exit(struct ipa *ipa);
return -EBUSY;
}
- val = IPV4_FILTER_HASH_FLUSH | IPV6_FILTER_HASH_FLUSH;
- val |= IPV6_ROUTER_HASH_FLUSH | IPV4_ROUTER_HASH_FLUSH;
+ val = IPV4_FILTER_HASH_FMASK | IPV6_FILTER_HASH_FMASK;
+ val |= IPV6_ROUTER_HASH_FMASK | IPV4_ROUTER_HASH_FMASK;
ipa_cmd_register_write_add(trans, offset, val, val, false);
/** enum ipa_uc_command - commands from the AP to the microcontroller */
enum ipa_uc_command {
- IPA_UC_COMMAND_NO_OP = 0,
- IPA_UC_COMMAND_UPDATE_FLAGS = 1,
- IPA_UC_COMMAND_DEBUG_RUN_TEST = 2,
- IPA_UC_COMMAND_DEBUG_GET_INFO = 3,
- IPA_UC_COMMAND_ERR_FATAL = 4,
- IPA_UC_COMMAND_CLK_GATE = 5,
- IPA_UC_COMMAND_CLK_UNGATE = 6,
- IPA_UC_COMMAND_MEMCPY = 7,
- IPA_UC_COMMAND_RESET_PIPE = 8,
- IPA_UC_COMMAND_REG_WRITE = 9,
- IPA_UC_COMMAND_GSI_CH_EMPTY = 10,
+ IPA_UC_COMMAND_NO_OP = 0x0,
+ IPA_UC_COMMAND_UPDATE_FLAGS = 0x1,
+ IPA_UC_COMMAND_DEBUG_RUN_TEST = 0x2,
+ IPA_UC_COMMAND_DEBUG_GET_INFO = 0x3,
+ IPA_UC_COMMAND_ERR_FATAL = 0x4,
+ IPA_UC_COMMAND_CLK_GATE = 0x5,
+ IPA_UC_COMMAND_CLK_UNGATE = 0x6,
+ IPA_UC_COMMAND_MEMCPY = 0x7,
+ IPA_UC_COMMAND_RESET_PIPE = 0x8,
+ IPA_UC_COMMAND_REG_WRITE = 0x9,
+ IPA_UC_COMMAND_GSI_CH_EMPTY = 0xa,
};
/** enum ipa_uc_response - microcontroller response codes */
enum ipa_uc_response {
- IPA_UC_RESPONSE_NO_OP = 0,
- IPA_UC_RESPONSE_INIT_COMPLETED = 1,
- IPA_UC_RESPONSE_CMD_COMPLETED = 2,
- IPA_UC_RESPONSE_DEBUG_GET_INFO = 3,
+ IPA_UC_RESPONSE_NO_OP = 0x0,
+ IPA_UC_RESPONSE_INIT_COMPLETED = 0x1,
+ IPA_UC_RESPONSE_CMD_COMPLETED = 0x2,
+ IPA_UC_RESPONSE_DEBUG_GET_INFO = 0x3,
};
/** enum ipa_uc_event - common cpu events reported by the microcontroller */
enum ipa_uc_event {
- IPA_UC_EVENT_NO_OP = 0,
- IPA_UC_EVENT_ERROR = 1,
- IPA_UC_EVENT_LOG_INFO = 2,
+ IPA_UC_EVENT_NO_OP = 0x0,
+ IPA_UC_EVENT_ERROR = 0x1,
+ IPA_UC_EVENT_LOG_INFO = 0x2,
};
static struct ipa_uc_mem_area *ipa_uc_shared(struct ipa *ipa)
static void send_uc_command(struct ipa *ipa, u32 command, u32 command_param)
{
struct ipa_uc_mem_area *shared = ipa_uc_shared(ipa);
+ u32 val;
+ /* Fill in the command data */
shared->command = command;
shared->command_param = cpu_to_le32(command_param);
shared->command_param_hi = 0;
shared->response = 0;
shared->response_param = 0;
- iowrite32(1, ipa->reg_virt + IPA_REG_IRQ_UC_OFFSET);
+ /* Use an interrupt to tell the microcontroller the command is ready */
+ val = u32_encode_bits(1, UC_INTR_FMASK);
+
+ iowrite32(val, ipa->reg_virt + IPA_REG_IRQ_UC_OFFSET);
}
/* Tell the microcontroller the AP is shutting down */
IPA_VERSION_4_0, /* GSI version 2.0 */
IPA_VERSION_4_1, /* GSI version 2.1 */
IPA_VERSION_4_2, /* GSI version 2.2 */
+ IPA_VERSION_4_5, /* GSI version 2.5 */
};
#endif /* _IPA_VERSION_H_ */
/* Copyright (c) 2014 Mahesh Bandewar <maheshb@google.com>
*/
+#include <linux/ethtool.h>
+
#include "ipvlan.h"
static int ipvlan_set_port_mode(struct ipvl_port *port, u16 nval,
#define MACVLAN_HASH_BITS 8
#define MACVLAN_HASH_SIZE (1<<MACVLAN_HASH_BITS)
-#define MACVLAN_BC_QUEUE_LEN 1000
+#define MACVLAN_DEFAULT_BC_QUEUE_LEN 1000
#define MACVLAN_F_PASSTHRU 1
#define MACVLAN_F_ADDRCHANGE 2
struct list_head vlans;
struct sk_buff_head bc_queue;
struct work_struct bc_work;
+ u32 bc_queue_len_used;
u32 flags;
int count;
struct hlist_head vlan_source_hash[MACVLAN_HASH_SIZE];
#define MACVLAN_SKB_CB(__skb) ((struct macvlan_skb_cb *)&((__skb)->cb[0]))
static void macvlan_port_destroy(struct net_device *dev);
+static void update_port_bc_queue_len(struct macvlan_port *port);
static inline bool macvlan_passthru(const struct macvlan_port *port)
{
MACVLAN_SKB_CB(nskb)->src = src;
spin_lock(&port->bc_queue.lock);
- if (skb_queue_len(&port->bc_queue) < MACVLAN_BC_QUEUE_LEN) {
+ if (skb_queue_len(&port->bc_queue) < port->bc_queue_len_used) {
if (src)
dev_hold(src->dev);
__skb_queue_tail(&port->bc_queue, nskb);
for (i = 0; i < MACVLAN_HASH_SIZE; i++)
INIT_HLIST_HEAD(&port->vlan_source_hash[i]);
+ port->bc_queue_len_used = 0;
skb_queue_head_init(&port->bc_queue);
INIT_WORK(&port->bc_work, macvlan_process_broadcast);
goto destroy_macvlan_port;
}
+ vlan->bc_queue_len_req = MACVLAN_DEFAULT_BC_QUEUE_LEN;
+ if (data && data[IFLA_MACVLAN_BC_QUEUE_LEN])
+ vlan->bc_queue_len_req = nla_get_u32(data[IFLA_MACVLAN_BC_QUEUE_LEN]);
+
err = register_netdevice(dev);
if (err < 0)
goto destroy_macvlan_port;
goto unregister_netdev;
list_add_tail_rcu(&vlan->list, &port->vlans);
+ update_port_bc_queue_len(vlan->port);
netif_stacked_transfer_operstate(lowerdev, dev);
linkwatch_fire_event(dev);
if (vlan->mode == MACVLAN_MODE_SOURCE)
macvlan_flush_sources(vlan->port, vlan);
list_del_rcu(&vlan->list);
+ update_port_bc_queue_len(vlan->port);
unregister_netdevice_queue(dev, head);
netdev_upper_dev_unlink(vlan->lowerdev, dev);
}
}
vlan->flags = flags;
}
+
+ if (data && data[IFLA_MACVLAN_BC_QUEUE_LEN]) {
+ vlan->bc_queue_len_req = nla_get_u32(data[IFLA_MACVLAN_BC_QUEUE_LEN]);
+ update_port_bc_queue_len(vlan->port);
+ }
+
if (set_mode)
vlan->mode = mode;
if (data && data[IFLA_MACVLAN_MACADDR_MODE]) {
+ nla_total_size(2) /* IFLA_MACVLAN_FLAGS */
+ nla_total_size(4) /* IFLA_MACVLAN_MACADDR_COUNT */
+ macvlan_get_size_mac(vlan) /* IFLA_MACVLAN_MACADDR */
+ + nla_total_size(4) /* IFLA_MACVLAN_BC_QUEUE_LEN */
+ + nla_total_size(4) /* IFLA_MACVLAN_BC_QUEUE_LEN_USED */
);
}
const struct net_device *dev)
{
struct macvlan_dev *vlan = netdev_priv(dev);
+ struct macvlan_port *port = vlan->port;
int i;
struct nlattr *nest;
}
nla_nest_end(skb, nest);
}
+ if (nla_put_u32(skb, IFLA_MACVLAN_BC_QUEUE_LEN, vlan->bc_queue_len_req))
+ goto nla_put_failure;
+ if (nla_put_u32(skb, IFLA_MACVLAN_BC_QUEUE_LEN_USED, port->bc_queue_len_used))
+ goto nla_put_failure;
return 0;
nla_put_failure:
[IFLA_MACVLAN_MACADDR] = { .type = NLA_BINARY, .len = MAX_ADDR_LEN },
[IFLA_MACVLAN_MACADDR_DATA] = { .type = NLA_NESTED },
[IFLA_MACVLAN_MACADDR_COUNT] = { .type = NLA_U32 },
+ [IFLA_MACVLAN_BC_QUEUE_LEN] = { .type = NLA_U32 },
+ [IFLA_MACVLAN_BC_QUEUE_LEN_USED] = { .type = NLA_REJECT },
};
int macvlan_link_register(struct rtnl_link_ops *ops)
.priv_size = sizeof(struct macvlan_dev),
};
+static void update_port_bc_queue_len(struct macvlan_port *port)
+{
+ u32 max_bc_queue_len_req = 0;
+ struct macvlan_dev *vlan;
+
+ list_for_each_entry(vlan, &port->vlans, list) {
+ if (vlan->bc_queue_len_req > max_bc_queue_len_req)
+ max_bc_queue_len_req = vlan->bc_queue_len_req;
+ }
+ port->bc_queue_len_used = max_bc_queue_len_req;
+}
+
static int macvlan_device_event(struct notifier_block *unused,
unsigned long event, void *ptr)
{
.open = simple_open,
.write = nsim_dev_take_snapshot_write,
.llseek = generic_file_llseek,
+ .owner = THIS_MODULE,
};
static ssize_t nsim_dev_trap_fa_cookie_read(struct file *file,
.read = nsim_dev_trap_fa_cookie_read,
.write = nsim_dev_trap_fa_cookie_write,
.llseek = generic_file_llseek,
+ .owner = THIS_MODULE,
};
static int nsim_dev_debugfs_init(struct nsim_dev *nsim_dev)
return -EOPNOTSUPP;
if (nsim_dev->fw_update_status) {
- devlink_flash_update_begin_notify(devlink);
devlink_flash_update_status_notify(devlink,
"Preparing to flash",
params->component, 0, 0);
params->component, 81);
devlink_flash_update_status_notify(devlink, "Flashing done",
params->component, 0, 0);
- devlink_flash_update_end_notify(devlink);
}
return 0;
{
struct netdevsim *ns = netdev_priv(dev);
- if (ns->ethtool.report_stats_rx)
+ if (ns->ethtool.pauseparam.report_stats_rx)
pause_stats->rx_pause_frames = 1;
- if (ns->ethtool.report_stats_tx)
+ if (ns->ethtool.pauseparam.report_stats_tx)
pause_stats->tx_pause_frames = 2;
}
struct netdevsim *ns = netdev_priv(dev);
pause->autoneg = 0; /* We don't support ksettings, so can't pretend */
- pause->rx_pause = ns->ethtool.rx;
- pause->tx_pause = ns->ethtool.tx;
+ pause->rx_pause = ns->ethtool.pauseparam.rx;
+ pause->tx_pause = ns->ethtool.pauseparam.tx;
}
static int
if (pause->autoneg)
return -EINVAL;
- ns->ethtool.rx = pause->rx_pause;
- ns->ethtool.tx = pause->tx_pause;
+ ns->ethtool.pauseparam.rx = pause->rx_pause;
+ ns->ethtool.pauseparam.tx = pause->tx_pause;
+ return 0;
+}
+
+static int nsim_get_coalesce(struct net_device *dev,
+ struct ethtool_coalesce *coal)
+{
+ struct netdevsim *ns = netdev_priv(dev);
+
+ memcpy(coal, &ns->ethtool.coalesce, sizeof(ns->ethtool.coalesce));
+ return 0;
+}
+
+static int nsim_set_coalesce(struct net_device *dev,
+ struct ethtool_coalesce *coal)
+{
+ struct netdevsim *ns = netdev_priv(dev);
+
+ memcpy(&ns->ethtool.coalesce, coal, sizeof(ns->ethtool.coalesce));
+ return 0;
+}
+
+static void nsim_get_ringparam(struct net_device *dev,
+ struct ethtool_ringparam *ring)
+{
+ struct netdevsim *ns = netdev_priv(dev);
+
+ memcpy(ring, &ns->ethtool.ring, sizeof(ns->ethtool.ring));
+}
+
+static int nsim_set_ringparam(struct net_device *dev,
+ struct ethtool_ringparam *ring)
+{
+ struct netdevsim *ns = netdev_priv(dev);
+
+ memcpy(&ns->ethtool.ring, ring, sizeof(ns->ethtool.ring));
return 0;
}
static const struct ethtool_ops nsim_ethtool_ops = {
- .get_pause_stats = nsim_get_pause_stats,
- .get_pauseparam = nsim_get_pauseparam,
- .set_pauseparam = nsim_set_pauseparam,
+ .supported_coalesce_params = ETHTOOL_COALESCE_ALL_PARAMS,
+ .get_pause_stats = nsim_get_pause_stats,
+ .get_pauseparam = nsim_get_pauseparam,
+ .set_pauseparam = nsim_set_pauseparam,
+ .set_coalesce = nsim_set_coalesce,
+ .get_coalesce = nsim_get_coalesce,
+ .get_ringparam = nsim_get_ringparam,
+ .set_ringparam = nsim_set_ringparam,
};
+static void nsim_ethtool_ring_init(struct netdevsim *ns)
+{
+ ns->ethtool.ring.rx_max_pending = 4096;
+ ns->ethtool.ring.rx_jumbo_max_pending = 4096;
+ ns->ethtool.ring.rx_mini_max_pending = 4096;
+ ns->ethtool.ring.tx_max_pending = 4096;
+}
+
void nsim_ethtool_init(struct netdevsim *ns)
{
struct dentry *ethtool, *dir;
ns->netdev->ethtool_ops = &nsim_ethtool_ops;
+ nsim_ethtool_ring_init(ns);
+
ethtool = debugfs_create_dir("ethtool", ns->nsim_dev_port->ddir);
dir = debugfs_create_dir("pause", ethtool);
debugfs_create_bool("report_stats_rx", 0600, dir,
- &ns->ethtool.report_stats_rx);
+ &ns->ethtool.pauseparam.report_stats_rx);
debugfs_create_bool("report_stats_tx", 0600, dir,
- &ns->ethtool.report_stats_tx);
+ &ns->ethtool.pauseparam.report_stats_tx);
+
+ dir = debugfs_create_dir("ring", ethtool);
+ debugfs_create_u32("rx_max_pending", 0600, dir,
+ &ns->ethtool.ring.rx_max_pending);
+ debugfs_create_u32("rx_jumbo_max_pending", 0600, dir,
+ &ns->ethtool.ring.rx_jumbo_max_pending);
+ debugfs_create_u32("rx_mini_max_pending", 0600, dir,
+ &ns->ethtool.ring.rx_mini_max_pending);
+ debugfs_create_u32("tx_max_pending", 0600, dir,
+ &ns->ethtool.ring.tx_max_pending);
}
.open = simple_open,
.write = nsim_dev_health_break_write,
.llseek = generic_file_llseek,
+ .owner = THIS_MODULE,
};
int nsim_dev_health_init(struct nsim_dev *nsim_dev, struct devlink *devlink)
#include <linux/debugfs.h>
#include <linux/device.h>
+#include <linux/ethtool.h>
#include <linux/kernel.h>
#include <linux/list.h>
#include <linux/netdevice.h>
u32 ok;
};
-struct nsim_ethtool {
+struct nsim_ethtool_pauseparam {
bool rx;
bool tx;
bool report_stats_rx;
bool report_stats_tx;
};
+struct nsim_ethtool {
+ struct nsim_ethtool_pauseparam pauseparam;
+ struct ethtool_coalesce coalesce;
+ struct ethtool_ringparam ring;
+};
+
struct netdevsim {
struct net_device *netdev;
struct nsim_dev *nsim_dev;
.open = simple_open,
.write = nsim_udp_tunnels_info_reset_write,
.llseek = generic_file_llseek,
+ .owner = THIS_MODULE,
};
int nsim_udp_tunnels_info_create(struct nsim_dev *nsim_dev,
// SPDX-License-Identifier: GPL-2.0-only
+#include <linux/ethtool.h>
#include <linux/module.h>
#include <linux/kernel.h>
#include <linux/netdevice.h>
static int adin_phy_config_intr(struct phy_device *phydev)
{
- if (phydev->interrupts == PHY_INTERRUPT_ENABLED)
- return phy_set_bits(phydev, ADIN1300_INT_MASK_REG,
- ADIN1300_INT_MASK_EN);
+ int err;
+
+ if (phydev->interrupts == PHY_INTERRUPT_ENABLED) {
+ err = adin_phy_ack_intr(phydev);
+ if (err)
+ return err;
+
+ err = phy_set_bits(phydev, ADIN1300_INT_MASK_REG,
+ ADIN1300_INT_MASK_EN);
+ } else {
+ err = phy_clear_bits(phydev, ADIN1300_INT_MASK_REG,
+ ADIN1300_INT_MASK_EN);
+ if (err)
+ return err;
+
+ err = adin_phy_ack_intr(phydev);
+ }
+
+ return err;
+}
+
+static irqreturn_t adin_phy_handle_interrupt(struct phy_device *phydev)
+{
+ int irq_status;
+
+ irq_status = phy_read(phydev, ADIN1300_INT_STATUS_REG);
+ if (irq_status < 0) {
+ phy_error(phydev);
+ return IRQ_NONE;
+ }
+
+ if (!(irq_status & ADIN1300_INT_LINK_STAT_CHNG_EN))
+ return IRQ_NONE;
+
+ phy_trigger_machine(phydev);
- return phy_clear_bits(phydev, ADIN1300_INT_MASK_REG,
- ADIN1300_INT_MASK_EN);
+ return IRQ_HANDLED;
}
static int adin_cl45_to_adin_reg(struct phy_device *phydev, int devad,
.read_status = adin_read_status,
.get_tunable = adin_get_tunable,
.set_tunable = adin_set_tunable,
- .ack_interrupt = adin_phy_ack_intr,
.config_intr = adin_phy_config_intr,
+ .handle_interrupt = adin_phy_handle_interrupt,
.get_sset_count = adin_get_sset_count,
.get_strings = adin_get_strings,
.get_stats = adin_get_stats,
.read_status = adin_read_status,
.get_tunable = adin_get_tunable,
.set_tunable = adin_set_tunable,
- .ack_interrupt = adin_phy_ack_intr,
.config_intr = adin_phy_config_intr,
+ .handle_interrupt = adin_phy_handle_interrupt,
.get_sset_count = adin_get_sset_count,
.get_strings = adin_get_strings,
.get_stats = adin_get_stats,
#define MII_AM79C_IR_EN_ANEG 0x0100 /* IR enable Aneg Complete */
#define MII_AM79C_IR_IMASK_INIT (MII_AM79C_IR_EN_LINK | MII_AM79C_IR_EN_ANEG)
+#define MII_AM79C_IR_LINK_DOWN BIT(2)
+#define MII_AM79C_IR_ANEG_DONE BIT(0)
+#define MII_AM79C_IR_IMASK_STAT (MII_AM79C_IR_LINK_DOWN | MII_AM79C_IR_ANEG_DONE)
+
MODULE_DESCRIPTION("AMD PHY driver");
MODULE_AUTHOR("Heiko Schocher <hs@denx.de>");
MODULE_LICENSE("GPL");
{
int err;
- if (phydev->interrupts == PHY_INTERRUPT_ENABLED)
+ if (phydev->interrupts == PHY_INTERRUPT_ENABLED) {
+ err = am79c_ack_interrupt(phydev);
+ if (err)
+ return err;
+
err = phy_write(phydev, MII_AM79C_IR, MII_AM79C_IR_IMASK_INIT);
- else
+ } else {
err = phy_write(phydev, MII_AM79C_IR, 0);
+ if (err)
+ return err;
+
+ err = am79c_ack_interrupt(phydev);
+ }
return err;
}
+static irqreturn_t am79c_handle_interrupt(struct phy_device *phydev)
+{
+ int irq_status;
+
+ irq_status = phy_read(phydev, MII_AM79C_IR);
+ if (irq_status < 0) {
+ phy_error(phydev);
+ return IRQ_NONE;
+ }
+
+ if (!(irq_status & MII_AM79C_IR_IMASK_STAT))
+ return IRQ_NONE;
+
+ phy_trigger_machine(phydev);
+
+ return IRQ_HANDLED;
+}
+
static struct phy_driver am79c_driver[] = { {
.phy_id = PHY_ID_AM79C874,
.name = "AM79C874",
.phy_id_mask = 0xfffffff0,
/* PHY_BASIC_FEATURES */
.config_init = am79c_config_init,
- .ack_interrupt = am79c_ack_interrupt,
.config_intr = am79c_config_intr,
+ .handle_interrupt = am79c_handle_interrupt,
} };
module_phy_driver(am79c_driver);
#define MII_DP83640_MISR_LINK_INT_EN 0x20
#define MII_DP83640_MISR_ED_INT_EN 0x40
#define MII_DP83640_MISR_LQ_INT_EN 0x80
+#define MII_DP83640_MISR_ANC_INT 0x400
+#define MII_DP83640_MISR_DUP_INT 0x800
+#define MII_DP83640_MISR_SPD_INT 0x1000
+#define MII_DP83640_MISR_LINK_INT 0x2000
+#define MII_DP83640_MISR_INT_MASK (MII_DP83640_MISR_ANC_INT |\
+ MII_DP83640_MISR_DUP_INT |\
+ MII_DP83640_MISR_SPD_INT |\
+ MII_DP83640_MISR_LINK_INT)
/* phyter seems to miss the mark by 16 ns */
#define ADJTIME_FIX 16
static int is_sync(struct sk_buff *skb, int type)
{
struct ptp_header *hdr;
- u8 msgtype;
hdr = ptp_parse_header(skb, type);
if (!hdr)
return 0;
- msgtype = ptp_get_msgtype(hdr, type);
-
- return (msgtype & 0xf) == 0;
+ return ptp_get_msgtype(hdr, type) == PTP_MSGTYPE_SYNC;
}
static void dp83640_free_clocks(void)
int err;
if (phydev->interrupts == PHY_INTERRUPT_ENABLED) {
+ err = dp83640_ack_interrupt(phydev);
+ if (err)
+ return err;
+
misr = phy_read(phydev, MII_DP83640_MISR);
if (misr < 0)
return misr;
MII_DP83640_MISR_DUP_INT_EN |
MII_DP83640_MISR_SPD_INT_EN |
MII_DP83640_MISR_LINK_INT_EN);
- return phy_write(phydev, MII_DP83640_MISR, misr);
+ err = phy_write(phydev, MII_DP83640_MISR, misr);
+ if (err)
+ return err;
+
+ return dp83640_ack_interrupt(phydev);
}
}
+static irqreturn_t dp83640_handle_interrupt(struct phy_device *phydev)
+{
+ int irq_status;
+
+ irq_status = phy_read(phydev, MII_DP83640_MISR);
+ if (irq_status < 0) {
+ phy_error(phydev);
+ return IRQ_NONE;
+ }
+
+ if (!(irq_status & MII_DP83640_MISR_INT_MASK))
+ return IRQ_NONE;
+
+ phy_trigger_machine(phydev);
+
+ return IRQ_HANDLED;
+}
+
static int dp83640_hwtstamp(struct mii_timestamper *mii_ts, struct ifreq *ifr)
{
struct dp83640_private *dp83640 =
.remove = dp83640_remove,
.soft_reset = dp83640_soft_reset,
.config_init = dp83640_config_init,
- .ack_interrupt = dp83640_ack_interrupt,
.config_intr = dp83640_config_intr,
+ .handle_interrupt = dp83640_handle_interrupt,
};
static int __init dp83640_init(void)
u16 fx_sd_enable;
};
-static int dp83822_ack_interrupt(struct phy_device *phydev)
-{
- int err;
-
- err = phy_read(phydev, MII_DP83822_MISR1);
- if (err < 0)
- return err;
-
- err = phy_read(phydev, MII_DP83822_MISR2);
- if (err < 0)
- return err;
-
- return 0;
-}
-
static int dp83822_set_wol(struct phy_device *phydev,
struct ethtool_wolinfo *wol)
{
return phy_write(phydev, MII_DP83822_PHYSCR, physcr_status);
}
+static irqreturn_t dp83822_handle_interrupt(struct phy_device *phydev)
+{
+ int irq_status;
+
+ /* The MISR1 and MISR2 registers are holding the interrupt status in
+ * the upper half (15:8), while the lower half (7:0) is used for
+ * controlling the interrupt enable state of those individual interrupt
+ * sources. To determine the possible interrupt sources, just read the
+ * MISR* register and use it directly to know which interrupts have
+ * been enabled previously or not.
+ */
+ irq_status = phy_read(phydev, MII_DP83822_MISR1);
+ if (irq_status < 0) {
+ phy_error(phydev);
+ return IRQ_NONE;
+ }
+ if (irq_status & ((irq_status & GENMASK(7, 0)) << 8))
+ goto trigger_machine;
+
+ irq_status = phy_read(phydev, MII_DP83822_MISR2);
+ if (irq_status < 0) {
+ phy_error(phydev);
+ return IRQ_NONE;
+ }
+ if (irq_status & ((irq_status & GENMASK(7, 0)) << 8))
+ goto trigger_machine;
+
+ return IRQ_NONE;
+
+trigger_machine:
+ phy_trigger_machine(phydev);
+
+ return IRQ_HANDLED;
+}
+
static int dp8382x_disable_wol(struct phy_device *phydev)
{
int value = DP83822_WOL_EN | DP83822_WOL_MAGIC_EN |
.read_status = dp83822_read_status, \
.get_wol = dp83822_get_wol, \
.set_wol = dp83822_set_wol, \
- .ack_interrupt = dp83822_ack_interrupt, \
.config_intr = dp83822_config_intr, \
+ .handle_interrupt = dp83822_handle_interrupt, \
.suspend = dp83822_suspend, \
.resume = dp83822_resume, \
}
.config_init = dp8382x_config_init, \
.get_wol = dp83822_get_wol, \
.set_wol = dp83822_set_wol, \
- .ack_interrupt = dp83822_ack_interrupt, \
.config_intr = dp83822_config_intr, \
+ .handle_interrupt = dp83822_handle_interrupt, \
.suspend = dp83822_suspend, \
.resume = dp83822_resume, \
}
DP83848_MISR_SPD_INT_EN | \
DP83848_MISR_LINK_INT_EN)
+#define DP83848_MISR_RHF_INT BIT(8)
+#define DP83848_MISR_FHF_INT BIT(9)
+#define DP83848_MISR_ANC_INT BIT(10)
+#define DP83848_MISR_DUP_INT BIT(11)
+#define DP83848_MISR_SPD_INT BIT(12)
+#define DP83848_MISR_LINK_INT BIT(13)
+#define DP83848_MISR_ED_INT BIT(14)
+
+#define DP83848_INT_MASK \
+ (DP83848_MISR_ANC_INT | \
+ DP83848_MISR_DUP_INT | \
+ DP83848_MISR_SPD_INT | \
+ DP83848_MISR_LINK_INT)
+
static int dp83848_ack_interrupt(struct phy_device *phydev)
{
int err = phy_read(phydev, DP83848_MISR);
return control;
if (phydev->interrupts == PHY_INTERRUPT_ENABLED) {
+ ret = dp83848_ack_interrupt(phydev);
+ if (ret)
+ return ret;
+
control |= DP83848_MICR_INT_OE;
control |= DP83848_MICR_INTEN;
ret = phy_write(phydev, DP83848_MISR, DP83848_INT_EN_MASK);
if (ret < 0)
return ret;
+
+ ret = phy_write(phydev, DP83848_MICR, control);
} else {
control &= ~DP83848_MICR_INTEN;
+ ret = phy_write(phydev, DP83848_MICR, control);
+ if (ret)
+ return ret;
+
+ ret = dp83848_ack_interrupt(phydev);
+ }
+
+ return ret;
+}
+
+static irqreturn_t dp83848_handle_interrupt(struct phy_device *phydev)
+{
+ int irq_status;
+
+ irq_status = phy_read(phydev, DP83848_MISR);
+ if (irq_status < 0) {
+ phy_error(phydev);
+ return IRQ_NONE;
}
- return phy_write(phydev, DP83848_MICR, control);
+ if (!(irq_status & DP83848_INT_MASK))
+ return IRQ_NONE;
+
+ phy_trigger_machine(phydev);
+
+ return IRQ_HANDLED;
}
static int dp83848_config_init(struct phy_device *phydev)
.resume = genphy_resume, \
\
/* IRQ related */ \
- .ack_interrupt = dp83848_ack_interrupt, \
.config_intr = dp83848_config_intr, \
+ .handle_interrupt = dp83848_handle_interrupt, \
}
static struct phy_driver dp83848_driver[] = {
static int dp83867_config_intr(struct phy_device *phydev)
{
- int micr_status;
+ int micr_status, err;
if (phydev->interrupts == PHY_INTERRUPT_ENABLED) {
+ err = dp83867_ack_interrupt(phydev);
+ if (err)
+ return err;
+
micr_status = phy_read(phydev, MII_DP83867_MICR);
if (micr_status < 0)
return micr_status;
MII_DP83867_MICR_DUP_MODE_CHNG_INT_EN |
MII_DP83867_MICR_SLEEP_MODE_CHNG_INT_EN);
- return phy_write(phydev, MII_DP83867_MICR, micr_status);
+ err = phy_write(phydev, MII_DP83867_MICR, micr_status);
+ } else {
+ micr_status = 0x0;
+ err = phy_write(phydev, MII_DP83867_MICR, micr_status);
+ if (err)
+ return err;
+
+ err = dp83867_ack_interrupt(phydev);
}
- micr_status = 0x0;
- return phy_write(phydev, MII_DP83867_MICR, micr_status);
+ return err;
+}
+
+static irqreturn_t dp83867_handle_interrupt(struct phy_device *phydev)
+{
+ int irq_status, irq_enabled;
+
+ irq_status = phy_read(phydev, MII_DP83867_ISR);
+ if (irq_status < 0) {
+ phy_error(phydev);
+ return IRQ_NONE;
+ }
+
+ irq_enabled = phy_read(phydev, MII_DP83867_MICR);
+ if (irq_enabled < 0) {
+ phy_error(phydev);
+ return IRQ_NONE;
+ }
+
+ if (!(irq_status & irq_enabled))
+ return IRQ_NONE;
+
+ phy_trigger_machine(phydev);
+
+ return IRQ_HANDLED;
}
static int dp83867_read_status(struct phy_device *phydev)
.set_wol = dp83867_set_wol,
/* IRQ related */
- .ack_interrupt = dp83867_ack_interrupt,
.config_intr = dp83867_config_intr,
+ .handle_interrupt = dp83867_handle_interrupt,
.suspend = genphy_suspend,
.resume = genphy_resume,
static int dp83869_config_intr(struct phy_device *phydev)
{
- int micr_status = 0;
+ int micr_status = 0, err;
if (phydev->interrupts == PHY_INTERRUPT_ENABLED) {
+ err = dp83869_ack_interrupt(phydev);
+ if (err)
+ return err;
+
micr_status = phy_read(phydev, MII_DP83869_MICR);
if (micr_status < 0)
return micr_status;
MII_DP83869_MICR_DUP_MODE_CHNG_INT_EN |
MII_DP83869_MICR_SLEEP_MODE_CHNG_INT_EN);
- return phy_write(phydev, MII_DP83869_MICR, micr_status);
+ err = phy_write(phydev, MII_DP83869_MICR, micr_status);
+ } else {
+ err = phy_write(phydev, MII_DP83869_MICR, micr_status);
+ if (err)
+ return err;
+
+ err = dp83869_ack_interrupt(phydev);
}
- return phy_write(phydev, MII_DP83869_MICR, micr_status);
+ return err;
+}
+
+static irqreturn_t dp83869_handle_interrupt(struct phy_device *phydev)
+{
+ int irq_status, irq_enabled;
+
+ irq_status = phy_read(phydev, MII_DP83869_ISR);
+ if (irq_status < 0) {
+ phy_error(phydev);
+ return IRQ_NONE;
+ }
+
+ irq_enabled = phy_read(phydev, MII_DP83869_MICR);
+ if (irq_enabled < 0) {
+ phy_error(phydev);
+ return IRQ_NONE;
+ }
+
+ if (!(irq_status & irq_enabled))
+ return IRQ_NONE;
+
+ phy_trigger_machine(phydev);
+
+ return IRQ_HANDLED;
}
static int dp83869_set_wol(struct phy_device *phydev,
.soft_reset = dp83869_phy_reset,
/* IRQ related */
- .ack_interrupt = dp83869_ack_interrupt,
.config_intr = dp83869_config_intr,
+ .handle_interrupt = dp83869_handle_interrupt,
.read_status = dp83869_read_status,
.get_tunable = dp83869_get_tunable,
int misr_status, err;
if (phydev->interrupts == PHY_INTERRUPT_ENABLED) {
+ err = dp83811_ack_interrupt(phydev);
+ if (err)
+ return err;
+
misr_status = phy_read(phydev, MII_DP83811_INT_STAT1);
if (misr_status < 0)
return misr_status;
return err;
err = phy_write(phydev, MII_DP83811_INT_STAT3, 0);
+ if (err < 0)
+ return err;
+
+ err = dp83811_ack_interrupt(phydev);
}
return err;
}
+static irqreturn_t dp83811_handle_interrupt(struct phy_device *phydev)
+{
+ int irq_status;
+
+ /* The INT_STAT registers 1, 2 and 3 are holding the interrupt status
+ * in the upper half (15:8), while the lower half (7:0) is used for
+ * controlling the interrupt enable state of those individual interrupt
+ * sources. To determine the possible interrupt sources, just read the
+ * INT_STAT* register and use it directly to know which interrupts have
+ * been enabled previously or not.
+ */
+ irq_status = phy_read(phydev, MII_DP83811_INT_STAT1);
+ if (irq_status < 0) {
+ phy_error(phydev);
+ return IRQ_NONE;
+ }
+ if (irq_status & ((irq_status & GENMASK(7, 0)) << 8))
+ goto trigger_machine;
+
+ irq_status = phy_read(phydev, MII_DP83811_INT_STAT2);
+ if (irq_status < 0) {
+ phy_error(phydev);
+ return IRQ_NONE;
+ }
+ if (irq_status & ((irq_status & GENMASK(7, 0)) << 8))
+ goto trigger_machine;
+
+ irq_status = phy_read(phydev, MII_DP83811_INT_STAT3);
+ if (irq_status < 0) {
+ phy_error(phydev);
+ return IRQ_NONE;
+ }
+ if (irq_status & ((irq_status & GENMASK(7, 0)) << 8))
+ goto trigger_machine;
+
+ return IRQ_NONE;
+
+trigger_machine:
+ phy_trigger_machine(phydev);
+
+ return IRQ_HANDLED;
+}
+
static int dp83811_config_aneg(struct phy_device *phydev)
{
int value, err;
.soft_reset = dp83811_phy_reset,
.get_wol = dp83811_get_wol,
.set_wol = dp83811_set_wol,
- .ack_interrupt = dp83811_ack_interrupt,
.config_intr = dp83811_config_intr,
+ .handle_interrupt = dp83811_handle_interrupt,
.suspend = dp83811_suspend,
.resume = dp83811_resume,
},
return phy_write(phydev, IP10XX_SPEC_CTRL_STATUS, c);
}
+static int ip101a_g_ack_interrupt(struct phy_device *phydev)
+{
+ int err = phy_read(phydev, IP101A_G_IRQ_CONF_STATUS);
+
+ if (err < 0)
+ return err;
+
+ return 0;
+}
+
static int ip101a_g_config_intr(struct phy_device *phydev)
{
u16 val;
+ int err;
+
+ if (phydev->interrupts == PHY_INTERRUPT_ENABLED) {
+ err = ip101a_g_ack_interrupt(phydev);
+ if (err)
+ return err;
- if (phydev->interrupts == PHY_INTERRUPT_ENABLED)
/* INTR pin used: Speed/link/duplex will cause an interrupt */
val = IP101A_G_IRQ_PIN_USED;
- else
+ err = phy_write(phydev, IP101A_G_IRQ_CONF_STATUS, val);
+ } else {
val = IP101A_G_IRQ_ALL_MASK;
+ err = phy_write(phydev, IP101A_G_IRQ_CONF_STATUS, val);
+ if (err)
+ return err;
+
+ err = ip101a_g_ack_interrupt(phydev);
+ }
- return phy_write(phydev, IP101A_G_IRQ_CONF_STATUS, val);
+ return err;
}
-static int ip101a_g_did_interrupt(struct phy_device *phydev)
+static irqreturn_t ip101a_g_handle_interrupt(struct phy_device *phydev)
{
- int val = phy_read(phydev, IP101A_G_IRQ_CONF_STATUS);
+ int irq_status;
- if (val < 0)
- return 0;
+ irq_status = phy_read(phydev, IP101A_G_IRQ_CONF_STATUS);
+ if (irq_status < 0) {
+ phy_error(phydev);
+ return IRQ_NONE;
+ }
- return val & (IP101A_G_IRQ_SPEED_CHANGE |
- IP101A_G_IRQ_DUPLEX_CHANGE |
- IP101A_G_IRQ_LINK_CHANGE);
-}
+ if (!(irq_status & (IP101A_G_IRQ_SPEED_CHANGE |
+ IP101A_G_IRQ_DUPLEX_CHANGE |
+ IP101A_G_IRQ_LINK_CHANGE)))
+ return IRQ_NONE;
-static int ip101a_g_ack_interrupt(struct phy_device *phydev)
-{
- int err = phy_read(phydev, IP101A_G_IRQ_CONF_STATUS);
- if (err < 0)
- return err;
+ phy_trigger_machine(phydev);
- return 0;
+ return IRQ_HANDLED;
}
static struct phy_driver icplus_driver[] = {
/* PHY_BASIC_FEATURES */
.probe = ip101a_g_probe,
.config_intr = ip101a_g_config_intr,
- .did_interrupt = ip101a_g_did_interrupt,
- .ack_interrupt = ip101a_g_ack_interrupt,
+ .handle_interrupt = ip101a_g_handle_interrupt,
.config_init = &ip101a_g_config_init,
.suspend = genphy_suspend,
.resume = genphy_resume,
return (reg < 0) ? reg : 0;
}
-static int xway_gphy_did_interrupt(struct phy_device *phydev)
+static int xway_gphy_config_intr(struct phy_device *phydev)
{
- int reg;
+ u16 mask = 0;
+ int err;
- reg = phy_read(phydev, XWAY_MDIO_ISTAT);
- return reg & XWAY_MDIO_INIT_MASK;
+ if (phydev->interrupts == PHY_INTERRUPT_ENABLED) {
+ err = xway_gphy_ack_interrupt(phydev);
+ if (err)
+ return err;
+
+ mask = XWAY_MDIO_INIT_MASK;
+ err = phy_write(phydev, XWAY_MDIO_IMASK, mask);
+ } else {
+ err = phy_write(phydev, XWAY_MDIO_IMASK, mask);
+ if (err)
+ return err;
+
+ err = xway_gphy_ack_interrupt(phydev);
+ }
+
+ return err;
}
-static int xway_gphy_config_intr(struct phy_device *phydev)
+static irqreturn_t xway_gphy_handle_interrupt(struct phy_device *phydev)
{
- u16 mask = 0;
+ int irq_status;
- if (phydev->interrupts == PHY_INTERRUPT_ENABLED)
- mask = XWAY_MDIO_INIT_MASK;
+ irq_status = phy_read(phydev, XWAY_MDIO_ISTAT);
+ if (irq_status < 0) {
+ phy_error(phydev);
+ return IRQ_NONE;
+ }
+
+ if (!(irq_status & XWAY_MDIO_INIT_MASK))
+ return IRQ_NONE;
+
+ phy_trigger_machine(phydev);
- return phy_write(phydev, XWAY_MDIO_IMASK, mask);
+ return IRQ_HANDLED;
}
static struct phy_driver xway_gphy[] = {
/* PHY_GBIT_FEATURES */
.config_init = xway_gphy_config_init,
.config_aneg = xway_gphy14_config_aneg,
- .ack_interrupt = xway_gphy_ack_interrupt,
- .did_interrupt = xway_gphy_did_interrupt,
+ .handle_interrupt = xway_gphy_handle_interrupt,
.config_intr = xway_gphy_config_intr,
.suspend = genphy_suspend,
.resume = genphy_resume,
/* PHY_BASIC_FEATURES */
.config_init = xway_gphy_config_init,
.config_aneg = xway_gphy14_config_aneg,
- .ack_interrupt = xway_gphy_ack_interrupt,
- .did_interrupt = xway_gphy_did_interrupt,
+ .handle_interrupt = xway_gphy_handle_interrupt,
.config_intr = xway_gphy_config_intr,
.suspend = genphy_suspend,
.resume = genphy_resume,
/* PHY_GBIT_FEATURES */
.config_init = xway_gphy_config_init,
.config_aneg = xway_gphy14_config_aneg,
- .ack_interrupt = xway_gphy_ack_interrupt,
- .did_interrupt = xway_gphy_did_interrupt,
+ .handle_interrupt = xway_gphy_handle_interrupt,
.config_intr = xway_gphy_config_intr,
.suspend = genphy_suspend,
.resume = genphy_resume,
/* PHY_BASIC_FEATURES */
.config_init = xway_gphy_config_init,
.config_aneg = xway_gphy14_config_aneg,
- .ack_interrupt = xway_gphy_ack_interrupt,
- .did_interrupt = xway_gphy_did_interrupt,
+ .handle_interrupt = xway_gphy_handle_interrupt,
.config_intr = xway_gphy_config_intr,
.suspend = genphy_suspend,
.resume = genphy_resume,
.name = "Intel XWAY PHY11G (PEF 7071/PEF 7072) v1.5 / v1.6",
/* PHY_GBIT_FEATURES */
.config_init = xway_gphy_config_init,
- .ack_interrupt = xway_gphy_ack_interrupt,
- .did_interrupt = xway_gphy_did_interrupt,
+ .handle_interrupt = xway_gphy_handle_interrupt,
.config_intr = xway_gphy_config_intr,
.suspend = genphy_suspend,
.resume = genphy_resume,
.name = "Intel XWAY PHY22F (PEF 7061) v1.5 / v1.6",
/* PHY_BASIC_FEATURES */
.config_init = xway_gphy_config_init,
- .ack_interrupt = xway_gphy_ack_interrupt,
- .did_interrupt = xway_gphy_did_interrupt,
+ .handle_interrupt = xway_gphy_handle_interrupt,
.config_intr = xway_gphy_config_intr,
.suspend = genphy_suspend,
.resume = genphy_resume,
.name = "Intel XWAY PHY11G (xRX v1.1 integrated)",
/* PHY_GBIT_FEATURES */
.config_init = xway_gphy_config_init,
- .ack_interrupt = xway_gphy_ack_interrupt,
- .did_interrupt = xway_gphy_did_interrupt,
+ .handle_interrupt = xway_gphy_handle_interrupt,
.config_intr = xway_gphy_config_intr,
.suspend = genphy_suspend,
.resume = genphy_resume,
.name = "Intel XWAY PHY22F (xRX v1.1 integrated)",
/* PHY_BASIC_FEATURES */
.config_init = xway_gphy_config_init,
- .ack_interrupt = xway_gphy_ack_interrupt,
- .did_interrupt = xway_gphy_did_interrupt,
+ .handle_interrupt = xway_gphy_handle_interrupt,
.config_intr = xway_gphy_config_intr,
.suspend = genphy_suspend,
.resume = genphy_resume,
.name = "Intel XWAY PHY11G (xRX v1.2 integrated)",
/* PHY_GBIT_FEATURES */
.config_init = xway_gphy_config_init,
- .ack_interrupt = xway_gphy_ack_interrupt,
- .did_interrupt = xway_gphy_did_interrupt,
+ .handle_interrupt = xway_gphy_handle_interrupt,
.config_intr = xway_gphy_config_intr,
.suspend = genphy_suspend,
.resume = genphy_resume,
.name = "Intel XWAY PHY22F (xRX v1.2 integrated)",
/* PHY_BASIC_FEATURES */
.config_init = xway_gphy_config_init,
- .ack_interrupt = xway_gphy_ack_interrupt,
- .did_interrupt = xway_gphy_did_interrupt,
+ .handle_interrupt = xway_gphy_handle_interrupt,
.config_intr = xway_gphy_config_intr,
.suspend = genphy_suspend,
.resume = genphy_resume,
#define MII_LXT970_ISR 18 /* Interrupt Status Register */
+#define MII_LXT970_IRS_MINT BIT(15)
+
#define MII_LXT970_CONFIG 19 /* Configuration Register */
/* ------------------------------------------------------------------------- */
#define MII_LXT971_IER_IEN 0x00f2
#define MII_LXT971_ISR 19 /* Interrupt Status Register */
+#define MII_LXT971_ISR_MASK 0x00f0
/* register definitions for the 973 */
#define MII_LXT973_PCR 16 /* Port Configuration Register */
static int lxt970_config_intr(struct phy_device *phydev)
{
- if (phydev->interrupts == PHY_INTERRUPT_ENABLED)
- return phy_write(phydev, MII_LXT970_IER, MII_LXT970_IER_IEN);
- else
- return phy_write(phydev, MII_LXT970_IER, 0);
+ int err;
+
+ if (phydev->interrupts == PHY_INTERRUPT_ENABLED) {
+ err = lxt970_ack_interrupt(phydev);
+ if (err)
+ return err;
+
+ err = phy_write(phydev, MII_LXT970_IER, MII_LXT970_IER_IEN);
+ } else {
+ err = phy_write(phydev, MII_LXT970_IER, 0);
+ if (err)
+ return err;
+
+ err = lxt970_ack_interrupt(phydev);
+ }
+
+ return err;
+}
+
+static irqreturn_t lxt970_handle_interrupt(struct phy_device *phydev)
+{
+ int irq_status;
+
+ /* The interrupt status register is cleared by reading BMSR
+ * followed by MII_LXT970_ISR
+ */
+ irq_status = phy_read(phydev, MII_BMSR);
+ if (irq_status < 0) {
+ phy_error(phydev);
+ return IRQ_NONE;
+ }
+
+ irq_status = phy_read(phydev, MII_LXT970_ISR);
+ if (irq_status < 0) {
+ phy_error(phydev);
+ return IRQ_NONE;
+ }
+
+ if (!(irq_status & MII_LXT970_IRS_MINT))
+ return IRQ_NONE;
+
+ phy_trigger_machine(phydev);
+
+ return IRQ_HANDLED;
}
static int lxt970_config_init(struct phy_device *phydev)
static int lxt971_config_intr(struct phy_device *phydev)
{
- if (phydev->interrupts == PHY_INTERRUPT_ENABLED)
- return phy_write(phydev, MII_LXT971_IER, MII_LXT971_IER_IEN);
- else
- return phy_write(phydev, MII_LXT971_IER, 0);
+ int err;
+
+ if (phydev->interrupts == PHY_INTERRUPT_ENABLED) {
+ err = lxt971_ack_interrupt(phydev);
+ if (err)
+ return err;
+
+ err = phy_write(phydev, MII_LXT971_IER, MII_LXT971_IER_IEN);
+ } else {
+ err = phy_write(phydev, MII_LXT971_IER, 0);
+ if (err)
+ return err;
+
+ err = lxt971_ack_interrupt(phydev);
+ }
+
+ return err;
+}
+
+static irqreturn_t lxt971_handle_interrupt(struct phy_device *phydev)
+{
+ int irq_status;
+
+ irq_status = phy_read(phydev, MII_LXT971_ISR);
+ if (irq_status < 0) {
+ phy_error(phydev);
+ return IRQ_NONE;
+ }
+
+ if (!(irq_status & MII_LXT971_ISR_MASK))
+ return IRQ_NONE;
+
+ phy_trigger_machine(phydev);
+
+ return IRQ_HANDLED;
}
/*
.phy_id_mask = 0xfffffff0,
/* PHY_BASIC_FEATURES */
.config_init = lxt970_config_init,
- .ack_interrupt = lxt970_ack_interrupt,
.config_intr = lxt970_config_intr,
+ .handle_interrupt = lxt970_handle_interrupt,
}, {
.phy_id = 0x001378e0,
.name = "LXT971",
.phy_id_mask = 0xfffffff0,
/* PHY_BASIC_FEATURES */
- .ack_interrupt = lxt971_ack_interrupt,
.config_intr = lxt971_config_intr,
+ .handle_interrupt = lxt971_handle_interrupt,
.suspend = genphy_suspend,
.resume = genphy_resume,
}, {
{
int err;
- if (phydev->interrupts == PHY_INTERRUPT_ENABLED)
+ if (phydev->interrupts == PHY_INTERRUPT_ENABLED) {
+ err = marvell_ack_interrupt(phydev);
+ if (err)
+ return err;
+
err = phy_write(phydev, MII_M1011_IMASK,
MII_M1011_IMASK_INIT);
- else
+ } else {
err = phy_write(phydev, MII_M1011_IMASK,
MII_M1011_IMASK_CLEAR);
+ if (err)
+ return err;
+
+ err = marvell_ack_interrupt(phydev);
+ }
return err;
}
+static irqreturn_t marvell_handle_interrupt(struct phy_device *phydev)
+{
+ int irq_status;
+
+ irq_status = phy_read(phydev, MII_M1011_IEVENT);
+ if (irq_status < 0) {
+ phy_error(phydev);
+ return IRQ_NONE;
+ }
+
+ if (!(irq_status & MII_M1011_IMASK_INIT))
+ return IRQ_NONE;
+
+ phy_trigger_machine(phydev);
+
+ return IRQ_HANDLED;
+}
+
static int marvell_set_polarity(struct phy_device *phydev, int polarity)
{
int reg;
return err;
/* PHY reset is necessary after changing MODE[2:0] */
- err = phy_modify(phydev, MII_88E1510_GEN_CTRL_REG_1, 0,
- MII_88E1510_GEN_CTRL_REG_1_RESET);
+ err = phy_set_bits(phydev, MII_88E1510_GEN_CTRL_REG_1,
+ MII_88E1510_GEN_CTRL_REG_1_RESET);
if (err < 0)
return err;
return (retval < 0) ? retval : (retval & MII_M1011_PHY_STATUS_RESOLVED);
}
-static int m88e1121_did_interrupt(struct phy_device *phydev)
-{
- int imask;
-
- imask = phy_read(phydev, MII_M1011_IEVENT);
-
- if (imask & MII_M1011_IMASK_INIT)
- return 1;
-
- return 0;
-}
-
static void m88e1318_get_wol(struct phy_device *phydev,
struct ethtool_wolinfo *wol)
{
__phy_read(phydev, MII_M1011_IEVENT);
/* Enable the WOL interrupt */
- err = __phy_modify(phydev, MII_88E1318S_PHY_CSIER, 0,
- MII_88E1318S_PHY_CSIER_WOL_EIE);
+ err = __phy_set_bits(phydev, MII_88E1318S_PHY_CSIER,
+ MII_88E1318S_PHY_CSIER_WOL_EIE);
if (err < 0)
goto error;
goto error;
/* Clear WOL status and enable magic packet matching */
- err = __phy_modify(phydev, MII_88E1318S_PHY_WOL_CTRL, 0,
- MII_88E1318S_PHY_WOL_CTRL_CLEAR_WOL_STATUS |
- MII_88E1318S_PHY_WOL_CTRL_MAGIC_PACKET_MATCH_ENABLE);
+ err = __phy_set_bits(phydev, MII_88E1318S_PHY_WOL_CTRL,
+ MII_88E1318S_PHY_WOL_CTRL_CLEAR_WOL_STATUS |
+ MII_88E1318S_PHY_WOL_CTRL_MAGIC_PACKET_MATCH_ENABLE);
if (err < 0)
goto error;
} else {
return bmsr;
if (bmcr & BMCR_ANENABLE) {
- ret = phy_modify(phydev, MII_BMCR, BMCR_ANENABLE, 0);
+ ret = phy_clear_bits(phydev, MII_BMCR, BMCR_ANENABLE);
if (ret < 0)
return ret;
ret = genphy_soft_reset(phydev);
.probe = marvell_probe,
.config_init = marvell_config_init,
.config_aneg = m88e1101_config_aneg,
- .ack_interrupt = marvell_ack_interrupt,
.config_intr = marvell_config_intr,
+ .handle_interrupt = marvell_handle_interrupt,
.resume = genphy_resume,
.suspend = genphy_suspend,
.read_page = marvell_read_page,
.probe = marvell_probe,
.config_init = m88e1111_config_init,
.config_aneg = marvell_config_aneg,
- .ack_interrupt = marvell_ack_interrupt,
.config_intr = marvell_config_intr,
+ .handle_interrupt = marvell_handle_interrupt,
.resume = genphy_resume,
.suspend = genphy_suspend,
.read_page = marvell_read_page,
.config_init = m88e1111_config_init,
.config_aneg = m88e1111_config_aneg,
.read_status = marvell_read_status,
- .ack_interrupt = marvell_ack_interrupt,
.config_intr = marvell_config_intr,
+ .handle_interrupt = marvell_handle_interrupt,
.resume = genphy_resume,
.suspend = genphy_suspend,
.read_page = marvell_read_page,
.config_init = m88e1111_config_init,
.config_aneg = m88e1111_config_aneg,
.read_status = marvell_read_status,
- .ack_interrupt = marvell_ack_interrupt,
.config_intr = marvell_config_intr,
+ .handle_interrupt = marvell_handle_interrupt,
.resume = genphy_resume,
.suspend = genphy_suspend,
.read_page = marvell_read_page,
.probe = marvell_probe,
.config_init = m88e1118_config_init,
.config_aneg = m88e1118_config_aneg,
- .ack_interrupt = marvell_ack_interrupt,
.config_intr = marvell_config_intr,
+ .handle_interrupt = marvell_handle_interrupt,
.resume = genphy_resume,
.suspend = genphy_suspend,
.read_page = marvell_read_page,
.config_init = marvell_config_init,
.config_aneg = m88e1121_config_aneg,
.read_status = marvell_read_status,
- .ack_interrupt = marvell_ack_interrupt,
.config_intr = marvell_config_intr,
- .did_interrupt = m88e1121_did_interrupt,
+ .handle_interrupt = marvell_handle_interrupt,
.resume = genphy_resume,
.suspend = genphy_suspend,
.read_page = marvell_read_page,
.config_init = m88e1318_config_init,
.config_aneg = m88e1318_config_aneg,
.read_status = marvell_read_status,
- .ack_interrupt = marvell_ack_interrupt,
.config_intr = marvell_config_intr,
- .did_interrupt = m88e1121_did_interrupt,
+ .handle_interrupt = marvell_handle_interrupt,
.get_wol = m88e1318_get_wol,
.set_wol = m88e1318_set_wol,
.resume = genphy_resume,
.config_init = m88e1145_config_init,
.config_aneg = m88e1101_config_aneg,
.read_status = genphy_read_status,
- .ack_interrupt = marvell_ack_interrupt,
.config_intr = marvell_config_intr,
+ .handle_interrupt = marvell_handle_interrupt,
.resume = genphy_resume,
.suspend = genphy_suspend,
.read_page = marvell_read_page,
.probe = marvell_probe,
.config_init = m88e1149_config_init,
.config_aneg = m88e1118_config_aneg,
- .ack_interrupt = marvell_ack_interrupt,
.config_intr = marvell_config_intr,
+ .handle_interrupt = marvell_handle_interrupt,
.resume = genphy_resume,
.suspend = genphy_suspend,
.read_page = marvell_read_page,
.probe = marvell_probe,
.config_init = m88e1111_config_init,
.config_aneg = marvell_config_aneg,
- .ack_interrupt = marvell_ack_interrupt,
.config_intr = marvell_config_intr,
+ .handle_interrupt = marvell_handle_interrupt,
.resume = genphy_resume,
.suspend = genphy_suspend,
.read_page = marvell_read_page,
/* PHY_GBIT_FEATURES */
.probe = marvell_probe,
.config_init = m88e1116r_config_init,
- .ack_interrupt = marvell_ack_interrupt,
.config_intr = marvell_config_intr,
+ .handle_interrupt = marvell_handle_interrupt,
.resume = genphy_resume,
.suspend = genphy_suspend,
.read_page = marvell_read_page,
.config_init = m88e1510_config_init,
.config_aneg = m88e1510_config_aneg,
.read_status = marvell_read_status,
- .ack_interrupt = marvell_ack_interrupt,
.config_intr = marvell_config_intr,
- .did_interrupt = m88e1121_did_interrupt,
+ .handle_interrupt = marvell_handle_interrupt,
.get_wol = m88e1318_get_wol,
.set_wol = m88e1318_set_wol,
.resume = marvell_resume,
.config_init = marvell_config_init,
.config_aneg = m88e1510_config_aneg,
.read_status = marvell_read_status,
- .ack_interrupt = marvell_ack_interrupt,
.config_intr = marvell_config_intr,
- .did_interrupt = m88e1121_did_interrupt,
+ .handle_interrupt = marvell_handle_interrupt,
.resume = genphy_resume,
.suspend = genphy_suspend,
.read_page = marvell_read_page,
.config_init = marvell_config_init,
.config_aneg = m88e1510_config_aneg,
.read_status = marvell_read_status,
- .ack_interrupt = marvell_ack_interrupt,
.config_intr = marvell_config_intr,
- .did_interrupt = m88e1121_did_interrupt,
+ .handle_interrupt = marvell_handle_interrupt,
.resume = genphy_resume,
.suspend = genphy_suspend,
.read_page = marvell_read_page,
.config_init = m88e3016_config_init,
.aneg_done = marvell_aneg_done,
.read_status = marvell_read_status,
- .ack_interrupt = marvell_ack_interrupt,
.config_intr = marvell_config_intr,
- .did_interrupt = m88e1121_did_interrupt,
+ .handle_interrupt = marvell_handle_interrupt,
.resume = genphy_resume,
.suspend = genphy_suspend,
.read_page = marvell_read_page,
.config_init = marvell_config_init,
.config_aneg = m88e6390_config_aneg,
.read_status = marvell_read_status,
- .ack_interrupt = marvell_ack_interrupt,
.config_intr = marvell_config_intr,
- .did_interrupt = m88e1121_did_interrupt,
+ .handle_interrupt = marvell_handle_interrupt,
.resume = genphy_resume,
.suspend = genphy_suspend,
.read_page = marvell_read_page,
.config_init = marvell_config_init,
.config_aneg = m88e1510_config_aneg,
.read_status = marvell_read_status,
- .ack_interrupt = marvell_ack_interrupt,
.config_intr = marvell_config_intr,
- .did_interrupt = m88e1121_did_interrupt,
+ .handle_interrupt = marvell_handle_interrupt,
.resume = genphy_resume,
.suspend = genphy_suspend,
.read_page = marvell_read_page,
.config_init = marvell_config_init,
.config_aneg = m88e1510_config_aneg,
.read_status = marvell_read_status,
- .ack_interrupt = marvell_ack_interrupt,
.config_intr = marvell_config_intr,
- .did_interrupt = m88e1121_did_interrupt,
+ .handle_interrupt = marvell_handle_interrupt,
.resume = genphy_resume,
.suspend = genphy_suspend,
.read_page = marvell_read_page,
#endif
/**
- * mdiobus_create_device_from_board_info - create a full MDIO device given
+ * mdiobus_create_device - create a full MDIO device given
* a mdio_board_info structure
* @bus: MDIO bus to create the devices on
* @bi: mdio_board_info structure describing the devices
/* de-assert bus level PHY GPIO reset */
gpiod = devm_gpiod_get_optional(&bus->dev, "reset", GPIOD_OUT_LOW);
if (IS_ERR(gpiod)) {
- dev_err(&bus->dev, "mii_bus %s couldn't get reset GPIO\n",
- bus->id);
+ err = dev_err_probe(&bus->dev, PTR_ERR(gpiod),
+ "mii_bus %s couldn't get reset GPIO\n",
+ bus->id);
device_del(&bus->dev);
- return PTR_ERR(gpiod);
+ return err;
} else if (gpiod) {
bus->reset_gpiod = gpiod;
int ret;
if (phydev->interrupts == PHY_INTERRUPT_ENABLED) {
+ /* Ack any pending IRQ */
+ ret = meson_gxl_ack_interrupt(phydev);
+ if (ret)
+ return ret;
+
val = INTSRC_ANEG_PR
| INTSRC_PARALLEL_FAULT
| INTSRC_ANEG_LP_ACK
| INTSRC_LINK_DOWN
| INTSRC_REMOTE_FAULT
| INTSRC_ANEG_COMPLETE;
+ ret = phy_write(phydev, INTSRC_MASK, val);
} else {
val = 0;
+ ret = phy_write(phydev, INTSRC_MASK, val);
+
+ /* Ack any pending IRQ */
+ ret = meson_gxl_ack_interrupt(phydev);
}
- /* Ack any pending IRQ */
- ret = meson_gxl_ack_interrupt(phydev);
- if (ret)
- return ret;
+ return ret;
+}
+
+static irqreturn_t meson_gxl_handle_interrupt(struct phy_device *phydev)
+{
+ int irq_status;
+
+ irq_status = phy_read(phydev, INTSRC_FLAG);
+ if (irq_status < 0) {
+ phy_error(phydev);
+ return IRQ_NONE;
+ }
+
+ if (irq_status == 0)
+ return IRQ_NONE;
+
+ phy_trigger_machine(phydev);
- return phy_write(phydev, INTSRC_MASK, val);
+ return IRQ_HANDLED;
}
static struct phy_driver meson_gxl_phy[] = {
.soft_reset = genphy_soft_reset,
.config_init = meson_gxl_config_init,
.read_status = meson_gxl_read_status,
- .ack_interrupt = meson_gxl_ack_interrupt,
.config_intr = meson_gxl_config_intr,
+ .handle_interrupt = meson_gxl_handle_interrupt,
.suspend = genphy_suspend,
.resume = genphy_resume,
}, {
/* PHY_BASIC_FEATURES */
.flags = PHY_IS_INTERNAL,
.soft_reset = genphy_soft_reset,
- .ack_interrupt = meson_gxl_ack_interrupt,
.config_intr = meson_gxl_config_intr,
+ .handle_interrupt = meson_gxl_handle_interrupt,
.suspend = genphy_suspend,
.resume = genphy_resume,
},
#define KSZPHY_INTCS_LINK_UP BIT(8)
#define KSZPHY_INTCS_ALL (KSZPHY_INTCS_LINK_UP |\
KSZPHY_INTCS_LINK_DOWN)
+#define KSZPHY_INTCS_LINK_DOWN_STATUS BIT(2)
+#define KSZPHY_INTCS_LINK_UP_STATUS BIT(0)
+#define KSZPHY_INTCS_STATUS (KSZPHY_INTCS_LINK_DOWN_STATUS |\
+ KSZPHY_INTCS_LINK_UP_STATUS)
/* PHY Control 1 */
#define MII_KSZPHY_CTRL_1 0x1e
static int kszphy_config_intr(struct phy_device *phydev)
{
const struct kszphy_type *type = phydev->drv->driver_data;
- int temp;
+ int temp, err;
u16 mask;
if (type && type->interrupt_level_mask)
phy_write(phydev, MII_KSZPHY_CTRL, temp);
/* enable / disable interrupts */
- if (phydev->interrupts == PHY_INTERRUPT_ENABLED)
+ if (phydev->interrupts == PHY_INTERRUPT_ENABLED) {
+ err = kszphy_ack_interrupt(phydev);
+ if (err)
+ return err;
+
temp = KSZPHY_INTCS_ALL;
- else
+ err = phy_write(phydev, MII_KSZPHY_INTCS, temp);
+ } else {
temp = 0;
+ err = phy_write(phydev, MII_KSZPHY_INTCS, temp);
+ if (err)
+ return err;
+
+ err = kszphy_ack_interrupt(phydev);
+ }
+
+ return err;
+}
+
+static irqreturn_t kszphy_handle_interrupt(struct phy_device *phydev)
+{
+ int irq_status;
+
+ irq_status = phy_read(phydev, MII_KSZPHY_INTCS);
+ if (irq_status < 0) {
+ phy_error(phydev);
+ return IRQ_NONE;
+ }
+
+ if (!(irq_status & KSZPHY_INTCS_STATUS))
+ return IRQ_NONE;
+
+ phy_trigger_machine(phydev);
- return phy_write(phydev, MII_KSZPHY_INTCS, temp);
+ return IRQ_HANDLED;
}
static int kszphy_rmii_clk_sel(struct phy_device *phydev, bool val)
/* PHY_BASIC_FEATURES */
.driver_data = &ks8737_type,
.config_init = kszphy_config_init,
- .ack_interrupt = kszphy_ack_interrupt,
.config_intr = kszphy_config_intr,
+ .handle_interrupt = kszphy_handle_interrupt,
.suspend = genphy_suspend,
.resume = genphy_resume,
}, {
.driver_data = &ksz8021_type,
.probe = kszphy_probe,
.config_init = kszphy_config_init,
- .ack_interrupt = kszphy_ack_interrupt,
.config_intr = kszphy_config_intr,
+ .handle_interrupt = kszphy_handle_interrupt,
.get_sset_count = kszphy_get_sset_count,
.get_strings = kszphy_get_strings,
.get_stats = kszphy_get_stats,
.driver_data = &ksz8021_type,
.probe = kszphy_probe,
.config_init = kszphy_config_init,
- .ack_interrupt = kszphy_ack_interrupt,
.config_intr = kszphy_config_intr,
+ .handle_interrupt = kszphy_handle_interrupt,
.get_sset_count = kszphy_get_sset_count,
.get_strings = kszphy_get_strings,
.get_stats = kszphy_get_stats,
.probe = kszphy_probe,
.config_init = ksz8041_config_init,
.config_aneg = ksz8041_config_aneg,
- .ack_interrupt = kszphy_ack_interrupt,
.config_intr = kszphy_config_intr,
+ .handle_interrupt = kszphy_handle_interrupt,
.get_sset_count = kszphy_get_sset_count,
.get_strings = kszphy_get_strings,
.get_stats = kszphy_get_stats,
.driver_data = &ksz8041_type,
.probe = kszphy_probe,
.config_init = kszphy_config_init,
- .ack_interrupt = kszphy_ack_interrupt,
.config_intr = kszphy_config_intr,
+ .handle_interrupt = kszphy_handle_interrupt,
.get_sset_count = kszphy_get_sset_count,
.get_strings = kszphy_get_strings,
.get_stats = kszphy_get_stats,
.driver_data = &ksz8051_type,
.probe = kszphy_probe,
.config_init = kszphy_config_init,
- .ack_interrupt = kszphy_ack_interrupt,
.config_intr = kszphy_config_intr,
+ .handle_interrupt = kszphy_handle_interrupt,
.get_sset_count = kszphy_get_sset_count,
.get_strings = kszphy_get_strings,
.get_stats = kszphy_get_stats,
.driver_data = &ksz8041_type,
.probe = kszphy_probe,
.config_init = kszphy_config_init,
- .ack_interrupt = kszphy_ack_interrupt,
.config_intr = kszphy_config_intr,
+ .handle_interrupt = kszphy_handle_interrupt,
.get_sset_count = kszphy_get_sset_count,
.get_strings = kszphy_get_strings,
.get_stats = kszphy_get_stats,
.driver_data = &ksz8081_type,
.probe = kszphy_probe,
.config_init = ksz8081_config_init,
- .ack_interrupt = kszphy_ack_interrupt,
.config_intr = kszphy_config_intr,
+ .handle_interrupt = kszphy_handle_interrupt,
.get_sset_count = kszphy_get_sset_count,
.get_strings = kszphy_get_strings,
.get_stats = kszphy_get_stats,
.phy_id_mask = MICREL_PHY_ID_MASK,
/* PHY_BASIC_FEATURES */
.config_init = ksz8061_config_init,
- .ack_interrupt = kszphy_ack_interrupt,
.config_intr = kszphy_config_intr,
+ .handle_interrupt = kszphy_handle_interrupt,
.suspend = genphy_suspend,
.resume = genphy_resume,
}, {
.probe = kszphy_probe,
.get_features = ksz9031_get_features,
.config_init = ksz9021_config_init,
- .ack_interrupt = kszphy_ack_interrupt,
.config_intr = kszphy_config_intr,
+ .handle_interrupt = kszphy_handle_interrupt,
.get_sset_count = kszphy_get_sset_count,
.get_strings = kszphy_get_strings,
.get_stats = kszphy_get_stats,
.config_init = ksz9031_config_init,
.soft_reset = genphy_soft_reset,
.read_status = ksz9031_read_status,
- .ack_interrupt = kszphy_ack_interrupt,
.config_intr = kszphy_config_intr,
+ .handle_interrupt = kszphy_handle_interrupt,
.get_sset_count = kszphy_get_sset_count,
.get_strings = kszphy_get_strings,
.get_stats = kszphy_get_stats,
.probe = kszphy_probe,
.config_init = ksz9131_config_init,
.read_status = genphy_read_status,
- .ack_interrupt = kszphy_ack_interrupt,
.config_intr = kszphy_config_intr,
+ .handle_interrupt = kszphy_handle_interrupt,
.get_sset_count = kszphy_get_sset_count,
.get_strings = kszphy_get_strings,
.get_stats = kszphy_get_stats,
LAN88XX_INT_MASK_LINK_CHANGE_);
} else {
rc = phy_write(phydev, LAN88XX_INT_MASK, 0);
+ if (rc)
+ return rc;
+
+ /* Ack interrupts after they have been disabled */
+ rc = phy_read(phydev, LAN88XX_INT_STS);
}
return rc < 0 ? rc : 0;
}
-static int lan88xx_phy_ack_interrupt(struct phy_device *phydev)
+static irqreturn_t lan88xx_handle_interrupt(struct phy_device *phydev)
{
- int rc = phy_read(phydev, LAN88XX_INT_STS);
+ int irq_status;
- return rc < 0 ? rc : 0;
+ irq_status = phy_read(phydev, LAN88XX_INT_STS);
+ if (irq_status < 0) {
+ phy_error(phydev);
+ return IRQ_NONE;
+ }
+
+ if (!(irq_status & LAN88XX_INT_STS_LINK_CHANGE_))
+ return IRQ_NONE;
+
+ phy_trigger_machine(phydev);
+
+ return IRQ_HANDLED;
}
static int lan88xx_suspend(struct phy_device *phydev)
.config_init = lan88xx_config_init,
.config_aneg = lan88xx_config_aneg,
- .ack_interrupt = lan88xx_phy_ack_interrupt,
.config_intr = lan88xx_phy_config_intr,
+ .handle_interrupt = lan88xx_handle_interrupt,
.suspend = lan88xx_suspend,
.resume = genphy_resume,
rc = phy_write(phydev, LAN87XX_INTERRUPT_MASK, 0x7FFF);
rc = phy_read(phydev, LAN87XX_INTERRUPT_SOURCE);
val = LAN87XX_MASK_LINK_UP | LAN87XX_MASK_LINK_DOWN;
- }
+ rc = phy_write(phydev, LAN87XX_INTERRUPT_MASK, val);
+ } else {
+ rc = phy_write(phydev, LAN87XX_INTERRUPT_MASK, val);
+ if (rc)
+ return rc;
- rc = phy_write(phydev, LAN87XX_INTERRUPT_MASK, val);
+ rc = phy_read(phydev, LAN87XX_INTERRUPT_SOURCE);
+ }
return rc < 0 ? rc : 0;
}
-static int lan87xx_phy_ack_interrupt(struct phy_device *phydev)
+static irqreturn_t lan87xx_handle_interrupt(struct phy_device *phydev)
{
- int rc = phy_read(phydev, LAN87XX_INTERRUPT_SOURCE);
+ int irq_status;
- return rc < 0 ? rc : 0;
+ irq_status = phy_read(phydev, LAN87XX_INTERRUPT_SOURCE);
+ if (irq_status < 0) {
+ phy_error(phydev);
+ return IRQ_NONE;
+ }
+
+ if (irq_status == 0)
+ return IRQ_NONE;
+
+ phy_trigger_machine(phydev);
+
+ return IRQ_HANDLED;
}
static int lan87xx_config_init(struct phy_device *phydev)
.config_init = lan87xx_config_init,
- .ack_interrupt = lan87xx_phy_ack_interrupt,
.config_intr = lan87xx_phy_config_intr,
+ .handle_interrupt = lan87xx_handle_interrupt,
.suspend = genphy_suspend,
.resume = genphy_resume,
switch (phydev->phy_id & phydev->drv->phy_id_mask) {
case PHY_ID_VSC856X:
- case PHY_ID_VSC8575:
case PHY_ID_VSC8582:
case PHY_ID_VSC8584:
INIT_LIST_HEAD(&vsc8531->macsec_flows);
phy_ts_base_write(phydev, MSCC_EXT_PAGE_ACCESS, MSCC_PHY_PAGE_1588);
- if (!cond || (cond && upper))
+ if (!cond || upper)
phy_ts_base_write(phydev, MSCC_PHY_TS_CSR_DATA_MSB, upper);
phy_ts_base_write(phydev, MSCC_PHY_TS_CSR_DATA_LSB, lower);
{
struct vsc8531_private *vsc8531 = phydev->priv;
bool base = phydev->mdio.addr == vsc8531->ts_base_addr;
- enum vsc85xx_ptp_msg_type msgs[] = {
- PTP_MSG_TYPE_SYNC,
- PTP_MSG_TYPE_DELAY_REQ
+ u8 msgs[] = {
+ PTP_MSGTYPE_SYNC,
+ PTP_MSGTYPE_DELAY_REQ
};
u32 val;
u8 i;
static int vsc85xx_ptp_conf(struct phy_device *phydev, enum ts_blk blk,
bool one_step, bool enable)
{
- enum vsc85xx_ptp_msg_type msgs[] = {
- PTP_MSG_TYPE_SYNC,
- PTP_MSG_TYPE_DELAY_REQ
+ u8 msgs[] = {
+ PTP_MSGTYPE_SYNC,
+ PTP_MSGTYPE_DELAY_REQ
};
u32 val;
u8 i;
if (blk == INGRESS)
vsc85xx_ts_ptp_action_flow(phydev, blk, msgs[i],
PTP_WRITE_NS);
- else if (msgs[i] == PTP_MSG_TYPE_SYNC && one_step)
+ else if (msgs[i] == PTP_MSGTYPE_SYNC && one_step)
/* no need to know Sync t when sending in one_step */
vsc85xx_ts_ptp_action_flow(phydev, blk, msgs[i],
PTP_WRITE_1588);
PTP_SAVE_IN_TS_FIFO = 11, /* invalid when writing in reg */
};
-enum vsc85xx_ptp_msg_type {
- PTP_MSG_TYPE_SYNC,
- PTP_MSG_TYPE_DELAY_REQ,
-};
-
struct vsc85xx_ptphdr {
u8 tsmt; /* transportSpecific | messageType */
u8 ver; /* reserved0 | versionPTP */
phy_write(phydev, NS_EXP_MEM_DATA, data);
}
-static int ns_config_intr(struct phy_device *phydev)
-{
- int err;
-
- if (phydev->interrupts == PHY_INTERRUPT_ENABLED)
- err = phy_write(phydev, DP83865_INT_MASK,
- DP83865_INT_MASK_DEFAULT);
- else
- err = phy_write(phydev, DP83865_INT_MASK, 0);
-
- return err;
-}
-
static int ns_ack_interrupt(struct phy_device *phydev)
{
int ret = phy_read(phydev, DP83865_INT_STATUS);
return ret;
}
+static irqreturn_t ns_handle_interrupt(struct phy_device *phydev)
+{
+ int irq_status;
+
+ irq_status = phy_read(phydev, DP83865_INT_STATUS);
+ if (irq_status < 0) {
+ phy_error(phydev);
+ return IRQ_NONE;
+ }
+
+ if (!(irq_status & DP83865_INT_MASK_DEFAULT))
+ return IRQ_NONE;
+
+ /* clear the interrupt */
+ phy_write(phydev, DP83865_INT_CLEAR, irq_status & ~0x7);
+
+ phy_trigger_machine(phydev);
+
+ return IRQ_HANDLED;
+}
+
+static int ns_config_intr(struct phy_device *phydev)
+{
+ int err;
+
+ if (phydev->interrupts == PHY_INTERRUPT_ENABLED) {
+ err = ns_ack_interrupt(phydev);
+ if (err)
+ return err;
+
+ err = phy_write(phydev, DP83865_INT_MASK,
+ DP83865_INT_MASK_DEFAULT);
+ } else {
+ err = phy_write(phydev, DP83865_INT_MASK, 0);
+ if (err)
+ return err;
+
+ err = ns_ack_interrupt(phydev);
+ }
+
+ return err;
+}
+
static void ns_giga_speed_fallback(struct phy_device *phydev, int mode)
{
int bmcr = phy_read(phydev, MII_BMCR);
.name = "NatSemi DP83865",
/* PHY_GBIT_FEATURES */
.config_init = ns_config_init,
- .ack_interrupt = ns_ack_interrupt,
.config_intr = ns_config_intr,
+ .handle_interrupt = ns_handle_interrupt,
} };
module_phy_driver(dp83865_driver);
#define MII_CFG2_SLEEP_REQUEST_TO_16MS 0x3
#define MII_INTSRC 21
+#define MII_INTSRC_LINK_FAIL BIT(10)
+#define MII_INTSRC_LINK_UP BIT(9)
+#define MII_INTSRC_MASK (MII_INTSRC_LINK_FAIL | MII_INTSRC_LINK_UP)
#define MII_INTSRC_TEMP_ERR BIT(1)
#define MII_INTSRC_UV_ERR BIT(3)
static int tja11xx_config_intr(struct phy_device *phydev)
{
int value = 0;
+ int err;
+
+ if (phydev->interrupts == PHY_INTERRUPT_ENABLED) {
+ err = tja11xx_ack_interrupt(phydev);
+ if (err)
+ return err;
- if (phydev->interrupts == PHY_INTERRUPT_ENABLED)
value = MII_INTEN_LINK_FAIL | MII_INTEN_LINK_UP;
+ err = phy_write(phydev, MII_INTEN, value);
+ } else {
+ err = phy_write(phydev, MII_INTEN, value);
+ if (err)
+ return err;
+
+ err = tja11xx_ack_interrupt(phydev);
+ }
+
+ return err;
+}
+
+static irqreturn_t tja11xx_handle_interrupt(struct phy_device *phydev)
+{
+ int irq_status;
+
+ irq_status = phy_read(phydev, MII_INTSRC);
+ if (irq_status < 0) {
+ phy_error(phydev);
+ return IRQ_NONE;
+ }
+
+ if (!(irq_status & MII_INTSRC_MASK))
+ return IRQ_NONE;
+
+ phy_trigger_machine(phydev);
- return phy_write(phydev, MII_INTEN, value);
+ return IRQ_HANDLED;
}
static int tja11xx_cable_test_start(struct phy_device *phydev)
.get_sset_count = tja11xx_get_sset_count,
.get_strings = tja11xx_get_strings,
.get_stats = tja11xx_get_stats,
- .ack_interrupt = tja11xx_ack_interrupt,
.config_intr = tja11xx_config_intr,
+ .handle_interrupt = tja11xx_handle_interrupt,
.cable_test_start = tja11xx_cable_test_start,
.cable_test_get_status = tja11xx_cable_test_get_status,
}, {
.get_sset_count = tja11xx_get_sset_count,
.get_strings = tja11xx_get_strings,
.get_stats = tja11xx_get_stats,
- .ack_interrupt = tja11xx_ack_interrupt,
.config_intr = tja11xx_config_intr,
+ .handle_interrupt = tja11xx_handle_interrupt,
.cable_test_start = tja11xx_cable_test_start,
.cable_test_get_status = tja11xx_cable_test_get_status,
}
#include <linux/phy.h>
/**
- * genphy_c45_setup_forced - configures a forced speed
+ * genphy_c45_pma_setup_forced - configures a forced speed
* @phydev: target phy_device struct
*/
int genphy_c45_pma_setup_forced(struct phy_device *phydev)
}
EXPORT_SYMBOL(phy_print_status);
-/**
- * phy_clear_interrupt - Ack the phy device's interrupt
- * @phydev: the phy_device struct
- *
- * If the @phydev driver has an ack_interrupt function, call it to
- * ack and clear the phy device's interrupt.
- *
- * Returns 0 on success or < 0 on error.
- */
-static int phy_clear_interrupt(struct phy_device *phydev)
-{
- if (phydev->drv->ack_interrupt)
- return phydev->drv->ack_interrupt(phydev);
-
- return 0;
-}
-
/**
* phy_config_interrupt - configure the PHY device for the requested interrupts
* @phydev: the phy_device struct
EXPORT_SYMBOL(phy_queue_state_machine);
/**
- * phy_queue_state_machine - Trigger the state machine to run now
+ * phy_trigger_machine - Trigger the state machine to run now
*
* @phydev: the phy_device struct
*/
*/
int phy_disable_interrupts(struct phy_device *phydev)
{
- int err;
-
/* Disable PHY interrupts */
- err = phy_config_interrupt(phydev, PHY_INTERRUPT_DISABLED);
- if (err)
- return err;
-
- /* Clear the interrupt */
- return phy_clear_interrupt(phydev);
+ return phy_config_interrupt(phydev, PHY_INTERRUPT_DISABLED);
}
/**
struct phy_device *phydev = phy_dat;
struct phy_driver *drv = phydev->drv;
- if (drv->handle_interrupt)
- return drv->handle_interrupt(phydev);
-
- if (drv->did_interrupt && !drv->did_interrupt(phydev))
- return IRQ_NONE;
-
- /* reschedule state queue work to run as soon as possible */
- phy_trigger_machine(phydev);
-
- /* did_interrupt() may have cleared the interrupt already */
- if (!drv->did_interrupt && phy_clear_interrupt(phydev)) {
- phy_error(phydev);
- return IRQ_NONE;
- }
-
- return IRQ_HANDLED;
+ return drv->handle_interrupt(phydev);
}
/**
*/
static int phy_enable_interrupts(struct phy_device *phydev)
{
- int err = phy_clear_interrupt(phydev);
-
- if (err < 0)
- return err;
-
return phy_config_interrupt(phydev, PHY_INTERRUPT_ENABLED);
}
}
EXPORT_SYMBOL(phy_attached_info);
-#define ATTACHED_FMT "attached PHY driver [%s] (mii_bus:phy_addr=%s, irq=%s)"
+#define ATTACHED_FMT "attached PHY driver %s(mii_bus:phy_addr=%s, irq=%s)"
char *phy_attached_info_irq(struct phy_device *phydev)
{
char *irq_str;
void phy_attached_print(struct phy_device *phydev, const char *fmt, ...)
{
- const char *drv_name = phydev->drv ? phydev->drv->name : "unbound";
+ const char *unbound = phydev->drv ? "" : "[unbound] ";
char *irq_str = phy_attached_info_irq(phydev);
if (!fmt) {
- phydev_info(phydev, ATTACHED_FMT "\n",
- drv_name, phydev_name(phydev),
- irq_str);
+ phydev_info(phydev, ATTACHED_FMT "\n", unbound,
+ phydev_name(phydev), irq_str);
} else {
va_list ap;
- phydev_info(phydev, ATTACHED_FMT,
- drv_name, phydev_name(phydev),
- irq_str);
+ phydev_info(phydev, ATTACHED_FMT, unbound,
+ phydev_name(phydev), irq_str);
va_start(ap, fmt);
vprintk(fmt, ap);
#endif
/**
- * phy_get_delay_index - returns the index of the internal delay
+ * phy_get_internal_delay - returns the index of the internal delay
* @phydev: phy_device struct
* @dev: pointer to the devices device struct
* @delay_values: array of delays the PHY supports
static bool phy_drv_supports_irq(struct phy_driver *phydrv)
{
- return phydrv->config_intr && (phydrv->ack_interrupt || phydrv->handle_interrupt);
+ return phydrv->config_intr && phydrv->handle_interrupt;
}
/**
EXPORT_SYMBOL_GPL(phylink_ethtool_set_pauseparam);
/**
- * phylink_ethtool_get_eee_err() - read the energy efficient ethernet error
+ * phylink_get_eee_err() - read the energy efficient ethernet error
* counter
* @pl: a pointer to a &struct phylink returned from phylink_create().
*
{
int err;
+ /* The Interrupt Source register is not self-clearing, bits 4 and 5 are
+ * cleared when MII_BMSR is read and bits 1 and 3 are cleared when
+ * MII_EXPANSION is read
+ */
err = phy_read(phydev, MII_QS6612_ISR);
if (err < 0)
static int qs6612_config_intr(struct phy_device *phydev)
{
int err;
- if (phydev->interrupts == PHY_INTERRUPT_ENABLED)
+ if (phydev->interrupts == PHY_INTERRUPT_ENABLED) {
+ /* clear any interrupts before enabling them */
+ err = qs6612_ack_interrupt(phydev);
+ if (err)
+ return err;
+
err = phy_write(phydev, MII_QS6612_IMR,
MII_QS6612_IMR_INIT);
- else
+ } else {
err = phy_write(phydev, MII_QS6612_IMR, 0);
+ if (err)
+ return err;
+
+ /* clear any leftover interrupts */
+ err = qs6612_ack_interrupt(phydev);
+ }
return err;
}
+static irqreturn_t qs6612_handle_interrupt(struct phy_device *phydev)
+{
+ int irq_status;
+
+ irq_status = phy_read(phydev, MII_QS6612_ISR);
+ if (irq_status < 0) {
+ phy_error(phydev);
+ return IRQ_NONE;
+ }
+
+ if (!(irq_status & MII_QS6612_IMR_INIT))
+ return IRQ_NONE;
+
+ /* the interrupt source register is not self-clearing */
+ qs6612_ack_interrupt(phydev);
+
+ phy_trigger_machine(phydev);
+
+ return IRQ_HANDLED;
+}
+
static struct phy_driver qs6612_driver[] = { {
.phy_id = 0x00181440,
.name = "QS6612",
.phy_id_mask = 0xfffffff0,
/* PHY_BASIC_FEATURES */
.config_init = qs6612_config_init,
- .ack_interrupt = qs6612_ack_interrupt,
.config_intr = qs6612_config_intr,
+ .handle_interrupt = qs6612_handle_interrupt,
} };
module_phy_driver(qs6612_driver);
PHY_ID_MATCH_EXACT(0x001cc916),
.name = "RTL8211F Gigabit Ethernet",
.config_init = &rtl8211f_config_init,
+ .read_status = rtlgen_read_status,
.config_intr = &rtl8211f_config_intr,
.handle_interrupt = rtl8211f_handle_interrupt,
.suspend = genphy_suspend,
struct clk *refclk;
};
+static int smsc_phy_ack_interrupt(struct phy_device *phydev)
+{
+ int rc = phy_read(phydev, MII_LAN83C185_ISF);
+
+ return rc < 0 ? rc : 0;
+}
+
static int smsc_phy_config_intr(struct phy_device *phydev)
{
struct smsc_phy_priv *priv = phydev->priv;
int rc;
if (phydev->interrupts == PHY_INTERRUPT_ENABLED) {
+ rc = smsc_phy_ack_interrupt(phydev);
+ if (rc)
+ return rc;
+
intmask = MII_LAN83C185_ISF_INT4 | MII_LAN83C185_ISF_INT6;
if (priv->energy_enable)
intmask |= MII_LAN83C185_ISF_INT7;
- }
+ rc = phy_write(phydev, MII_LAN83C185_IM, intmask);
+ } else {
+ rc = phy_write(phydev, MII_LAN83C185_IM, intmask);
+ if (rc)
+ return rc;
- rc = phy_write(phydev, MII_LAN83C185_IM, intmask);
+ rc = smsc_phy_ack_interrupt(phydev);
+ }
return rc < 0 ? rc : 0;
}
-static int smsc_phy_ack_interrupt(struct phy_device *phydev)
+static irqreturn_t smsc_phy_handle_interrupt(struct phy_device *phydev)
{
- int rc = phy_read (phydev, MII_LAN83C185_ISF);
+ int irq_status, irq_enabled;
- return rc < 0 ? rc : 0;
+ irq_enabled = phy_read(phydev, MII_LAN83C185_IM);
+ if (irq_enabled < 0) {
+ phy_error(phydev);
+ return IRQ_NONE;
+ }
+
+ irq_status = phy_read(phydev, MII_LAN83C185_ISF);
+ if (irq_status < 0) {
+ phy_error(phydev);
+ return IRQ_NONE;
+ }
+
+ if (!(irq_status & irq_enabled))
+ return IRQ_NONE;
+
+ phy_trigger_machine(phydev);
+
+ return IRQ_HANDLED;
}
static int smsc_phy_config_init(struct phy_device *phydev)
return ret;
ret = clk_set_rate(priv->refclk, 50 * 1000 * 1000);
- if (ret)
+ if (ret) {
+ clk_disable_unprepare(priv->refclk);
return ret;
+ }
return 0;
}
.soft_reset = smsc_phy_reset,
/* IRQ related */
- .ack_interrupt = smsc_phy_ack_interrupt,
.config_intr = smsc_phy_config_intr,
+ .handle_interrupt = smsc_phy_handle_interrupt,
.suspend = genphy_suspend,
.resume = genphy_resume,
.soft_reset = smsc_phy_reset,
/* IRQ related */
- .ack_interrupt = smsc_phy_ack_interrupt,
.config_intr = smsc_phy_config_intr,
+ .handle_interrupt = smsc_phy_handle_interrupt,
/* Statistics */
.get_sset_count = smsc_get_sset_count,
.config_aneg = lan87xx_config_aneg,
/* IRQ related */
- .ack_interrupt = smsc_phy_ack_interrupt,
.config_intr = smsc_phy_config_intr,
+ .handle_interrupt = smsc_phy_handle_interrupt,
/* Statistics */
.get_sset_count = smsc_get_sset_count,
.config_init = lan911x_config_init,
/* IRQ related */
- .ack_interrupt = smsc_phy_ack_interrupt,
.config_intr = smsc_phy_config_intr,
+ .handle_interrupt = smsc_phy_handle_interrupt,
.suspend = genphy_suspend,
.resume = genphy_resume,
.config_aneg = lan87xx_config_aneg_ext,
/* IRQ related */
- .ack_interrupt = smsc_phy_ack_interrupt,
.config_intr = smsc_phy_config_intr,
+ .handle_interrupt = smsc_phy_handle_interrupt,
/* Statistics */
.get_sset_count = smsc_get_sset_count,
.soft_reset = smsc_phy_reset,
/* IRQ related */
- .ack_interrupt = smsc_phy_ack_interrupt,
.config_intr = smsc_phy_config_intr,
+ .handle_interrupt = smsc_phy_handle_interrupt,
/* Statistics */
.get_sset_count = smsc_get_sset_count,
return 0;
}
+static int ste10Xp_ack_interrupt(struct phy_device *phydev)
+{
+ int err = phy_read(phydev, MII_XCIIS);
+
+ if (err < 0)
+ return err;
+
+ return 0;
+}
+
static int ste10Xp_config_intr(struct phy_device *phydev)
{
- int err, value;
+ int err;
if (phydev->interrupts == PHY_INTERRUPT_ENABLED) {
+ /* clear any pending interrupts */
+ err = ste10Xp_ack_interrupt(phydev);
+ if (err)
+ return err;
+
/* Enable all STe101P interrupts (PR12) */
err = phy_write(phydev, MII_XIE, MII_XIE_DEFAULT_MASK);
- /* clear any pending interrupts */
- if (err == 0) {
- value = phy_read(phydev, MII_XCIIS);
- if (value < 0)
- err = value;
- }
- } else
+ } else {
err = phy_write(phydev, MII_XIE, 0);
+ if (err)
+ return err;
+
+ err = ste10Xp_ack_interrupt(phydev);
+ }
return err;
}
-static int ste10Xp_ack_interrupt(struct phy_device *phydev)
+static irqreturn_t ste10Xp_handle_interrupt(struct phy_device *phydev)
{
- int err = phy_read(phydev, MII_XCIIS);
- if (err < 0)
- return err;
+ int irq_status;
- return 0;
+ irq_status = phy_read(phydev, MII_XCIIS);
+ if (irq_status < 0) {
+ phy_error(phydev);
+ return IRQ_NONE;
+ }
+
+ if (!(irq_status & MII_XIE_DEFAULT_MASK))
+ return IRQ_NONE;
+
+ phy_trigger_machine(phydev);
+
+ return IRQ_HANDLED;
}
static struct phy_driver ste10xp_pdriver[] = {
.name = "STe101p",
/* PHY_BASIC_FEATURES */
.config_init = ste10Xp_config_init,
- .ack_interrupt = ste10Xp_ack_interrupt,
.config_intr = ste10Xp_config_intr,
+ .handle_interrupt = ste10Xp_handle_interrupt,
.suspend = genphy_suspend,
.resume = genphy_resume,
}, {
.name = "STe100p",
/* PHY_BASIC_FEATURES */
.config_init = ste10Xp_config_init,
- .ack_interrupt = ste10Xp_ack_interrupt,
.config_intr = ste10Xp_config_intr,
+ .handle_interrupt = ste10Xp_handle_interrupt,
.suspend = genphy_suspend,
.resume = genphy_resume,
} };
#define MII_VSC8244_ISTAT_SPEED 0x4000
#define MII_VSC8244_ISTAT_LINK 0x2000
#define MII_VSC8244_ISTAT_DUPLEX 0x1000
+#define MII_VSC8244_ISTAT_MASK (MII_VSC8244_ISTAT_SPEED | \
+ MII_VSC8244_ISTAT_LINK | \
+ MII_VSC8244_ISTAT_DUPLEX)
+
+#define MII_VSC8221_ISTAT_MASK MII_VSC8244_ISTAT_LINK
/* Vitesse Auxiliary Control/Status Register */
#define MII_VSC8244_AUX_CONSTAT 0x1c
return 0;
}
-static int vsc824x_ack_interrupt(struct phy_device *phydev)
-{
- int err = 0;
-
- /* Don't bother to ACK the interrupts if interrupts
- * are disabled. The 824x cannot clear the interrupts
- * if they are disabled.
- */
- if (phydev->interrupts == PHY_INTERRUPT_ENABLED)
- err = phy_read(phydev, MII_VSC8244_ISTAT);
-
- return (err < 0) ? err : 0;
-}
-
static int vsc82xx_config_intr(struct phy_device *phydev)
{
int err;
if (phydev->interrupts == PHY_INTERRUPT_ENABLED)
+ /* Don't bother to ACK the interrupts since the 824x cannot
+ * clear the interrupts if they are disabled.
+ */
err = phy_write(phydev, MII_VSC8244_IMASK,
(phydev->drv->phy_id == PHY_ID_VSC8234 ||
phydev->drv->phy_id == PHY_ID_VSC8244 ||
return err;
}
+static irqreturn_t vsc82xx_handle_interrupt(struct phy_device *phydev)
+{
+ int irq_status, irq_mask;
+
+ if (phydev->drv->phy_id == PHY_ID_VSC8244 ||
+ phydev->drv->phy_id == PHY_ID_VSC8572 ||
+ phydev->drv->phy_id == PHY_ID_VSC8601)
+ irq_mask = MII_VSC8244_ISTAT_MASK;
+ else
+ irq_mask = MII_VSC8221_ISTAT_MASK;
+
+ irq_status = phy_read(phydev, MII_VSC8244_ISTAT);
+ if (irq_status < 0) {
+ phy_error(phydev);
+ return IRQ_NONE;
+ }
+
+ if (!(irq_status & irq_mask))
+ return IRQ_NONE;
+
+ phy_trigger_machine(phydev);
+
+ return IRQ_HANDLED;
+}
+
static int vsc8221_config_init(struct phy_device *phydev)
{
int err;
/* PHY_GBIT_FEATURES */
.config_init = &vsc824x_config_init,
.config_aneg = &vsc82x4_config_aneg,
- .ack_interrupt = &vsc824x_ack_interrupt,
.config_intr = &vsc82xx_config_intr,
+ .handle_interrupt = &vsc82xx_handle_interrupt,
}, {
.phy_id = PHY_ID_VSC8244,
.name = "Vitesse VSC8244",
/* PHY_GBIT_FEATURES */
.config_init = &vsc824x_config_init,
.config_aneg = &vsc82x4_config_aneg,
- .ack_interrupt = &vsc824x_ack_interrupt,
.config_intr = &vsc82xx_config_intr,
+ .handle_interrupt = &vsc82xx_handle_interrupt,
}, {
.phy_id = PHY_ID_VSC8572,
.name = "Vitesse VSC8572",
/* PHY_GBIT_FEATURES */
.config_init = &vsc824x_config_init,
.config_aneg = &vsc82x4_config_aneg,
- .ack_interrupt = &vsc824x_ack_interrupt,
.config_intr = &vsc82xx_config_intr,
+ .handle_interrupt = &vsc82xx_handle_interrupt,
}, {
.phy_id = PHY_ID_VSC8601,
.name = "Vitesse VSC8601",
.phy_id_mask = 0x000ffff0,
/* PHY_GBIT_FEATURES */
.config_init = &vsc8601_config_init,
- .ack_interrupt = &vsc824x_ack_interrupt,
.config_intr = &vsc82xx_config_intr,
+ .handle_interrupt = &vsc82xx_handle_interrupt,
}, {
.phy_id = PHY_ID_VSC7385,
.name = "Vitesse VSC7385",
/* PHY_GBIT_FEATURES */
.config_init = &vsc824x_config_init,
.config_aneg = &vsc82x4_config_aneg,
- .ack_interrupt = &vsc824x_ack_interrupt,
.config_intr = &vsc82xx_config_intr,
+ .handle_interrupt = &vsc82xx_handle_interrupt,
}, {
/* Vitesse 8221 */
.phy_id = PHY_ID_VSC8221,
.name = "Vitesse VSC8221",
/* PHY_GBIT_FEATURES */
.config_init = &vsc8221_config_init,
- .ack_interrupt = &vsc824x_ack_interrupt,
.config_intr = &vsc82xx_config_intr,
+ .handle_interrupt = &vsc82xx_handle_interrupt,
}, {
/* Vitesse 8211 */
.phy_id = PHY_ID_VSC8211,
.name = "Vitesse VSC8211",
/* PHY_GBIT_FEATURES */
.config_init = &vsc8221_config_init,
- .ack_interrupt = &vsc824x_ack_interrupt,
.config_intr = &vsc82xx_config_intr,
+ .handle_interrupt = &vsc82xx_handle_interrupt,
} };
module_phy_driver(vsc82xx_driver);
* Copyright (c) 2011 Jiri Pirko <jpirko@redhat.com>
*/
+#include <linux/ethtool.h>
#include <linux/kernel.h>
#include <linux/types.h>
#include <linux/module.h>
struct tun_file *tfile = file->private_data;
struct tun_struct *tun = tun_get(tfile);
ssize_t result;
+ int noblock = 0;
if (!tun)
return -EBADFD;
- result = tun_get_user(tun, tfile, NULL, from,
- file->f_flags & O_NONBLOCK, false);
+ if ((file->f_flags & O_NONBLOCK) || (iocb->ki_flags & IOCB_NOWAIT))
+ noblock = 1;
+
+ result = tun_get_user(tun, tfile, NULL, from, noblock, false);
tun_put(tun);
return result;
struct tun_file *tfile = file->private_data;
struct tun_struct *tun = tun_get(tfile);
ssize_t len = iov_iter_count(to), ret;
+ int noblock = 0;
if (!tun)
return -EBADFD;
- ret = tun_do_read(tun, tfile, to, file->f_flags & O_NONBLOCK, NULL);
+
+ if ((file->f_flags & O_NONBLOCK) || (iocb->ki_flags & IOCB_NOWAIT))
+ noblock = 1;
+
+ ret = tun_do_read(tun, tfile, to, noblock, NULL);
ret = min_t(ssize_t, ret, len);
if (ret > 0)
iocb->ki_pos = ret;
"Linktype set failed because interface is up\n");
ret = -EBUSY;
} else {
+ ret = call_netdevice_notifiers(NETDEV_PRE_TYPE_CHANGE,
+ tun->dev);
+ ret = notifier_to_errno(ret);
+ if (ret) {
+ netif_info(tun, drv, tun->dev,
+ "Refused to change device type\n");
+ break;
+ }
tun->dev->type = (int) arg;
netif_info(tun, drv, tun->dev, "linktype set to %d\n",
tun->dev->type);
- ret = 0;
+ call_netdevice_notifiers(NETDEV_POST_TYPE_CHANGE,
+ tun->dev);
}
break;
This driver should work with at least the following devices:
* Aquantia AQtion USB to 5GbE
+config USB_RTL8153_ECM
+ tristate "RTL8153 ECM support"
+ depends on USB_NET_CDCETHER && (USB_RTL8152 || USB_RTL8152=n)
+ default y
+ help
+ This option supports ECM mode for RTL8153 ethernet adapter, when
+ CONFIG_USB_RTL8152 is not set, or the RTL8153 device is not
+ supported by r8152 driver.
+
endif # USB_NET_DRIVERS
obj-$(CONFIG_USB_NET_AX8817X) += asix.o
asix-y := asix_devices.o asix_common.o ax88172a.o
obj-$(CONFIG_USB_NET_AX88179_178A) += ax88179_178a.o
-obj-$(CONFIG_USB_NET_CDCETHER) += cdc_ether.o r8153_ecm.o
+obj-$(CONFIG_USB_NET_CDCETHER) += cdc_ether.o
obj-$(CONFIG_USB_NET_CDC_EEM) += cdc_eem.o
obj-$(CONFIG_USB_NET_DM9601) += dm9601.o
obj-$(CONFIG_USB_NET_SR9700) += sr9700.o
obj-$(CONFIG_USB_NET_CDC_MBIM) += cdc_mbim.o
obj-$(CONFIG_USB_NET_CH9200) += ch9200.o
obj-$(CONFIG_USB_NET_AQC111) += aqc111.o
+obj-$(CONFIG_USB_RTL8153_ECM) += r8153_ecm.o
}
/* enable ethernet mode (?) */
- if (cx82310_enable_ethernet(dev))
+ ret = cx82310_enable_ethernet(dev);
+ if (ret)
goto err;
/* get the MAC address */
#define IPHETH_USBINTF_SUBCLASS 253
#define IPHETH_USBINTF_PROTO 1
-#define IPHETH_BUF_SIZE 1516
+#define IPHETH_BUF_SIZE 1514
#define IPHETH_IP_ALIGN 2 /* padding at front of URB */
#define IPHETH_TX_TIMEOUT (5 * HZ)
{QMI_FIXED_INTF(0x05c6, 0x9011, 4)},
{QMI_FIXED_INTF(0x05c6, 0x9021, 1)},
{QMI_FIXED_INTF(0x05c6, 0x9022, 2)},
- {QMI_FIXED_INTF(0x05c6, 0x9025, 4)}, /* Alcatel-sbell ASB TL131 TDD LTE (China Mobile) */
+ {QMI_QUIRK_SET_DTR(0x05c6, 0x9025, 4)}, /* Alcatel-sbell ASB TL131 TDD LTE (China Mobile) */
{QMI_FIXED_INTF(0x05c6, 0x9026, 3)},
{QMI_FIXED_INTF(0x05c6, 0x902e, 5)},
{QMI_FIXED_INTF(0x05c6, 0x9031, 5)},
}
if (ifmp && tbp[IFLA_IFNAME]) {
- nla_strlcpy(ifname, tbp[IFLA_IFNAME], IFNAMSIZ);
+ nla_strscpy(ifname, tbp[IFLA_IFNAME], IFNAMSIZ);
name_assign_type = NET_NAME_USER;
} else {
snprintf(ifname, IFNAMSIZ, DRV_NAME "%%d");
eth_hw_addr_random(dev);
if (tb[IFLA_IFNAME])
- nla_strlcpy(dev->name, tb[IFLA_IFNAME], IFNAMSIZ);
+ nla_strscpy(dev->name, tb[IFLA_IFNAME], IFNAMSIZ);
else
snprintf(dev->name, IFNAMSIZ, DRV_NAME "%%d");
* Based on dummy, team and ipvlan drivers
*/
+#include <linux/ethtool.h>
#include <linux/module.h>
#include <linux/kernel.h>
#include <linux/netdevice.h>
// SPDX-License-Identifier: GPL-2.0-only
+#include <linux/ethtool.h>
#include <linux/module.h>
#include <linux/kernel.h>
#include <linux/if_arp.h>
dev->gso_max_segs = lowerdev->gso_max_segs;
needed_headroom = lowerdev->hard_header_len;
+ needed_headroom += lowerdev->needed_headroom;
+
+ dev->needed_tailroom = lowerdev->needed_tailroom;
max_mtu = lowerdev->mtu - (use_ipv6 ? VXLAN6_HEADROOM :
VXLAN_HEADROOM);
if (dst->remote_ifindex) {
remote_dev = __dev_get_by_index(net, dst->remote_ifindex);
- if (!remote_dev)
+ if (!remote_dev) {
+ err = -ENODEV;
goto errout;
+ }
err = netdev_upper_dev_link(remote_dev, dev, extack);
if (err)
Say Y here if you want to use built-in HSS ports
on IXP4xx processor.
-config DLCI
- tristate "Frame Relay DLCI support"
- help
- Support for the Frame Relay protocol.
-
- Frame Relay is a fast low-cost way to connect to a remote Internet
- access provider or to form a private wide area network. The one
- physical line from your box to the local "switch" (i.e. the entry
- point to the Frame Relay network, usually at the phone company) can
- carry several logical point-to-point connections to other computers
- connected to the Frame Relay network. For a general explanation of
- the protocol, check out <http://www.mplsforum.org/>.
-
- To use frame relay, you need supporting hardware (called FRAD) and
- certain programs from the net-tools package as explained in
- <file:Documentation/networking/framerelay.rst>.
-
- To compile this driver as a module, choose M here: the
- module will be called dlci.
-
-config DLCI_MAX
- int "Max DLCI per device"
- depends on DLCI
- default "8"
- help
- How many logical point-to-point frame relay connections (the
- identifiers of which are called DCLIs) should be handled by each
- of your hardware frame relay access devices.
-
- Go with the default.
-
-config SDLA
- tristate "SDLA (Sangoma S502/S508) support"
- depends on DLCI && ISA
- help
- Driver for the Sangoma S502A, S502E, and S508 Frame Relay Access
- Devices.
-
- These are multi-protocol cards, but only Frame Relay is supported
- by the driver at this time. Please read
- <file:Documentation/networking/framerelay.rst>.
-
- To compile this driver as a module, choose M here: the
- module will be called sdla.
-
# X.25 network drivers
config LAPBETHER
tristate "LAPB over Ethernet driver"
obj-$(CONFIG_LANMEDIA) += lmc/
-obj-$(CONFIG_DLCI) += dlci.o
-obj-$(CONFIG_SDLA) += sdla.o
obj-$(CONFIG_LAPBETHER) += lapbether.o
obj-$(CONFIG_SBNI) += sbni.o
obj-$(CONFIG_N2) += n2.o
+++ /dev/null
-// SPDX-License-Identifier: GPL-2.0-or-later
-/*
- * DLCI Implementation of Frame Relay protocol for Linux, according to
- * RFC 1490. This generic device provides en/decapsulation for an
- * underlying hardware driver. Routes & IPs are assigned to these
- * interfaces. Requires 'dlcicfg' program to create usable
- * interfaces, the initial one, 'dlci' is for IOCTL use only.
- *
- * Version: @(#)dlci.c 0.35 4 Jan 1997
- *
- * Author: Mike McLagan <mike.mclagan@linux.org>
- *
- * Changes:
- *
- * 0.15 Mike Mclagan Packet freeing, bug in kmalloc call
- * DLCI_RET handling
- * 0.20 Mike McLagan More conservative on which packets
- * are returned for retry and which are
- * are dropped. If DLCI_RET_DROP is
- * returned from the FRAD, the packet is
- * sent back to Linux for re-transmission
- * 0.25 Mike McLagan Converted to use SIOC IOCTL calls
- * 0.30 Jim Freeman Fixed to allow IPX traffic
- * 0.35 Michael Elizabeth Fixed incorrect memcpy_fromfs
- */
-
-#define pr_fmt(fmt) KBUILD_MODNAME ": " fmt
-
-#include <linux/module.h>
-#include <linux/kernel.h>
-#include <linux/types.h>
-#include <linux/fcntl.h>
-#include <linux/interrupt.h>
-#include <linux/ptrace.h>
-#include <linux/ioport.h>
-#include <linux/in.h>
-#include <linux/init.h>
-#include <linux/slab.h>
-#include <linux/string.h>
-#include <linux/errno.h>
-#include <linux/netdevice.h>
-#include <linux/skbuff.h>
-#include <linux/if_arp.h>
-#include <linux/if_frad.h>
-#include <linux/bitops.h>
-
-#include <net/sock.h>
-
-#include <asm/io.h>
-#include <asm/dma.h>
-#include <linux/uaccess.h>
-
-static const char version[] = "DLCI driver v0.35, 4 Jan 1997, mike.mclagan@linux.org";
-
-static LIST_HEAD(dlci_devs);
-
-static void dlci_setup(struct net_device *);
-
-/*
- * these encapsulate the RFC 1490 requirements as well as
- * deal with packet transmission and reception, working with
- * the upper network layers
- */
-
-static int dlci_header(struct sk_buff *skb, struct net_device *dev,
- unsigned short type, const void *daddr,
- const void *saddr, unsigned len)
-{
- struct frhdr hdr;
- unsigned int hlen;
- char *dest;
-
- hdr.control = FRAD_I_UI;
- switch (type)
- {
- case ETH_P_IP:
- hdr.IP_NLPID = FRAD_P_IP;
- hlen = sizeof(hdr.control) + sizeof(hdr.IP_NLPID);
- break;
-
- /* feel free to add other types, if necessary */
-
- default:
- hdr.pad = FRAD_P_PADDING;
- hdr.NLPID = FRAD_P_SNAP;
- memset(hdr.OUI, 0, sizeof(hdr.OUI));
- hdr.PID = htons(type);
- hlen = sizeof(hdr);
- break;
- }
-
- dest = skb_push(skb, hlen);
- if (!dest)
- return 0;
-
- memcpy(dest, &hdr, hlen);
-
- return hlen;
-}
-
-static void dlci_receive(struct sk_buff *skb, struct net_device *dev)
-{
- struct frhdr *hdr;
- int process, header;
-
- if (!pskb_may_pull(skb, sizeof(*hdr))) {
- netdev_notice(dev, "invalid data no header\n");
- dev->stats.rx_errors++;
- kfree_skb(skb);
- return;
- }
-
- hdr = (struct frhdr *) skb->data;
- process = 0;
- header = 0;
- skb->dev = dev;
-
- if (hdr->control != FRAD_I_UI)
- {
- netdev_notice(dev, "Invalid header flag 0x%02X\n",
- hdr->control);
- dev->stats.rx_errors++;
- }
- else
- switch (hdr->IP_NLPID)
- {
- case FRAD_P_PADDING:
- if (hdr->NLPID != FRAD_P_SNAP)
- {
- netdev_notice(dev, "Unsupported NLPID 0x%02X\n",
- hdr->NLPID);
- dev->stats.rx_errors++;
- break;
- }
-
- if (hdr->OUI[0] + hdr->OUI[1] + hdr->OUI[2] != 0)
- {
- netdev_notice(dev, "Unsupported organizationally unique identifier 0x%02X-%02X-%02X\n",
- hdr->OUI[0],
- hdr->OUI[1],
- hdr->OUI[2]);
- dev->stats.rx_errors++;
- break;
- }
-
- /* at this point, it's an EtherType frame */
- header = sizeof(struct frhdr);
- /* Already in network order ! */
- skb->protocol = hdr->PID;
- process = 1;
- break;
-
- case FRAD_P_IP:
- header = sizeof(hdr->control) + sizeof(hdr->IP_NLPID);
- skb->protocol = htons(ETH_P_IP);
- process = 1;
- break;
-
- case FRAD_P_SNAP:
- case FRAD_P_Q933:
- case FRAD_P_CLNP:
- netdev_notice(dev, "Unsupported NLPID 0x%02X\n",
- hdr->pad);
- dev->stats.rx_errors++;
- break;
-
- default:
- netdev_notice(dev, "Invalid pad byte 0x%02X\n",
- hdr->pad);
- dev->stats.rx_errors++;
- break;
- }
-
- if (process)
- {
- /* we've set up the protocol, so discard the header */
- skb_reset_mac_header(skb);
- skb_pull(skb, header);
- dev->stats.rx_bytes += skb->len;
- netif_rx(skb);
- dev->stats.rx_packets++;
- }
- else
- dev_kfree_skb(skb);
-}
-
-static netdev_tx_t dlci_transmit(struct sk_buff *skb, struct net_device *dev)
-{
- struct dlci_local *dlp = netdev_priv(dev);
-
- if (skb) {
- struct netdev_queue *txq = skb_get_tx_queue(dev, skb);
- netdev_start_xmit(skb, dlp->slave, txq, false);
- }
- return NETDEV_TX_OK;
-}
-
-static int dlci_config(struct net_device *dev, struct dlci_conf __user *conf, int get)
-{
- struct dlci_conf config;
- struct dlci_local *dlp;
- struct frad_local *flp;
- int err;
-
- dlp = netdev_priv(dev);
-
- flp = netdev_priv(dlp->slave);
-
- if (!get)
- {
- if (copy_from_user(&config, conf, sizeof(struct dlci_conf)))
- return -EFAULT;
- if (config.flags & ~DLCI_VALID_FLAGS)
- return -EINVAL;
- memcpy(&dlp->config, &config, sizeof(struct dlci_conf));
- dlp->configured = 1;
- }
-
- err = (*flp->dlci_conf)(dlp->slave, dev, get);
- if (err)
- return err;
-
- if (get)
- {
- if (copy_to_user(conf, &dlp->config, sizeof(struct dlci_conf)))
- return -EFAULT;
- }
-
- return 0;
-}
-
-static int dlci_dev_ioctl(struct net_device *dev, struct ifreq *ifr, int cmd)
-{
- struct dlci_local *dlp;
-
- if (!capable(CAP_NET_ADMIN))
- return -EPERM;
-
- dlp = netdev_priv(dev);
-
- switch (cmd)
- {
- case DLCI_GET_SLAVE:
- if (!*(short *)(dev->dev_addr))
- return -EINVAL;
-
- strncpy(ifr->ifr_slave, dlp->slave->name, sizeof(ifr->ifr_slave));
- break;
-
- case DLCI_GET_CONF:
- case DLCI_SET_CONF:
- if (!*(short *)(dev->dev_addr))
- return -EINVAL;
-
- return dlci_config(dev, ifr->ifr_data, cmd == DLCI_GET_CONF);
-
- default:
- return -EOPNOTSUPP;
- }
- return 0;
-}
-
-static int dlci_change_mtu(struct net_device *dev, int new_mtu)
-{
- struct dlci_local *dlp = netdev_priv(dev);
-
- return dev_set_mtu(dlp->slave, new_mtu);
-}
-
-static int dlci_open(struct net_device *dev)
-{
- struct dlci_local *dlp;
- struct frad_local *flp;
- int err;
-
- dlp = netdev_priv(dev);
-
- if (!*(short *)(dev->dev_addr))
- return -EINVAL;
-
- if (!netif_running(dlp->slave))
- return -ENOTCONN;
-
- flp = netdev_priv(dlp->slave);
- err = (*flp->activate)(dlp->slave, dev);
- if (err)
- return err;
-
- netif_start_queue(dev);
-
- return 0;
-}
-
-static int dlci_close(struct net_device *dev)
-{
- struct dlci_local *dlp;
- struct frad_local *flp;
-
- netif_stop_queue(dev);
-
- dlp = netdev_priv(dev);
-
- flp = netdev_priv(dlp->slave);
- (*flp->deactivate)(dlp->slave, dev);
-
- return 0;
-}
-
-static int dlci_add(struct dlci_add *dlci)
-{
- struct net_device *master, *slave;
- struct dlci_local *dlp;
- struct frad_local *flp;
- int err = -EINVAL;
-
-
- /* validate slave device */
- slave = dev_get_by_name(&init_net, dlci->devname);
- if (!slave)
- return -ENODEV;
-
- if (slave->type != ARPHRD_FRAD || netdev_priv(slave) == NULL)
- goto err1;
-
- /* create device name */
- master = alloc_netdev(sizeof(struct dlci_local), "dlci%d",
- NET_NAME_UNKNOWN, dlci_setup);
- if (!master) {
- err = -ENOMEM;
- goto err1;
- }
-
- /* make sure same slave not already registered */
- rtnl_lock();
- list_for_each_entry(dlp, &dlci_devs, list) {
- if (dlp->slave == slave) {
- err = -EBUSY;
- goto err2;
- }
- }
-
- *(short *)(master->dev_addr) = dlci->dlci;
-
- dlp = netdev_priv(master);
- dlp->slave = slave;
- dlp->master = master;
-
- flp = netdev_priv(slave);
- err = (*flp->assoc)(slave, master);
- if (err < 0)
- goto err2;
-
- err = register_netdevice(master);
- if (err < 0)
- goto err2;
-
- strcpy(dlci->devname, master->name);
-
- list_add(&dlp->list, &dlci_devs);
- rtnl_unlock();
-
- return 0;
-
- err2:
- rtnl_unlock();
- free_netdev(master);
- err1:
- dev_put(slave);
- return err;
-}
-
-static int dlci_del(struct dlci_add *dlci)
-{
- struct dlci_local *dlp;
- struct frad_local *flp;
- struct net_device *master, *slave;
- int err;
- bool found = false;
-
- rtnl_lock();
-
- /* validate slave device */
- master = __dev_get_by_name(&init_net, dlci->devname);
- if (!master) {
- err = -ENODEV;
- goto out;
- }
-
- list_for_each_entry(dlp, &dlci_devs, list) {
- if (dlp->master == master) {
- found = true;
- break;
- }
- }
- if (!found) {
- err = -ENODEV;
- goto out;
- }
-
- if (netif_running(master)) {
- err = -EBUSY;
- goto out;
- }
-
- dlp = netdev_priv(master);
- slave = dlp->slave;
- flp = netdev_priv(slave);
-
- err = (*flp->deassoc)(slave, master);
- if (!err) {
- list_del(&dlp->list);
-
- unregister_netdevice(master);
-
- dev_put(slave);
- }
-out:
- rtnl_unlock();
- return err;
-}
-
-static int dlci_ioctl(unsigned int cmd, void __user *arg)
-{
- struct dlci_add add;
- int err;
-
- if (!capable(CAP_NET_ADMIN))
- return -EPERM;
-
- if (copy_from_user(&add, arg, sizeof(struct dlci_add)))
- return -EFAULT;
-
- switch (cmd)
- {
- case SIOCADDDLCI:
- err = dlci_add(&add);
-
- if (!err)
- if (copy_to_user(arg, &add, sizeof(struct dlci_add)))
- return -EFAULT;
- break;
-
- case SIOCDELDLCI:
- err = dlci_del(&add);
- break;
-
- default:
- err = -EINVAL;
- }
-
- return err;
-}
-
-static const struct header_ops dlci_header_ops = {
- .create = dlci_header,
-};
-
-static const struct net_device_ops dlci_netdev_ops = {
- .ndo_open = dlci_open,
- .ndo_stop = dlci_close,
- .ndo_do_ioctl = dlci_dev_ioctl,
- .ndo_start_xmit = dlci_transmit,
- .ndo_change_mtu = dlci_change_mtu,
-};
-
-static void dlci_setup(struct net_device *dev)
-{
- struct dlci_local *dlp = netdev_priv(dev);
-
- dev->flags = 0;
- dev->header_ops = &dlci_header_ops;
- dev->netdev_ops = &dlci_netdev_ops;
- dev->needs_free_netdev = true;
-
- dlp->receive = dlci_receive;
-
- dev->type = ARPHRD_DLCI;
- dev->hard_header_len = sizeof(struct frhdr);
- dev->addr_len = sizeof(short);
-
-}
-
-/* if slave is unregistering, then cleanup master */
-static int dlci_dev_event(struct notifier_block *unused,
- unsigned long event, void *ptr)
-{
- struct net_device *dev = netdev_notifier_info_to_dev(ptr);
-
- if (dev_net(dev) != &init_net)
- return NOTIFY_DONE;
-
- if (event == NETDEV_UNREGISTER) {
- struct dlci_local *dlp;
-
- list_for_each_entry(dlp, &dlci_devs, list) {
- if (dlp->slave == dev) {
- list_del(&dlp->list);
- unregister_netdevice(dlp->master);
- dev_put(dlp->slave);
- break;
- }
- }
- }
- return NOTIFY_DONE;
-}
-
-static struct notifier_block dlci_notifier = {
- .notifier_call = dlci_dev_event,
-};
-
-static int __init init_dlci(void)
-{
- dlci_ioctl_set(dlci_ioctl);
- register_netdevice_notifier(&dlci_notifier);
-
- printk("%s.\n", version);
-
- return 0;
-}
-
-static void __exit dlci_exit(void)
-{
- struct dlci_local *dlp, *nxt;
-
- dlci_ioctl_set(NULL);
- unregister_netdevice_notifier(&dlci_notifier);
-
- rtnl_lock();
- list_for_each_entry_safe(dlp, nxt, &dlci_devs, list) {
- unregister_netdevice(dlp->master);
- dev_put(dlp->slave);
- }
- rtnl_unlock();
-}
-
-module_init(init_dlci);
-module_exit(dlci_exit);
-
-MODULE_AUTHOR("Mike McLagan");
-MODULE_DESCRIPTION("Frame Relay DLCI layer");
-MODULE_LICENSE("GPL");
#define get_port(card, port) (&card->ports[port])
-#define sca_flush(card) (sca_in(IER0, card));
+#define sca_flush(card) (sca_in(IER0, card))
static inline void new_memcpy_toio(char __iomem *dest, char *src, int length)
{
+++ /dev/null
-// SPDX-License-Identifier: GPL-2.0-or-later
-/*
- * SDLA An implementation of a driver for the Sangoma S502/S508 series
- * multi-protocol PC interface card. Initial offering is with
- * the DLCI driver, providing Frame Relay support for linux.
- *
- * Global definitions for the Frame relay interface.
- *
- * Version: @(#)sdla.c 0.30 12 Sep 1996
- *
- * Credits: Sangoma Technologies, for the use of 2 cards for an extended
- * period of time.
- * David Mandelstam <dm@sangoma.com> for getting me started on
- * this project, and incentive to complete it.
- * Gene Kozen <74604.152@compuserve.com> for providing me with
- * important information about the cards.
- *
- * Author: Mike McLagan <mike.mclagan@linux.org>
- *
- * Changes:
- * 0.15 Mike McLagan Improved error handling, packet dropping
- * 0.20 Mike McLagan New transmit/receive flags for config
- * If in FR mode, don't accept packets from
- * non DLCI devices.
- * 0.25 Mike McLagan Fixed problem with rejecting packets
- * from non DLCI devices.
- * 0.30 Mike McLagan Fixed kernel panic when used with modified
- * ifconfig
- */
-
-#define pr_fmt(fmt) KBUILD_MODNAME ": " fmt
-
-#include <linux/module.h>
-#include <linux/kernel.h>
-#include <linux/types.h>
-#include <linux/fcntl.h>
-#include <linux/interrupt.h>
-#include <linux/ptrace.h>
-#include <linux/ioport.h>
-#include <linux/in.h>
-#include <linux/slab.h>
-#include <linux/string.h>
-#include <linux/timer.h>
-#include <linux/errno.h>
-#include <linux/init.h>
-#include <linux/netdevice.h>
-#include <linux/skbuff.h>
-#include <linux/if_arp.h>
-#include <linux/if_frad.h>
-#include <linux/sdla.h>
-#include <linux/bitops.h>
-
-#include <asm/io.h>
-#include <asm/dma.h>
-#include <linux/uaccess.h>
-
-static const char* version = "SDLA driver v0.30, 12 Sep 1996, mike.mclagan@linux.org";
-
-static unsigned int valid_port[] = { 0x250, 0x270, 0x280, 0x300, 0x350, 0x360, 0x380, 0x390};
-
-static unsigned int valid_mem[] = {
- 0xA0000, 0xA2000, 0xA4000, 0xA6000, 0xA8000, 0xAA000, 0xAC000, 0xAE000,
- 0xB0000, 0xB2000, 0xB4000, 0xB6000, 0xB8000, 0xBA000, 0xBC000, 0xBE000,
- 0xC0000, 0xC2000, 0xC4000, 0xC6000, 0xC8000, 0xCA000, 0xCC000, 0xCE000,
- 0xD0000, 0xD2000, 0xD4000, 0xD6000, 0xD8000, 0xDA000, 0xDC000, 0xDE000,
- 0xE0000, 0xE2000, 0xE4000, 0xE6000, 0xE8000, 0xEA000, 0xEC000, 0xEE000};
-
-static DEFINE_SPINLOCK(sdla_lock);
-
-/*********************************************************
- *
- * these are the core routines that access the card itself
- *
- *********************************************************/
-
-#define SDLA_WINDOW(dev,addr) outb((((addr) >> 13) & 0x1F), (dev)->base_addr + SDLA_REG_Z80_WINDOW)
-
-static void __sdla_read(struct net_device *dev, int addr, void *buf, short len)
-{
- char *temp;
- const void *base;
- int offset, bytes;
-
- temp = buf;
- while(len)
- {
- offset = addr & SDLA_ADDR_MASK;
- bytes = offset + len > SDLA_WINDOW_SIZE ? SDLA_WINDOW_SIZE - offset : len;
- base = (const void *) (dev->mem_start + offset);
-
- SDLA_WINDOW(dev, addr);
- memcpy(temp, base, bytes);
-
- addr += bytes;
- temp += bytes;
- len -= bytes;
- }
-}
-
-static void sdla_read(struct net_device *dev, int addr, void *buf, short len)
-{
- unsigned long flags;
- spin_lock_irqsave(&sdla_lock, flags);
- __sdla_read(dev, addr, buf, len);
- spin_unlock_irqrestore(&sdla_lock, flags);
-}
-
-static void __sdla_write(struct net_device *dev, int addr,
- const void *buf, short len)
-{
- const char *temp;
- void *base;
- int offset, bytes;
-
- temp = buf;
- while(len)
- {
- offset = addr & SDLA_ADDR_MASK;
- bytes = offset + len > SDLA_WINDOW_SIZE ? SDLA_WINDOW_SIZE - offset : len;
- base = (void *) (dev->mem_start + offset);
-
- SDLA_WINDOW(dev, addr);
- memcpy(base, temp, bytes);
-
- addr += bytes;
- temp += bytes;
- len -= bytes;
- }
-}
-
-static void sdla_write(struct net_device *dev, int addr,
- const void *buf, short len)
-{
- unsigned long flags;
-
- spin_lock_irqsave(&sdla_lock, flags);
- __sdla_write(dev, addr, buf, len);
- spin_unlock_irqrestore(&sdla_lock, flags);
-}
-
-
-static void sdla_clear(struct net_device *dev)
-{
- unsigned long flags;
- char *base;
- int len, addr, bytes;
-
- len = 65536;
- addr = 0;
- bytes = SDLA_WINDOW_SIZE;
- base = (void *) dev->mem_start;
-
- spin_lock_irqsave(&sdla_lock, flags);
- while(len)
- {
- SDLA_WINDOW(dev, addr);
- memset(base, 0, bytes);
-
- addr += bytes;
- len -= bytes;
- }
- spin_unlock_irqrestore(&sdla_lock, flags);
-
-}
-
-static char sdla_byte(struct net_device *dev, int addr)
-{
- unsigned long flags;
- char byte, *temp;
-
- temp = (void *) (dev->mem_start + (addr & SDLA_ADDR_MASK));
-
- spin_lock_irqsave(&sdla_lock, flags);
- SDLA_WINDOW(dev, addr);
- byte = *temp;
- spin_unlock_irqrestore(&sdla_lock, flags);
-
- return byte;
-}
-
-static void sdla_stop(struct net_device *dev)
-{
- struct frad_local *flp;
-
- flp = netdev_priv(dev);
- switch(flp->type)
- {
- case SDLA_S502A:
- outb(SDLA_S502A_HALT, dev->base_addr + SDLA_REG_CONTROL);
- flp->state = SDLA_HALT;
- break;
- case SDLA_S502E:
- outb(SDLA_HALT, dev->base_addr + SDLA_REG_Z80_CONTROL);
- outb(SDLA_S502E_ENABLE, dev->base_addr + SDLA_REG_CONTROL);
- flp->state = SDLA_S502E_ENABLE;
- break;
- case SDLA_S507:
- flp->state &= ~SDLA_CPUEN;
- outb(flp->state, dev->base_addr + SDLA_REG_CONTROL);
- break;
- case SDLA_S508:
- flp->state &= ~SDLA_CPUEN;
- outb(flp->state, dev->base_addr + SDLA_REG_CONTROL);
- break;
- }
-}
-
-static void sdla_start(struct net_device *dev)
-{
- struct frad_local *flp;
-
- flp = netdev_priv(dev);
- switch(flp->type)
- {
- case SDLA_S502A:
- outb(SDLA_S502A_NMI, dev->base_addr + SDLA_REG_CONTROL);
- outb(SDLA_S502A_START, dev->base_addr + SDLA_REG_CONTROL);
- flp->state = SDLA_S502A_START;
- break;
- case SDLA_S502E:
- outb(SDLA_S502E_CPUEN, dev->base_addr + SDLA_REG_Z80_CONTROL);
- outb(0x00, dev->base_addr + SDLA_REG_CONTROL);
- flp->state = 0;
- break;
- case SDLA_S507:
- flp->state |= SDLA_CPUEN;
- outb(flp->state, dev->base_addr + SDLA_REG_CONTROL);
- break;
- case SDLA_S508:
- flp->state |= SDLA_CPUEN;
- outb(flp->state, dev->base_addr + SDLA_REG_CONTROL);
- break;
- }
-}
-
-/****************************************************
- *
- * this is used for the S502A/E cards to determine
- * the speed of the onboard CPU. Calibration is
- * necessary for the Frame Relay code uploaded
- * later. Incorrect results cause timing problems
- * with link checks & status messages
- *
- ***************************************************/
-
-static int sdla_z80_poll(struct net_device *dev, int z80_addr, int jiffs, char resp1, char resp2)
-{
- unsigned long start, done, now;
- char resp, *temp;
-
- start = now = jiffies;
- done = jiffies + jiffs;
-
- temp = (void *)dev->mem_start;
- temp += z80_addr & SDLA_ADDR_MASK;
-
- resp = ~resp1;
- while (time_before(jiffies, done) && (resp != resp1) && (!resp2 || (resp != resp2)))
- {
- if (jiffies != now)
- {
- SDLA_WINDOW(dev, z80_addr);
- now = jiffies;
- resp = *temp;
- }
- }
- return time_before(jiffies, done) ? jiffies - start : -1;
-}
-
-/* constants for Z80 CPU speed */
-#define Z80_READY '1' /* Z80 is ready to begin */
-#define LOADER_READY '2' /* driver is ready to begin */
-#define Z80_SCC_OK '3' /* SCC is on board */
-#define Z80_SCC_BAD '4' /* SCC was not found */
-
-static int sdla_cpuspeed(struct net_device *dev, struct ifreq *ifr)
-{
- int jiffs;
- char data;
-
- sdla_start(dev);
- if (sdla_z80_poll(dev, 0, 3*HZ, Z80_READY, 0) < 0)
- return -EIO;
-
- data = LOADER_READY;
- sdla_write(dev, 0, &data, 1);
-
- if ((jiffs = sdla_z80_poll(dev, 0, 8*HZ, Z80_SCC_OK, Z80_SCC_BAD)) < 0)
- return -EIO;
-
- sdla_stop(dev);
- sdla_read(dev, 0, &data, 1);
-
- if (data == Z80_SCC_BAD)
- {
- printk("%s: SCC bad\n", dev->name);
- return -EIO;
- }
-
- if (data != Z80_SCC_OK)
- return -EINVAL;
-
- if (jiffs < 165)
- ifr->ifr_mtu = SDLA_CPU_16M;
- else if (jiffs < 220)
- ifr->ifr_mtu = SDLA_CPU_10M;
- else if (jiffs < 258)
- ifr->ifr_mtu = SDLA_CPU_8M;
- else if (jiffs < 357)
- ifr->ifr_mtu = SDLA_CPU_7M;
- else if (jiffs < 467)
- ifr->ifr_mtu = SDLA_CPU_5M;
- else
- ifr->ifr_mtu = SDLA_CPU_3M;
-
- return 0;
-}
-
-/************************************************
- *
- * Direct interaction with the Frame Relay code
- * starts here.
- *
- ************************************************/
-
-struct _dlci_stat
-{
- short dlci;
- char flags;
-} __packed;
-
-struct _frad_stat
-{
- char flags;
- struct _dlci_stat dlcis[SDLA_MAX_DLCI];
-};
-
-static void sdla_errors(struct net_device *dev, int cmd, int dlci, int ret, int len, void *data)
-{
- struct _dlci_stat *pstatus;
- short *pdlci;
- int i;
- char *state, line[30];
-
- switch (ret)
- {
- case SDLA_RET_MODEM:
- state = data;
- if (*state & SDLA_MODEM_DCD_LOW)
- netdev_info(dev, "Modem DCD unexpectedly low!\n");
- if (*state & SDLA_MODEM_CTS_LOW)
- netdev_info(dev, "Modem CTS unexpectedly low!\n");
- /* I should probably do something about this! */
- break;
-
- case SDLA_RET_CHANNEL_OFF:
- netdev_info(dev, "Channel became inoperative!\n");
- /* same here */
- break;
-
- case SDLA_RET_CHANNEL_ON:
- netdev_info(dev, "Channel became operative!\n");
- /* same here */
- break;
-
- case SDLA_RET_DLCI_STATUS:
- netdev_info(dev, "Status change reported by Access Node\n");
- len /= sizeof(struct _dlci_stat);
- for(pstatus = data, i=0;i < len;i++,pstatus++)
- {
- if (pstatus->flags & SDLA_DLCI_NEW)
- state = "new";
- else if (pstatus->flags & SDLA_DLCI_DELETED)
- state = "deleted";
- else if (pstatus->flags & SDLA_DLCI_ACTIVE)
- state = "active";
- else
- {
- sprintf(line, "unknown status: %02X", pstatus->flags);
- state = line;
- }
- netdev_info(dev, "DLCI %i: %s\n",
- pstatus->dlci, state);
- /* same here */
- }
- break;
-
- case SDLA_RET_DLCI_UNKNOWN:
- netdev_info(dev, "Received unknown DLCIs:");
- len /= sizeof(short);
- for(pdlci = data,i=0;i < len;i++,pdlci++)
- pr_cont(" %i", *pdlci);
- pr_cont("\n");
- break;
-
- case SDLA_RET_TIMEOUT:
- netdev_err(dev, "Command timed out!\n");
- break;
-
- case SDLA_RET_BUF_OVERSIZE:
- netdev_info(dev, "Bc/CIR overflow, acceptable size is %i\n",
- len);
- break;
-
- case SDLA_RET_BUF_TOO_BIG:
- netdev_info(dev, "Buffer size over specified max of %i\n",
- len);
- break;
-
- case SDLA_RET_CHANNEL_INACTIVE:
- case SDLA_RET_DLCI_INACTIVE:
- case SDLA_RET_CIR_OVERFLOW:
- case SDLA_RET_NO_BUFS:
- if (cmd == SDLA_INFORMATION_WRITE)
- break;
- fallthrough;
-
- default:
- netdev_dbg(dev, "Cmd 0x%02X generated return code 0x%02X\n",
- cmd, ret);
- /* Further processing could be done here */
- break;
- }
-}
-
-static int sdla_cmd(struct net_device *dev, int cmd, short dlci, short flags,
- void *inbuf, short inlen, void *outbuf, short *outlen)
-{
- static struct _frad_stat status;
- struct frad_local *flp;
- struct sdla_cmd *cmd_buf;
- unsigned long pflags;
- unsigned long jiffs;
- int ret, waiting, len;
- long window;
-
- flp = netdev_priv(dev);
- window = flp->type == SDLA_S508 ? SDLA_508_CMD_BUF : SDLA_502_CMD_BUF;
- cmd_buf = (struct sdla_cmd *)(dev->mem_start + (window & SDLA_ADDR_MASK));
- ret = 0;
- len = 0;
- jiffs = jiffies + HZ; /* 1 second is plenty */
-
- spin_lock_irqsave(&sdla_lock, pflags);
- SDLA_WINDOW(dev, window);
- cmd_buf->cmd = cmd;
- cmd_buf->dlci = dlci;
- cmd_buf->flags = flags;
-
- if (inbuf)
- memcpy(cmd_buf->data, inbuf, inlen);
-
- cmd_buf->length = inlen;
-
- cmd_buf->opp_flag = 1;
- spin_unlock_irqrestore(&sdla_lock, pflags);
-
- waiting = 1;
- len = 0;
- while (waiting && time_before_eq(jiffies, jiffs))
- {
- if (waiting++ % 3)
- {
- spin_lock_irqsave(&sdla_lock, pflags);
- SDLA_WINDOW(dev, window);
- waiting = ((volatile int)(cmd_buf->opp_flag));
- spin_unlock_irqrestore(&sdla_lock, pflags);
- }
- }
-
- if (!waiting)
- {
-
- spin_lock_irqsave(&sdla_lock, pflags);
- SDLA_WINDOW(dev, window);
- ret = cmd_buf->retval;
- len = cmd_buf->length;
- if (outbuf && outlen)
- {
- *outlen = *outlen >= len ? len : *outlen;
-
- if (*outlen)
- memcpy(outbuf, cmd_buf->data, *outlen);
- }
-
- /* This is a local copy that's used for error handling */
- if (ret)
- memcpy(&status, cmd_buf->data, len > sizeof(status) ? sizeof(status) : len);
-
- spin_unlock_irqrestore(&sdla_lock, pflags);
- }
- else
- ret = SDLA_RET_TIMEOUT;
-
- if (ret != SDLA_RET_OK)
- sdla_errors(dev, cmd, dlci, ret, len, &status);
-
- return ret;
-}
-
-/***********************************************
- *
- * these functions are called by the DLCI driver
- *
- ***********************************************/
-
-static int sdla_reconfig(struct net_device *dev);
-
-static int sdla_activate(struct net_device *slave, struct net_device *master)
-{
- struct frad_local *flp;
- int i;
-
- flp = netdev_priv(slave);
-
- for(i=0;i<CONFIG_DLCI_MAX;i++)
- if (flp->master[i] == master)
- break;
-
- if (i == CONFIG_DLCI_MAX)
- return -ENODEV;
-
- flp->dlci[i] = abs(flp->dlci[i]);
-
- if (netif_running(slave) && (flp->config.station == FRAD_STATION_NODE))
- sdla_cmd(slave, SDLA_ACTIVATE_DLCI, 0, 0, &flp->dlci[i], sizeof(short), NULL, NULL);
-
- return 0;
-}
-
-static int sdla_deactivate(struct net_device *slave, struct net_device *master)
-{
- struct frad_local *flp;
- int i;
-
- flp = netdev_priv(slave);
-
- for(i=0;i<CONFIG_DLCI_MAX;i++)
- if (flp->master[i] == master)
- break;
-
- if (i == CONFIG_DLCI_MAX)
- return -ENODEV;
-
- flp->dlci[i] = -abs(flp->dlci[i]);
-
- if (netif_running(slave) && (flp->config.station == FRAD_STATION_NODE))
- sdla_cmd(slave, SDLA_DEACTIVATE_DLCI, 0, 0, &flp->dlci[i], sizeof(short), NULL, NULL);
-
- return 0;
-}
-
-static int sdla_assoc(struct net_device *slave, struct net_device *master)
-{
- struct frad_local *flp;
- int i;
-
- if (master->type != ARPHRD_DLCI)
- return -EINVAL;
-
- flp = netdev_priv(slave);
-
- for(i=0;i<CONFIG_DLCI_MAX;i++)
- {
- if (!flp->master[i])
- break;
- if (abs(flp->dlci[i]) == *(short *)(master->dev_addr))
- return -EADDRINUSE;
- }
-
- if (i == CONFIG_DLCI_MAX)
- return -EMLINK; /* #### Alan: Comments on this ?? */
-
-
- flp->master[i] = master;
- flp->dlci[i] = -*(short *)(master->dev_addr);
- master->mtu = slave->mtu;
-
- if (netif_running(slave)) {
- if (flp->config.station == FRAD_STATION_CPE)
- sdla_reconfig(slave);
- else
- sdla_cmd(slave, SDLA_ADD_DLCI, 0, 0, master->dev_addr, sizeof(short), NULL, NULL);
- }
-
- return 0;
-}
-
-static int sdla_deassoc(struct net_device *slave, struct net_device *master)
-{
- struct frad_local *flp;
- int i;
-
- flp = netdev_priv(slave);
-
- for(i=0;i<CONFIG_DLCI_MAX;i++)
- if (flp->master[i] == master)
- break;
-
- if (i == CONFIG_DLCI_MAX)
- return -ENODEV;
-
- flp->master[i] = NULL;
- flp->dlci[i] = 0;
-
-
- if (netif_running(slave)) {
- if (flp->config.station == FRAD_STATION_CPE)
- sdla_reconfig(slave);
- else
- sdla_cmd(slave, SDLA_DELETE_DLCI, 0, 0, master->dev_addr, sizeof(short), NULL, NULL);
- }
-
- return 0;
-}
-
-static int sdla_dlci_conf(struct net_device *slave, struct net_device *master, int get)
-{
- struct frad_local *flp;
- struct dlci_local *dlp;
- int i;
- short len, ret;
-
- flp = netdev_priv(slave);
-
- for(i=0;i<CONFIG_DLCI_MAX;i++)
- if (flp->master[i] == master)
- break;
-
- if (i == CONFIG_DLCI_MAX)
- return -ENODEV;
-
- dlp = netdev_priv(master);
-
- ret = SDLA_RET_OK;
- len = sizeof(struct dlci_conf);
- if (netif_running(slave)) {
- if (get)
- ret = sdla_cmd(slave, SDLA_READ_DLCI_CONFIGURATION, abs(flp->dlci[i]), 0,
- NULL, 0, &dlp->config, &len);
- else
- ret = sdla_cmd(slave, SDLA_SET_DLCI_CONFIGURATION, abs(flp->dlci[i]), 0,
- &dlp->config, sizeof(struct dlci_conf) - 4 * sizeof(short), NULL, NULL);
- }
-
- return ret == SDLA_RET_OK ? 0 : -EIO;
-}
-
-/**************************
- *
- * now for the Linux driver
- *
- **************************/
-
-/* NOTE: the DLCI driver deals with freeing the SKB!! */
-static netdev_tx_t sdla_transmit(struct sk_buff *skb,
- struct net_device *dev)
-{
- struct frad_local *flp;
- int ret, addr, accept, i;
- short size;
- unsigned long flags;
- struct buf_entry *pbuf;
-
- flp = netdev_priv(dev);
- ret = 0;
- accept = 1;
-
- netif_stop_queue(dev);
-
- /*
- * stupid GateD insists on setting up the multicast router thru us
- * and we're ill equipped to handle a non Frame Relay packet at this
- * time!
- */
-
- accept = 1;
- switch (dev->type)
- {
- case ARPHRD_FRAD:
- if (skb->dev->type != ARPHRD_DLCI)
- {
- netdev_warn(dev, "Non DLCI device, type %i, tried to send on FRAD module\n",
- skb->dev->type);
- accept = 0;
- }
- break;
- default:
- netdev_warn(dev, "unknown firmware type 0x%04X\n",
- dev->type);
- accept = 0;
- break;
- }
- if (accept)
- {
- /* this is frame specific, but till there's a PPP module, it's the default */
- switch (flp->type)
- {
- case SDLA_S502A:
- case SDLA_S502E:
- ret = sdla_cmd(dev, SDLA_INFORMATION_WRITE, *(short *)(skb->dev->dev_addr), 0, skb->data, skb->len, NULL, NULL);
- break;
- case SDLA_S508:
- size = sizeof(addr);
- ret = sdla_cmd(dev, SDLA_INFORMATION_WRITE, *(short *)(skb->dev->dev_addr), 0, NULL, skb->len, &addr, &size);
- if (ret == SDLA_RET_OK)
- {
-
- spin_lock_irqsave(&sdla_lock, flags);
- SDLA_WINDOW(dev, addr);
- pbuf = (void *)(dev->mem_start + (addr & SDLA_ADDR_MASK));
- __sdla_write(dev, pbuf->buf_addr, skb->data, skb->len);
- SDLA_WINDOW(dev, addr);
- pbuf->opp_flag = 1;
- spin_unlock_irqrestore(&sdla_lock, flags);
- }
- break;
- }
-
- switch (ret)
- {
- case SDLA_RET_OK:
- dev->stats.tx_packets++;
- break;
-
- case SDLA_RET_CIR_OVERFLOW:
- case SDLA_RET_BUF_OVERSIZE:
- case SDLA_RET_NO_BUFS:
- dev->stats.tx_dropped++;
- break;
-
- default:
- dev->stats.tx_errors++;
- break;
- }
- }
- netif_wake_queue(dev);
- for(i=0;i<CONFIG_DLCI_MAX;i++)
- {
- if(flp->master[i]!=NULL)
- netif_wake_queue(flp->master[i]);
- }
-
- dev_kfree_skb(skb);
- return NETDEV_TX_OK;
-}
-
-static void sdla_receive(struct net_device *dev)
-{
- struct net_device *master;
- struct frad_local *flp;
- struct dlci_local *dlp;
- struct sk_buff *skb;
-
- struct sdla_cmd *cmd;
- struct buf_info *pbufi;
- struct buf_entry *pbuf;
-
- unsigned long flags;
- int i=0, received, success, addr, buf_base, buf_top;
- short dlci, len, len2, split;
-
- flp = netdev_priv(dev);
- success = 1;
- received = addr = buf_top = buf_base = 0;
- len = dlci = 0;
- skb = NULL;
- master = NULL;
- cmd = NULL;
- pbufi = NULL;
- pbuf = NULL;
-
- spin_lock_irqsave(&sdla_lock, flags);
-
- switch (flp->type)
- {
- case SDLA_S502A:
- case SDLA_S502E:
- cmd = (void *) (dev->mem_start + (SDLA_502_RCV_BUF & SDLA_ADDR_MASK));
- SDLA_WINDOW(dev, SDLA_502_RCV_BUF);
- success = cmd->opp_flag;
- if (!success)
- break;
-
- dlci = cmd->dlci;
- len = cmd->length;
- break;
-
- case SDLA_S508:
- pbufi = (void *) (dev->mem_start + (SDLA_508_RXBUF_INFO & SDLA_ADDR_MASK));
- SDLA_WINDOW(dev, SDLA_508_RXBUF_INFO);
- pbuf = (void *) (dev->mem_start + ((pbufi->rse_base + flp->buffer * sizeof(struct buf_entry)) & SDLA_ADDR_MASK));
- success = pbuf->opp_flag;
- if (!success)
- break;
-
- buf_top = pbufi->buf_top;
- buf_base = pbufi->buf_base;
- dlci = pbuf->dlci;
- len = pbuf->length;
- addr = pbuf->buf_addr;
- break;
- }
-
- /* common code, find the DLCI and get the SKB */
- if (success)
- {
- for (i=0;i<CONFIG_DLCI_MAX;i++)
- if (flp->dlci[i] == dlci)
- break;
-
- if (i == CONFIG_DLCI_MAX)
- {
- netdev_notice(dev, "Received packet from invalid DLCI %i, ignoring\n",
- dlci);
- dev->stats.rx_errors++;
- success = 0;
- }
- }
-
- if (success)
- {
- master = flp->master[i];
- skb = dev_alloc_skb(len + sizeof(struct frhdr));
- if (skb == NULL)
- {
- netdev_notice(dev, "Memory squeeze, dropping packet\n");
- dev->stats.rx_dropped++;
- success = 0;
- }
- else
- skb_reserve(skb, sizeof(struct frhdr));
- }
-
- /* pick up the data */
- switch (flp->type)
- {
- case SDLA_S502A:
- case SDLA_S502E:
- if (success)
- __sdla_read(dev, SDLA_502_RCV_BUF + SDLA_502_DATA_OFS, skb_put(skb,len), len);
-
- SDLA_WINDOW(dev, SDLA_502_RCV_BUF);
- cmd->opp_flag = 0;
- break;
-
- case SDLA_S508:
- if (success)
- {
- /* is this buffer split off the end of the internal ring buffer */
- split = addr + len > buf_top + 1 ? len - (buf_top - addr + 1) : 0;
- len2 = len - split;
-
- __sdla_read(dev, addr, skb_put(skb, len2), len2);
- if (split)
- __sdla_read(dev, buf_base, skb_put(skb, split), split);
- }
-
- /* increment the buffer we're looking at */
- SDLA_WINDOW(dev, SDLA_508_RXBUF_INFO);
- flp->buffer = (flp->buffer + 1) % pbufi->rse_num;
- pbuf->opp_flag = 0;
- break;
- }
-
- if (success)
- {
- dev->stats.rx_packets++;
- dlp = netdev_priv(master);
- (*dlp->receive)(skb, master);
- }
-
- spin_unlock_irqrestore(&sdla_lock, flags);
-}
-
-static irqreturn_t sdla_isr(int dummy, void *dev_id)
-{
- struct net_device *dev;
- struct frad_local *flp;
- char byte;
-
- dev = dev_id;
-
- flp = netdev_priv(dev);
-
- if (!flp->initialized)
- {
- netdev_warn(dev, "irq %d for uninitialized device\n", dev->irq);
- return IRQ_NONE;
- }
-
- byte = sdla_byte(dev, flp->type == SDLA_S508 ? SDLA_508_IRQ_INTERFACE : SDLA_502_IRQ_INTERFACE);
- switch (byte)
- {
- case SDLA_INTR_RX:
- sdla_receive(dev);
- break;
-
- /* the command will get an error return, which is processed above */
- case SDLA_INTR_MODEM:
- case SDLA_INTR_STATUS:
- sdla_cmd(dev, SDLA_READ_DLC_STATUS, 0, 0, NULL, 0, NULL, NULL);
- break;
-
- case SDLA_INTR_TX:
- case SDLA_INTR_COMPLETE:
- case SDLA_INTR_TIMER:
- netdev_warn(dev, "invalid irq flag 0x%02X\n", byte);
- break;
- }
-
- /* the S502E requires a manual acknowledgement of the interrupt */
- if (flp->type == SDLA_S502E)
- {
- flp->state &= ~SDLA_S502E_INTACK;
- outb(flp->state, dev->base_addr + SDLA_REG_CONTROL);
- flp->state |= SDLA_S502E_INTACK;
- outb(flp->state, dev->base_addr + SDLA_REG_CONTROL);
- }
-
- /* this clears the byte, informing the Z80 we're done */
- byte = 0;
- sdla_write(dev, flp->type == SDLA_S508 ? SDLA_508_IRQ_INTERFACE : SDLA_502_IRQ_INTERFACE, &byte, sizeof(byte));
- return IRQ_HANDLED;
-}
-
-static void sdla_poll(struct timer_list *t)
-{
- struct frad_local *flp = from_timer(flp, t, timer);
- struct net_device *dev = flp->dev;
-
- if (sdla_byte(dev, SDLA_502_RCV_BUF))
- sdla_receive(dev);
-
- flp->timer.expires = 1;
- add_timer(&flp->timer);
-}
-
-static int sdla_close(struct net_device *dev)
-{
- struct frad_local *flp;
- struct intr_info intr;
- int len, i;
- short dlcis[CONFIG_DLCI_MAX];
-
- flp = netdev_priv(dev);
-
- len = 0;
- for(i=0;i<CONFIG_DLCI_MAX;i++)
- if (flp->dlci[i])
- dlcis[len++] = abs(flp->dlci[i]);
- len *= 2;
-
- if (flp->config.station == FRAD_STATION_NODE)
- {
- for(i=0;i<CONFIG_DLCI_MAX;i++)
- if (flp->dlci[i] > 0)
- sdla_cmd(dev, SDLA_DEACTIVATE_DLCI, 0, 0, dlcis, len, NULL, NULL);
- sdla_cmd(dev, SDLA_DELETE_DLCI, 0, 0, &flp->dlci[i], sizeof(flp->dlci[i]), NULL, NULL);
- }
-
- memset(&intr, 0, sizeof(intr));
- /* let's start up the reception */
- switch(flp->type)
- {
- case SDLA_S502A:
- del_timer(&flp->timer);
- break;
-
- case SDLA_S502E:
- sdla_cmd(dev, SDLA_SET_IRQ_TRIGGER, 0, 0, &intr, sizeof(char) + sizeof(short), NULL, NULL);
- flp->state &= ~SDLA_S502E_INTACK;
- outb(flp->state, dev->base_addr + SDLA_REG_CONTROL);
- break;
-
- case SDLA_S507:
- break;
-
- case SDLA_S508:
- sdla_cmd(dev, SDLA_SET_IRQ_TRIGGER, 0, 0, &intr, sizeof(struct intr_info), NULL, NULL);
- flp->state &= ~SDLA_S508_INTEN;
- outb(flp->state, dev->base_addr + SDLA_REG_CONTROL);
- break;
- }
-
- sdla_cmd(dev, SDLA_DISABLE_COMMUNICATIONS, 0, 0, NULL, 0, NULL, NULL);
-
- netif_stop_queue(dev);
-
- return 0;
-}
-
-struct conf_data {
- struct frad_conf config;
- short dlci[CONFIG_DLCI_MAX];
-};
-
-static int sdla_open(struct net_device *dev)
-{
- struct frad_local *flp;
- struct dlci_local *dlp;
- struct conf_data data;
- struct intr_info intr;
- int len, i;
- char byte;
-
- flp = netdev_priv(dev);
-
- if (!flp->initialized)
- return -EPERM;
-
- if (!flp->configured)
- return -EPERM;
-
- /* time to send in the configuration */
- len = 0;
- for(i=0;i<CONFIG_DLCI_MAX;i++)
- if (flp->dlci[i])
- data.dlci[len++] = abs(flp->dlci[i]);
- len *= 2;
-
- memcpy(&data.config, &flp->config, sizeof(struct frad_conf));
- len += sizeof(struct frad_conf);
-
- sdla_cmd(dev, SDLA_DISABLE_COMMUNICATIONS, 0, 0, NULL, 0, NULL, NULL);
- sdla_cmd(dev, SDLA_SET_DLCI_CONFIGURATION, 0, 0, &data, len, NULL, NULL);
-
- if (flp->type == SDLA_S508)
- flp->buffer = 0;
-
- sdla_cmd(dev, SDLA_ENABLE_COMMUNICATIONS, 0, 0, NULL, 0, NULL, NULL);
-
- /* let's start up the reception */
- memset(&intr, 0, sizeof(intr));
- switch(flp->type)
- {
- case SDLA_S502A:
- flp->timer.expires = 1;
- add_timer(&flp->timer);
- break;
-
- case SDLA_S502E:
- flp->state |= SDLA_S502E_ENABLE;
- outb(flp->state, dev->base_addr + SDLA_REG_CONTROL);
- flp->state |= SDLA_S502E_INTACK;
- outb(flp->state, dev->base_addr + SDLA_REG_CONTROL);
- byte = 0;
- sdla_write(dev, SDLA_502_IRQ_INTERFACE, &byte, sizeof(byte));
- intr.flags = SDLA_INTR_RX | SDLA_INTR_STATUS | SDLA_INTR_MODEM;
- sdla_cmd(dev, SDLA_SET_IRQ_TRIGGER, 0, 0, &intr, sizeof(char) + sizeof(short), NULL, NULL);
- break;
-
- case SDLA_S507:
- break;
-
- case SDLA_S508:
- flp->state |= SDLA_S508_INTEN;
- outb(flp->state, dev->base_addr + SDLA_REG_CONTROL);
- byte = 0;
- sdla_write(dev, SDLA_508_IRQ_INTERFACE, &byte, sizeof(byte));
- intr.flags = SDLA_INTR_RX | SDLA_INTR_STATUS | SDLA_INTR_MODEM;
- intr.irq = dev->irq;
- sdla_cmd(dev, SDLA_SET_IRQ_TRIGGER, 0, 0, &intr, sizeof(struct intr_info), NULL, NULL);
- break;
- }
-
- if (flp->config.station == FRAD_STATION_CPE)
- {
- byte = SDLA_ICS_STATUS_ENQ;
- sdla_cmd(dev, SDLA_ISSUE_IN_CHANNEL_SIGNAL, 0, 0, &byte, sizeof(byte), NULL, NULL);
- }
- else
- {
- sdla_cmd(dev, SDLA_ADD_DLCI, 0, 0, data.dlci, len - sizeof(struct frad_conf), NULL, NULL);
- for(i=0;i<CONFIG_DLCI_MAX;i++)
- if (flp->dlci[i] > 0)
- sdla_cmd(dev, SDLA_ACTIVATE_DLCI, 0, 0, &flp->dlci[i], 2*sizeof(flp->dlci[i]), NULL, NULL);
- }
-
- /* configure any specific DLCI settings */
- for(i=0;i<CONFIG_DLCI_MAX;i++)
- if (flp->dlci[i])
- {
- dlp = netdev_priv(flp->master[i]);
- if (dlp->configured)
- sdla_cmd(dev, SDLA_SET_DLCI_CONFIGURATION, abs(flp->dlci[i]), 0, &dlp->config, sizeof(struct dlci_conf), NULL, NULL);
- }
-
- netif_start_queue(dev);
-
- return 0;
-}
-
-static int sdla_config(struct net_device *dev, struct frad_conf __user *conf, int get)
-{
- struct frad_local *flp;
- struct conf_data data;
- int i;
- short size;
-
- if (dev->type == 0xFFFF)
- return -EUNATCH;
-
- flp = netdev_priv(dev);
-
- if (!get)
- {
- if (netif_running(dev))
- return -EBUSY;
-
- if(copy_from_user(&data.config, conf, sizeof(struct frad_conf)))
- return -EFAULT;
-
- if (data.config.station & ~FRAD_STATION_NODE)
- return -EINVAL;
-
- if (data.config.flags & ~FRAD_VALID_FLAGS)
- return -EINVAL;
-
- if ((data.config.kbaud < 0) ||
- ((data.config.kbaud > 128) && (flp->type != SDLA_S508)))
- return -EINVAL;
-
- if (data.config.clocking & ~(FRAD_CLOCK_INT | SDLA_S508_PORT_RS232))
- return -EINVAL;
-
- if ((data.config.mtu < 0) || (data.config.mtu > SDLA_MAX_MTU))
- return -EINVAL;
-
- if ((data.config.T391 < 5) || (data.config.T391 > 30))
- return -EINVAL;
-
- if ((data.config.T392 < 5) || (data.config.T392 > 30))
- return -EINVAL;
-
- if ((data.config.N391 < 1) || (data.config.N391 > 255))
- return -EINVAL;
-
- if ((data.config.N392 < 1) || (data.config.N392 > 10))
- return -EINVAL;
-
- if ((data.config.N393 < 1) || (data.config.N393 > 10))
- return -EINVAL;
-
- memcpy(&flp->config, &data.config, sizeof(struct frad_conf));
- flp->config.flags |= SDLA_DIRECT_RECV;
-
- if (flp->type == SDLA_S508)
- flp->config.flags |= SDLA_TX70_RX30;
-
- if (dev->mtu != flp->config.mtu)
- {
- /* this is required to change the MTU */
- dev->mtu = flp->config.mtu;
- for(i=0;i<CONFIG_DLCI_MAX;i++)
- if (flp->master[i])
- flp->master[i]->mtu = flp->config.mtu;
- }
-
- flp->config.mtu += sizeof(struct frhdr);
-
- /* off to the races! */
- if (!flp->configured)
- sdla_start(dev);
-
- flp->configured = 1;
- }
- else
- {
- /* no sense reading if the CPU isn't started */
- if (netif_running(dev))
- {
- size = sizeof(data);
- if (sdla_cmd(dev, SDLA_READ_DLCI_CONFIGURATION, 0, 0, NULL, 0, &data, &size) != SDLA_RET_OK)
- return -EIO;
- }
- else
- if (flp->configured)
- memcpy(&data.config, &flp->config, sizeof(struct frad_conf));
- else
- memset(&data.config, 0, sizeof(struct frad_conf));
-
- memcpy(&flp->config, &data.config, sizeof(struct frad_conf));
- data.config.flags &= FRAD_VALID_FLAGS;
- data.config.mtu -= data.config.mtu > sizeof(struct frhdr) ? sizeof(struct frhdr) : data.config.mtu;
- return copy_to_user(conf, &data.config, sizeof(struct frad_conf))?-EFAULT:0;
- }
-
- return 0;
-}
-
-static int sdla_xfer(struct net_device *dev, struct sdla_mem __user *info, int read)
-{
- struct sdla_mem mem;
- char *temp;
-
- if(copy_from_user(&mem, info, sizeof(mem)))
- return -EFAULT;
-
- if (read)
- {
- temp = kzalloc(mem.len, GFP_KERNEL);
- if (!temp)
- return -ENOMEM;
- sdla_read(dev, mem.addr, temp, mem.len);
- if(copy_to_user(mem.data, temp, mem.len))
- {
- kfree(temp);
- return -EFAULT;
- }
- kfree(temp);
- }
- else
- {
- temp = memdup_user(mem.data, mem.len);
- if (IS_ERR(temp))
- return PTR_ERR(temp);
- sdla_write(dev, mem.addr, temp, mem.len);
- kfree(temp);
- }
- return 0;
-}
-
-static int sdla_reconfig(struct net_device *dev)
-{
- struct frad_local *flp;
- struct conf_data data;
- int i, len;
-
- flp = netdev_priv(dev);
-
- len = 0;
- for(i=0;i<CONFIG_DLCI_MAX;i++)
- if (flp->dlci[i])
- data.dlci[len++] = flp->dlci[i];
- len *= 2;
-
- memcpy(&data, &flp->config, sizeof(struct frad_conf));
- len += sizeof(struct frad_conf);
-
- sdla_cmd(dev, SDLA_DISABLE_COMMUNICATIONS, 0, 0, NULL, 0, NULL, NULL);
- sdla_cmd(dev, SDLA_SET_DLCI_CONFIGURATION, 0, 0, &data, len, NULL, NULL);
- sdla_cmd(dev, SDLA_ENABLE_COMMUNICATIONS, 0, 0, NULL, 0, NULL, NULL);
-
- return 0;
-}
-
-static int sdla_ioctl(struct net_device *dev, struct ifreq *ifr, int cmd)
-{
- struct frad_local *flp;
-
- if(!capable(CAP_NET_ADMIN))
- return -EPERM;
-
- flp = netdev_priv(dev);
-
- if (!flp->initialized)
- return -EINVAL;
-
- switch (cmd)
- {
- case FRAD_GET_CONF:
- case FRAD_SET_CONF:
- return sdla_config(dev, ifr->ifr_data, cmd == FRAD_GET_CONF);
-
- case SDLA_IDENTIFY:
- ifr->ifr_flags = flp->type;
- break;
-
- case SDLA_CPUSPEED:
- return sdla_cpuspeed(dev, ifr);
-
-/* ==========================================================
-NOTE: This is rather a useless action right now, as the
- current driver does not support protocols other than
- FR. However, Sangoma has modules for a number of
- other protocols in the works.
-============================================================*/
- case SDLA_PROTOCOL:
- if (flp->configured)
- return -EALREADY;
-
- switch (ifr->ifr_flags)
- {
- case ARPHRD_FRAD:
- dev->type = ifr->ifr_flags;
- break;
- default:
- return -ENOPROTOOPT;
- }
- break;
-
- case SDLA_CLEARMEM:
- sdla_clear(dev);
- break;
-
- case SDLA_WRITEMEM:
- case SDLA_READMEM:
- if(!capable(CAP_SYS_RAWIO))
- return -EPERM;
- return sdla_xfer(dev, ifr->ifr_data, cmd == SDLA_READMEM);
-
- case SDLA_START:
- sdla_start(dev);
- break;
-
- case SDLA_STOP:
- sdla_stop(dev);
- break;
-
- default:
- return -EOPNOTSUPP;
- }
- return 0;
-}
-
-static int sdla_change_mtu(struct net_device *dev, int new_mtu)
-{
- if (netif_running(dev))
- return -EBUSY;
-
- /* for now, you can't change the MTU! */
- return -EOPNOTSUPP;
-}
-
-static int sdla_set_config(struct net_device *dev, struct ifmap *map)
-{
- struct frad_local *flp;
- int i;
- char byte;
- unsigned base;
- int err = -EINVAL;
-
- flp = netdev_priv(dev);
-
- if (flp->initialized)
- return -EINVAL;
-
- for(i=0; i < ARRAY_SIZE(valid_port); i++)
- if (valid_port[i] == map->base_addr)
- break;
-
- if (i == ARRAY_SIZE(valid_port))
- return -EINVAL;
-
- if (!request_region(map->base_addr, SDLA_IO_EXTENTS, dev->name)){
- pr_warn("io-port 0x%04lx in use\n", dev->base_addr);
- return -EINVAL;
- }
- base = map->base_addr;
-
- /* test for card types, S502A, S502E, S507, S508 */
- /* these tests shut down the card completely, so clear the state */
- flp->type = SDLA_UNKNOWN;
- flp->state = 0;
-
- for(i=1;i<SDLA_IO_EXTENTS;i++)
- if (inb(base + i) != 0xFF)
- break;
-
- if (i == SDLA_IO_EXTENTS) {
- outb(SDLA_HALT, base + SDLA_REG_Z80_CONTROL);
- if ((inb(base + SDLA_S502_STS) & 0x0F) == 0x08) {
- outb(SDLA_S502E_INTACK, base + SDLA_REG_CONTROL);
- if ((inb(base + SDLA_S502_STS) & 0x0F) == 0x0C) {
- outb(SDLA_HALT, base + SDLA_REG_CONTROL);
- flp->type = SDLA_S502E;
- goto got_type;
- }
- }
- }
-
- for(byte=inb(base),i=0;i<SDLA_IO_EXTENTS;i++)
- if (inb(base + i) != byte)
- break;
-
- if (i == SDLA_IO_EXTENTS) {
- outb(SDLA_HALT, base + SDLA_REG_CONTROL);
- if ((inb(base + SDLA_S502_STS) & 0x7E) == 0x30) {
- outb(SDLA_S507_ENABLE, base + SDLA_REG_CONTROL);
- if ((inb(base + SDLA_S502_STS) & 0x7E) == 0x32) {
- outb(SDLA_HALT, base + SDLA_REG_CONTROL);
- flp->type = SDLA_S507;
- goto got_type;
- }
- }
- }
-
- outb(SDLA_HALT, base + SDLA_REG_CONTROL);
- if ((inb(base + SDLA_S508_STS) & 0x3F) == 0x00) {
- outb(SDLA_S508_INTEN, base + SDLA_REG_CONTROL);
- if ((inb(base + SDLA_S508_STS) & 0x3F) == 0x10) {
- outb(SDLA_HALT, base + SDLA_REG_CONTROL);
- flp->type = SDLA_S508;
- goto got_type;
- }
- }
-
- outb(SDLA_S502A_HALT, base + SDLA_REG_CONTROL);
- if (inb(base + SDLA_S502_STS) == 0x40) {
- outb(SDLA_S502A_START, base + SDLA_REG_CONTROL);
- if (inb(base + SDLA_S502_STS) == 0x40) {
- outb(SDLA_S502A_INTEN, base + SDLA_REG_CONTROL);
- if (inb(base + SDLA_S502_STS) == 0x44) {
- outb(SDLA_S502A_START, base + SDLA_REG_CONTROL);
- flp->type = SDLA_S502A;
- goto got_type;
- }
- }
- }
-
- netdev_notice(dev, "Unknown card type\n");
- err = -ENODEV;
- goto fail;
-
-got_type:
- switch(base) {
- case 0x270:
- case 0x280:
- case 0x380:
- case 0x390:
- if (flp->type != SDLA_S508 && flp->type != SDLA_S507)
- goto fail;
- }
-
- switch (map->irq) {
- case 2:
- if (flp->type != SDLA_S502E)
- goto fail;
- break;
-
- case 10:
- case 11:
- case 12:
- case 15:
- case 4:
- if (flp->type != SDLA_S508 && flp->type != SDLA_S507)
- goto fail;
- break;
- case 3:
- case 5:
- case 7:
- if (flp->type == SDLA_S502A)
- goto fail;
- break;
-
- default:
- goto fail;
- }
-
- err = -EAGAIN;
- if (request_irq(dev->irq, sdla_isr, 0, dev->name, dev))
- goto fail;
-
- if (flp->type == SDLA_S507) {
- switch(dev->irq) {
- case 3:
- flp->state = SDLA_S507_IRQ3;
- break;
- case 4:
- flp->state = SDLA_S507_IRQ4;
- break;
- case 5:
- flp->state = SDLA_S507_IRQ5;
- break;
- case 7:
- flp->state = SDLA_S507_IRQ7;
- break;
- case 10:
- flp->state = SDLA_S507_IRQ10;
- break;
- case 11:
- flp->state = SDLA_S507_IRQ11;
- break;
- case 12:
- flp->state = SDLA_S507_IRQ12;
- break;
- case 15:
- flp->state = SDLA_S507_IRQ15;
- break;
- }
- }
-
- for(i=0; i < ARRAY_SIZE(valid_mem); i++)
- if (valid_mem[i] == map->mem_start)
- break;
-
- err = -EINVAL;
- if (i == ARRAY_SIZE(valid_mem))
- goto fail2;
-
- if (flp->type == SDLA_S502A && (map->mem_start & 0xF000) >> 12 == 0x0E)
- goto fail2;
-
- if (flp->type != SDLA_S507 && map->mem_start >> 16 == 0x0B)
- goto fail2;
-
- if (flp->type == SDLA_S507 && map->mem_start >> 16 == 0x0D)
- goto fail2;
-
- byte = flp->type != SDLA_S508 ? SDLA_8K_WINDOW : 0;
- byte |= (map->mem_start & 0xF000) >> (12 + (flp->type == SDLA_S508 ? 1 : 0));
- switch(flp->type) {
- case SDLA_S502A:
- case SDLA_S502E:
- switch (map->mem_start >> 16) {
- case 0x0A:
- byte |= SDLA_S502_SEG_A;
- break;
- case 0x0C:
- byte |= SDLA_S502_SEG_C;
- break;
- case 0x0D:
- byte |= SDLA_S502_SEG_D;
- break;
- case 0x0E:
- byte |= SDLA_S502_SEG_E;
- break;
- }
- break;
- case SDLA_S507:
- switch (map->mem_start >> 16) {
- case 0x0A:
- byte |= SDLA_S507_SEG_A;
- break;
- case 0x0B:
- byte |= SDLA_S507_SEG_B;
- break;
- case 0x0C:
- byte |= SDLA_S507_SEG_C;
- break;
- case 0x0E:
- byte |= SDLA_S507_SEG_E;
- break;
- }
- break;
- case SDLA_S508:
- switch (map->mem_start >> 16) {
- case 0x0A:
- byte |= SDLA_S508_SEG_A;
- break;
- case 0x0C:
- byte |= SDLA_S508_SEG_C;
- break;
- case 0x0D:
- byte |= SDLA_S508_SEG_D;
- break;
- case 0x0E:
- byte |= SDLA_S508_SEG_E;
- break;
- }
- break;
- }
-
- /* set the memory bits, and enable access */
- outb(byte, base + SDLA_REG_PC_WINDOW);
-
- switch(flp->type)
- {
- case SDLA_S502E:
- flp->state = SDLA_S502E_ENABLE;
- break;
- case SDLA_S507:
- flp->state |= SDLA_MEMEN;
- break;
- case SDLA_S508:
- flp->state = SDLA_MEMEN;
- break;
- }
- outb(flp->state, base + SDLA_REG_CONTROL);
-
- dev->irq = map->irq;
- dev->base_addr = base;
- dev->mem_start = map->mem_start;
- dev->mem_end = dev->mem_start + 0x2000;
- flp->initialized = 1;
- return 0;
-
-fail2:
- free_irq(map->irq, dev);
-fail:
- release_region(base, SDLA_IO_EXTENTS);
- return err;
-}
-
-static const struct net_device_ops sdla_netdev_ops = {
- .ndo_open = sdla_open,
- .ndo_stop = sdla_close,
- .ndo_do_ioctl = sdla_ioctl,
- .ndo_set_config = sdla_set_config,
- .ndo_start_xmit = sdla_transmit,
- .ndo_change_mtu = sdla_change_mtu,
-};
-
-static void setup_sdla(struct net_device *dev)
-{
- struct frad_local *flp = netdev_priv(dev);
-
- netdev_boot_setup_check(dev);
-
- dev->netdev_ops = &sdla_netdev_ops;
- dev->flags = 0;
- dev->type = 0xFFFF;
- dev->hard_header_len = 0;
- dev->addr_len = 0;
- dev->mtu = SDLA_MAX_MTU;
-
- flp->activate = sdla_activate;
- flp->deactivate = sdla_deactivate;
- flp->assoc = sdla_assoc;
- flp->deassoc = sdla_deassoc;
- flp->dlci_conf = sdla_dlci_conf;
- flp->dev = dev;
-
- timer_setup(&flp->timer, sdla_poll, 0);
- flp->timer.expires = 1;
-}
-
-static struct net_device *sdla;
-
-static int __init init_sdla(void)
-{
- int err;
-
- printk("%s.\n", version);
-
- sdla = alloc_netdev(sizeof(struct frad_local), "sdla0",
- NET_NAME_UNKNOWN, setup_sdla);
- if (!sdla)
- return -ENOMEM;
-
- err = register_netdev(sdla);
- if (err)
- free_netdev(sdla);
-
- return err;
-}
-
-static void __exit exit_sdla(void)
-{
- struct frad_local *flp = netdev_priv(sdla);
-
- unregister_netdev(sdla);
- if (flp->initialized) {
- free_irq(sdla->irq, sdla);
- release_region(sdla->base_addr, SDLA_IO_EXTENTS);
- }
- del_timer_sync(&flp->timer);
- free_netdev(sdla);
-}
-
-MODULE_LICENSE("GPL");
-
-module_init(init_sdla);
-module_exit(exit_sdla);
*
* GPL LICENSE SUMMARY
*
- * Copyright(c) 2012 - 2014 Intel Corporation. All rights reserved.
* Copyright(c) 2013 - 2014 Intel Mobile Communications GmbH
* Copyright(c) 2016 - 2017 Intel Deutschland GmbH
- * Copyright(c) 2018 - 2019 Intel Corporation
+ * Copyright(c) 2012-2014, 2018 - 2020 Intel Corporation
*
* This program is free software; you can redistribute it and/or modify
* it under the terms of version 2 of the GNU General Public License as
*
* BSD LICENSE
*
- * Copyright(c) 2012 - 2014 Intel Corporation. All rights reserved.
* Copyright(c) 2013 - 2014 Intel Mobile Communications GmbH
* Copyright(c) 2016 - 2017 Intel Deutschland GmbH
- * Copyright(c) 2018 - 2019 Intel Corporation
+ * Copyright(c) 2012-2014, 2018 - 2020 Intel Corporation
* All rights reserved.
*
* Redistribution and use in source and binary forms, with or without
STA_FLG_MAX_AGG_SIZE_256K = (5 << STA_FLG_MAX_AGG_SIZE_SHIFT),
STA_FLG_MAX_AGG_SIZE_512K = (6 << STA_FLG_MAX_AGG_SIZE_SHIFT),
STA_FLG_MAX_AGG_SIZE_1024K = (7 << STA_FLG_MAX_AGG_SIZE_SHIFT),
- STA_FLG_MAX_AGG_SIZE_MSK = (7 << STA_FLG_MAX_AGG_SIZE_SHIFT),
+ STA_FLG_MAX_AGG_SIZE_2M = (8 << STA_FLG_MAX_AGG_SIZE_SHIFT),
+ STA_FLG_MAX_AGG_SIZE_4M = (9 << STA_FLG_MAX_AGG_SIZE_SHIFT),
+ STA_FLG_MAX_AGG_SIZE_MSK = (0xf << STA_FLG_MAX_AGG_SIZE_SHIFT),
STA_FLG_AGG_MPDU_DENS_SHIFT = 23,
STA_FLG_AGG_MPDU_DENS_2US = (4 << STA_FLG_AGG_MPDU_DENS_SHIFT),
* Copyright(c) 2012 - 2014 Intel Corporation. All rights reserved.
* Copyright(c) 2013 - 2015 Intel Mobile Communications GmbH
* Copyright(c) 2016 - 2017 Intel Deutschland GmbH
- * Copyright(c) 2018 - 2019 Intel Corporation
+ * Copyright(c) 2018 - 2020 Intel Corporation
*
* This program is free software; you can redistribute it and/or modify
* it under the terms of version 2 of the GNU General Public License as
* Copyright(c) 2012 - 2014 Intel Corporation. All rights reserved.
* Copyright(c) 2013 - 2015 Intel Mobile Communications GmbH
* Copyright(c) 2016 - 2017 Intel Deutschland GmbH
- * Copyright(c) 2018 - 2019 Intel Corporation
+ * Copyright(c) 2018 - 2020 Intel Corporation
* All rights reserved.
*
* Redistribution and use in source and binary forms, with or without
* able to run the GO Negotiation. Will not be fragmented and not
* repetitive. Valid only on the P2P Device MAC. Only the duration will
* be taken into account.
+ * @SESSION_PROTECT_CONF_MAX_ID: not used
*/
enum iwl_mvm_session_prot_conf_id {
SESSION_PROTECT_CONF_ASSOC,
SESSION_PROTECT_CONF_GO_CLIENT_ASSOC,
SESSION_PROTECT_CONF_P2P_DEVICE_DISCOV,
SESSION_PROTECT_CONF_P2P_GO_NEGOTIATION,
+ SESSION_PROTECT_CONF_MAX_ID,
}; /* SESSION_PROTECTION_CONF_ID_E_VER_1 */
/**
* @mac_id: the mac id for which the session protection started / ended
* @status: 1 means success, 0 means failure
* @start: 1 means the session protection started, 0 means it ended
- * @conf_id: the configuration id of the session that started / eneded
+ * @conf_id: see &enum iwl_mvm_session_prot_conf_id
*
* Note that any session protection will always get two notifications: start
* and end even the firmware could not schedule it.
#define IWL_CFG_RF_ID_HR 0x7
#define IWL_CFG_RF_ID_HR1 0x4
-#define IWL_CFG_NO_160 0x0
-#define IWL_CFG_160 0x1
+#define IWL_CFG_NO_160 0x1
+#define IWL_CFG_160 0x0
#define IWL_CFG_CORES_BT 0x0
#define IWL_CFG_CORES_BT_GNSS 0x5
#define CSR_MAC_SHADOW_REG_CTL2 (CSR_BASE + 0x0AC)
#define CSR_MAC_SHADOW_REG_CTL2_RX_WAKE 0xFFFF
+/* LTR control (since IWL_DEVICE_FAMILY_22000) */
+#define CSR_LTR_LONG_VAL_AD (CSR_BASE + 0x0D4)
+#define CSR_LTR_LONG_VAL_AD_NO_SNOOP_REQ 0x80000000
+#define CSR_LTR_LONG_VAL_AD_NO_SNOOP_SCALE 0x1c000000
+#define CSR_LTR_LONG_VAL_AD_NO_SNOOP_VAL 0x03ff0000
+#define CSR_LTR_LONG_VAL_AD_SNOOP_REQ 0x00008000
+#define CSR_LTR_LONG_VAL_AD_SNOOP_SCALE 0x00001c00
+#define CSR_LTR_LONG_VAL_AD_SNOOP_VAL 0x000003ff
+#define CSR_LTR_LONG_VAL_AD_SCALE_USEC 2
+
/* GIO Chicken Bits (PCI Express bus link power management) */
#define CSR_GIO_CHICKEN_BITS (CSR_BASE+0x100)
/* this would be a mac80211 bug ... but don't crash */
if (WARN_ON_ONCE(!mvmvif->phy_ctxt))
- return -EINVAL;
+ return test_bit(IWL_MVM_STATUS_HW_RESTART_REQUESTED, &mvm->status) ? 0 : -EINVAL;
/*
* If we are in a STA removal flow and in DQA mode:
goto out_unlock;
}
+ if (vif->type == NL80211_IFTYPE_STATION)
+ vif->bss_conf.he_support = sta->he_cap.has_he;
+
if (sta->tdls &&
(vif->p2p ||
iwl_mvm_tdls_sta_count(mvm, NULL) ==
mpdu_dens = sta->ht_cap.ampdu_density;
}
+
if (sta->vht_cap.vht_supported) {
agg_size = sta->vht_cap.cap &
IEEE80211_VHT_CAP_MAX_A_MPDU_LENGTH_EXPONENT_MASK;
agg_size = sta->ht_cap.ampdu_factor;
}
+ /* D6.0 10.12.2 A-MPDU length limit rules
+ * A STA indicates the maximum length of the A-MPDU preEOF padding
+ * that it can receive in an HE PPDU in the Maximum A-MPDU Length
+ * Exponent field in its HT Capabilities, VHT Capabilities,
+ * and HE 6 GHz Band Capabilities elements (if present) and the
+ * Maximum AMPDU Length Exponent Extension field in its HE
+ * Capabilities element
+ */
+ if (sta->he_cap.has_he)
+ agg_size += u8_get_bits(sta->he_cap.he_cap_elem.mac_cap_info[3],
+ IEEE80211_HE_MAC_CAP3_MAX_AMPDU_LEN_EXP_MASK);
+
+ /* Limit to max A-MPDU supported by FW */
+ if (agg_size > (STA_FLG_MAX_AGG_SIZE_4M >> STA_FLG_MAX_AGG_SIZE_SHIFT))
+ agg_size = (STA_FLG_MAX_AGG_SIZE_4M >>
+ STA_FLG_MAX_AGG_SIZE_SHIFT);
+
add_sta_cmd.station_flags |=
cpu_to_le32(agg_size << STA_FLG_MAX_AGG_SIZE_SHIFT);
add_sta_cmd.station_flags |=
}
}
+static void iwl_mvm_cancel_session_protection(struct iwl_mvm *mvm,
+ struct iwl_mvm_vif *mvmvif)
+{
+ struct iwl_mvm_session_prot_cmd cmd = {
+ .id_and_color =
+ cpu_to_le32(FW_CMD_ID_AND_COLOR(mvmvif->id,
+ mvmvif->color)),
+ .action = cpu_to_le32(FW_CTXT_ACTION_REMOVE),
+ .conf_id = cpu_to_le32(mvmvif->time_event_data.id),
+ };
+ int ret;
+
+ ret = iwl_mvm_send_cmd_pdu(mvm, iwl_cmd_id(SESSION_PROTECTION_CMD,
+ MAC_CONF_GROUP, 0),
+ 0, sizeof(cmd), &cmd);
+ if (ret)
+ IWL_ERR(mvm,
+ "Couldn't send the SESSION_PROTECTION_CMD: %d\n", ret);
+}
+
static bool __iwl_mvm_remove_time_event(struct iwl_mvm *mvm,
struct iwl_mvm_time_event_data *te_data,
u32 *uid)
{
u32 id;
+ struct iwl_mvm_vif *mvmvif = iwl_mvm_vif_from_mac80211(te_data->vif);
/*
* It is possible that by the time we got to this point the time
iwl_mvm_te_clear_data(mvm, te_data);
spin_unlock_bh(&mvm->time_event_lock);
- /*
- * It is possible that by the time we try to remove it, the time event
- * has already ended and removed. In such a case there is no need to
- * send a removal command.
+ /* When session protection is supported, the te_data->id field
+ * is reused to save session protection's configuration.
*/
- if (id == TE_MAX) {
- IWL_DEBUG_TE(mvm, "TE 0x%x has already ended\n", *uid);
+ if (fw_has_capa(&mvm->fw->ucode_capa,
+ IWL_UCODE_TLV_CAPA_SESSION_PROT_CMD)) {
+ if (mvmvif && id < SESSION_PROTECT_CONF_MAX_ID) {
+ /* Session protection is still ongoing. Cancel it */
+ iwl_mvm_cancel_session_protection(mvm, mvmvif);
+ if (te_data->vif->type == NL80211_IFTYPE_P2P_DEVICE) {
+ set_bit(IWL_MVM_STATUS_NEED_FLUSH_P2P, &mvm->status);
+ iwl_mvm_roc_finished(mvm);
+ }
+ }
return false;
+ } else {
+ /* It is possible that by the time we try to remove it, the
+ * time event has already ended and removed. In such a case
+ * there is no need to send a removal command.
+ */
+ if (id == TE_MAX) {
+ IWL_DEBUG_TE(mvm, "TE 0x%x has already ended\n", *uid);
+ return false;
+ }
}
return true;
struct iwl_rx_packet *pkt = rxb_addr(rxb);
struct iwl_mvm_session_prot_notif *notif = (void *)pkt->data;
struct ieee80211_vif *vif;
+ struct iwl_mvm_vif *mvmvif;
rcu_read_lock();
vif = iwl_mvm_rcu_dereference_vif_id(mvm, le32_to_cpu(notif->mac_id),
if (!vif)
goto out_unlock;
+ mvmvif = iwl_mvm_vif_from_mac80211(vif);
+
/* The vif is not a P2P_DEVICE, maintain its time_event_data */
if (vif->type != NL80211_IFTYPE_P2P_DEVICE) {
- struct iwl_mvm_vif *mvmvif = iwl_mvm_vif_from_mac80211(vif);
struct iwl_mvm_time_event_data *te_data =
&mvmvif->time_event_data;
if (!le32_to_cpu(notif->status) || !le32_to_cpu(notif->start)) {
/* End TE, notify mac80211 */
+ mvmvif->time_event_data.id = SESSION_PROTECT_CONF_MAX_ID;
ieee80211_remain_on_channel_expired(mvm->hw);
set_bit(IWL_MVM_STATUS_NEED_FLUSH_P2P, &mvm->status);
iwl_mvm_roc_finished(mvm);
} else if (le32_to_cpu(notif->start)) {
+ if (WARN_ON(mvmvif->time_event_data.id !=
+ le32_to_cpu(notif->conf_id)))
+ goto out_unlock;
set_bit(IWL_MVM_STATUS_ROC_RUNNING, &mvm->status);
ieee80211_ready_on_channel(mvm->hw); /* Start TE */
}
lockdep_assert_held(&mvm->mutex);
+ /* The time_event_data.id field is reused to save session
+ * protection's configuration.
+ */
switch (type) {
case IEEE80211_ROC_TYPE_NORMAL:
- cmd.conf_id =
- cpu_to_le32(SESSION_PROTECT_CONF_P2P_DEVICE_DISCOV);
+ mvmvif->time_event_data.id =
+ SESSION_PROTECT_CONF_P2P_DEVICE_DISCOV;
break;
case IEEE80211_ROC_TYPE_MGMT_TX:
- cmd.conf_id =
- cpu_to_le32(SESSION_PROTECT_CONF_P2P_GO_NEGOTIATION);
+ mvmvif->time_event_data.id =
+ SESSION_PROTECT_CONF_P2P_GO_NEGOTIATION;
break;
default:
WARN_ONCE(1, "Got an invalid ROC type\n");
return -EINVAL;
}
+ cmd.conf_id = cpu_to_le32(mvmvif->time_event_data.id);
return iwl_mvm_send_cmd_pdu(mvm, iwl_cmd_id(SESSION_PROTECTION_CMD,
MAC_CONF_GROUP, 0),
0, sizeof(cmd), &cmd);
__iwl_mvm_remove_time_event(mvm, te_data, &uid);
}
-static void iwl_mvm_cancel_session_protection(struct iwl_mvm *mvm,
- struct iwl_mvm_vif *mvmvif)
-{
- struct iwl_mvm_session_prot_cmd cmd = {
- .id_and_color =
- cpu_to_le32(FW_CMD_ID_AND_COLOR(mvmvif->id,
- mvmvif->color)),
- .action = cpu_to_le32(FW_CTXT_ACTION_REMOVE),
- };
- int ret;
-
- ret = iwl_mvm_send_cmd_pdu(mvm, iwl_cmd_id(SESSION_PROTECTION_CMD,
- MAC_CONF_GROUP, 0),
- 0, sizeof(cmd), &cmd);
- if (ret)
- IWL_ERR(mvm,
- "Couldn't send the SESSION_PROTECTION_CMD: %d\n", ret);
-}
-
void iwl_mvm_stop_roc(struct iwl_mvm *mvm, struct ieee80211_vif *vif)
{
struct iwl_mvm_vif *mvmvif;
IWL_UCODE_TLV_CAPA_SESSION_PROT_CMD)) {
mvmvif = iwl_mvm_vif_from_mac80211(vif);
- iwl_mvm_cancel_session_protection(mvm, mvmvif);
-
- if (vif->type == NL80211_IFTYPE_P2P_DEVICE)
+ if (vif->type == NL80211_IFTYPE_P2P_DEVICE) {
+ iwl_mvm_cancel_session_protection(mvm, mvmvif);
set_bit(IWL_MVM_STATUS_NEED_FLUSH_P2P, &mvm->status);
+ } else {
+ iwl_mvm_remove_aux_roc_te(mvm, mvmvif,
+ &mvmvif->time_event_data);
+ }
iwl_mvm_roc_finished(mvm);
cpu_to_le32(FW_CMD_ID_AND_COLOR(mvmvif->id,
mvmvif->color)),
.action = cpu_to_le32(FW_CTXT_ACTION_ADD),
- .conf_id = cpu_to_le32(SESSION_PROTECT_CONF_ASSOC),
.duration_tu = cpu_to_le32(MSEC_TO_TU(duration)),
};
+ /* The time_event_data.id field is reused to save session
+ * protection's configuration.
+ */
+ mvmvif->time_event_data.id = SESSION_PROTECT_CONF_ASSOC;
+ cmd.conf_id = cpu_to_le32(mvmvif->time_event_data.id);
+
lockdep_assert_held(&mvm->mutex);
spin_lock_bh(&mvm->time_event_lock);
iwl_set_bit(trans, CSR_CTXT_INFO_BOOT_CTRL,
CSR_AUTO_FUNC_BOOT_ENA);
+
+ if (trans->trans_cfg->device_family == IWL_DEVICE_FAMILY_AX210) {
+ /*
+ * The firmware initializes this again later (to a smaller
+ * value), but for the boot process initialize the LTR to
+ * ~250 usec.
+ */
+ u32 val = CSR_LTR_LONG_VAL_AD_NO_SNOOP_REQ |
+ u32_encode_bits(CSR_LTR_LONG_VAL_AD_SCALE_USEC,
+ CSR_LTR_LONG_VAL_AD_NO_SNOOP_SCALE) |
+ u32_encode_bits(250,
+ CSR_LTR_LONG_VAL_AD_NO_SNOOP_VAL) |
+ CSR_LTR_LONG_VAL_AD_SNOOP_REQ |
+ u32_encode_bits(CSR_LTR_LONG_VAL_AD_SCALE_USEC,
+ CSR_LTR_LONG_VAL_AD_SNOOP_SCALE) |
+ u32_encode_bits(250, CSR_LTR_LONG_VAL_AD_SNOOP_VAL);
+
+ iwl_write32(trans, CSR_LTR_LONG_VAL_AD, val);
+ }
+
if (trans->trans_cfg->device_family >= IWL_DEVICE_FAMILY_AX210)
iwl_write_umac_prph(trans, UREG_CPU_INIT_RUN, 1);
else
{IWL_PCI_DEVICE(0x2725, 0x0090, iwlax211_2ax_cfg_so_gf_a0)},
{IWL_PCI_DEVICE(0x2725, 0x0020, iwlax210_2ax_cfg_ty_gf_a0)},
+ {IWL_PCI_DEVICE(0x2725, 0x0024, iwlax210_2ax_cfg_ty_gf_a0)},
{IWL_PCI_DEVICE(0x2725, 0x0310, iwlax210_2ax_cfg_ty_gf_a0)},
{IWL_PCI_DEVICE(0x2725, 0x0510, iwlax210_2ax_cfg_ty_gf_a0)},
{IWL_PCI_DEVICE(0x2725, 0x0A10, iwlax210_2ax_cfg_ty_gf_a0)},
+ {IWL_PCI_DEVICE(0x2725, 0xE020, iwlax210_2ax_cfg_ty_gf_a0)},
+ {IWL_PCI_DEVICE(0x2725, 0xE024, iwlax210_2ax_cfg_ty_gf_a0)},
+ {IWL_PCI_DEVICE(0x2725, 0x4020, iwlax210_2ax_cfg_ty_gf_a0)},
+ {IWL_PCI_DEVICE(0x2725, 0x6020, iwlax210_2ax_cfg_ty_gf_a0)},
+ {IWL_PCI_DEVICE(0x2725, 0x6024, iwlax210_2ax_cfg_ty_gf_a0)},
{IWL_PCI_DEVICE(0x2725, 0x00B0, iwlax411_2ax_cfg_sosnj_gf4_a0)},
{IWL_PCI_DEVICE(0x2726, 0x0070, iwlax201_cfg_snj_hr_b0)},
{IWL_PCI_DEVICE(0x2726, 0x0074, iwlax201_cfg_snj_hr_b0)},
void *buf, int dwords)
{
unsigned long flags;
- int offs, ret = 0;
+ int offs = 0;
u32 *vals = buf;
- if (iwl_trans_grab_nic_access(trans, &flags)) {
- iwl_write32(trans, HBUS_TARG_MEM_RADDR, addr);
- for (offs = 0; offs < dwords; offs++)
- vals[offs] = iwl_read32(trans, HBUS_TARG_MEM_RDAT);
- iwl_trans_release_nic_access(trans, &flags);
- } else {
- ret = -EBUSY;
+ while (offs < dwords) {
+ /* limit the time we spin here under lock to 1/2s */
+ ktime_t timeout = ktime_add_us(ktime_get(), 500 * USEC_PER_MSEC);
+
+ if (iwl_trans_grab_nic_access(trans, &flags)) {
+ iwl_write32(trans, HBUS_TARG_MEM_RADDR,
+ addr + 4 * offs);
+
+ while (offs < dwords) {
+ vals[offs] = iwl_read32(trans,
+ HBUS_TARG_MEM_RDAT);
+ offs++;
+
+ /* calling ktime_get is expensive so
+ * do it once in 128 reads
+ */
+ if (offs % 128 == 0 && ktime_after(ktime_get(),
+ timeout))
+ break;
+ }
+ iwl_trans_release_nic_access(trans, &flags);
+ } else {
+ return -EBUSY;
+ }
}
- return ret;
+
+ return 0;
}
static int iwl_trans_pcie_write_mem(struct iwl_trans *trans, u32 addr,
{
int ret;
- mt76_worker_disable(&dev->tx_worker);
-
ret = wait_event_timeout(dev->tx_wait, !mt76_has_tx_pending(&dev->phy),
HZ / 5);
if (!ret) {
usb_kill_urb(q->entry[j].urb);
}
+ mt76_worker_disable(&dev->tx_worker);
+
/* On device removal we maight queue skb's, but mt76u_tx_kick()
* will fail to submit urb, cleanup those skb's manually.
*/
if (!q)
continue;
- entry = q->entry[q->tail];
- q->entry[q->tail].done = false;
-
- mt76_queue_tx_complete(dev, q, &entry);
+ while (q->queued > 0) {
+ entry = q->entry[q->tail];
+ q->entry[q->tail].done = false;
+ mt76_queue_tx_complete(dev, q, &entry);
+ }
}
+
+ mt76_worker_enable(&dev->tx_worker);
}
cancel_work_sync(&dev->usb.stat_work);
clear_bit(MT76_READING_STATS, &dev->phy.state);
- mt76_worker_enable(&dev->tx_worker);
-
mt76_tx_status_check(dev, NULL, true);
}
EXPORT_SYMBOL_GPL(mt76u_stop_tx);
{
int tmp_len;
+ memset(tmp, 0, size);
+
if (count < num)
return -EFAULT;
int rtw_fw_dump_fifo(struct rtw_dev *rtwdev, u8 fifo_sel, u32 addr, u32 size,
u32 *buffer)
{
- if (!rtwdev->chip->fw_fifo_addr) {
+ if (!rtwdev->chip->fw_fifo_addr[0]) {
rtw_dbg(rtwdev, RTW_DBG_FW, "chip not support dump fw fifo\n");
return -ENOTSUPP;
}
return PTR_ERR(phy->gpiod_en);
}
- phy->gpiod_fw = devm_gpiod_get(dev, "firmware", GPIOD_OUT_LOW);
+ phy->gpiod_fw = devm_gpiod_get_optional(dev, "firmware", GPIOD_OUT_LOW);
if (IS_ERR(phy->gpiod_fw)) {
nfc_err(dev, "Failed to get FW gpio\n");
return PTR_ERR(phy->gpiod_fw);
To compile this driver as a module, choose m here. The module will
be called s3fwrn5_i2c.ko.
Say N if unsure.
+
+config NFC_S3FWRN82_UART
+ tristate "Samsung S3FWRN82 UART support"
+ depends on NFC_NCI && SERIAL_DEV_BUS
+ select NFC_S3FWRN5
+ help
+ This module adds support for a UART interface to the S3FWRN82 chip.
+ Select this if your platform is using the UART bus.
+
+ To compile this driver as a module, choose m here. The module will
+ be called s3fwrn82_uart.ko.
+ Say N if unsure.
# Makefile for Samsung S3FWRN5 NFC driver
#
-s3fwrn5-objs = core.o firmware.o nci.o
+s3fwrn5-objs = core.o firmware.o nci.o phy_common.o
s3fwrn5_i2c-objs = i2c.o
+s3fwrn82_uart-objs = uart.o
obj-$(CONFIG_NFC_S3FWRN5) += s3fwrn5.o
obj-$(CONFIG_NFC_S3FWRN5_I2C) += s3fwrn5_i2c.o
+obj-$(CONFIG_NFC_S3FWRN82_UART) += s3fwrn82_uart.o
};
int s3fwrn5_probe(struct nci_dev **ndev, void *phy_id, struct device *pdev,
- const struct s3fwrn5_phy_ops *phy_ops, unsigned int max_payload)
+ const struct s3fwrn5_phy_ops *phy_ops)
{
struct s3fwrn5_info *info;
int ret;
info->phy_id = phy_id;
info->pdev = pdev;
info->phy_ops = phy_ops;
- info->max_payload = max_payload;
mutex_init(&info->mutex);
s3fwrn5_set_mode(info, S3FWRN5_MODE_COLD);
}
/*
- * Firmware header stucture:
+ * Firmware header structure:
*
* 0x00 - 0x0B : Date and time string (w/o NUL termination)
* 0x10 - 0x13 : Firmware version
#include <net/nfc/nfc.h>
-#include "s3fwrn5.h"
+#include "phy_common.h"
#define S3FWRN5_I2C_DRIVER_NAME "s3fwrn5_i2c"
-#define S3FWRN5_I2C_MAX_PAYLOAD 32
-#define S3FWRN5_EN_WAIT_TIME 150
-
struct s3fwrn5_i2c_phy {
+ struct phy_common common;
struct i2c_client *i2c_dev;
- struct nci_dev *ndev;
-
- unsigned int gpio_en;
- unsigned int gpio_fw_wake;
-
- struct mutex mutex;
- enum s3fwrn5_mode mode;
unsigned int irq_skip:1;
};
-static void s3fwrn5_i2c_set_wake(void *phy_id, bool wake)
-{
- struct s3fwrn5_i2c_phy *phy = phy_id;
-
- mutex_lock(&phy->mutex);
- gpio_set_value(phy->gpio_fw_wake, wake);
- msleep(S3FWRN5_EN_WAIT_TIME/2);
- mutex_unlock(&phy->mutex);
-}
-
static void s3fwrn5_i2c_set_mode(void *phy_id, enum s3fwrn5_mode mode)
{
struct s3fwrn5_i2c_phy *phy = phy_id;
- mutex_lock(&phy->mutex);
+ mutex_lock(&phy->common.mutex);
- if (phy->mode == mode)
+ if (s3fwrn5_phy_power_ctrl(&phy->common, mode) == false)
goto out;
- phy->mode = mode;
-
- gpio_set_value(phy->gpio_en, 1);
- gpio_set_value(phy->gpio_fw_wake, 0);
- if (mode == S3FWRN5_MODE_FW)
- gpio_set_value(phy->gpio_fw_wake, 1);
-
- if (mode != S3FWRN5_MODE_COLD) {
- msleep(S3FWRN5_EN_WAIT_TIME);
- gpio_set_value(phy->gpio_en, 0);
- msleep(S3FWRN5_EN_WAIT_TIME/2);
- }
-
phy->irq_skip = true;
out:
- mutex_unlock(&phy->mutex);
-}
-
-static enum s3fwrn5_mode s3fwrn5_i2c_get_mode(void *phy_id)
-{
- struct s3fwrn5_i2c_phy *phy = phy_id;
- enum s3fwrn5_mode mode;
-
- mutex_lock(&phy->mutex);
-
- mode = phy->mode;
-
- mutex_unlock(&phy->mutex);
-
- return mode;
+ mutex_unlock(&phy->common.mutex);
}
static int s3fwrn5_i2c_write(void *phy_id, struct sk_buff *skb)
struct s3fwrn5_i2c_phy *phy = phy_id;
int ret;
- mutex_lock(&phy->mutex);
+ mutex_lock(&phy->common.mutex);
phy->irq_skip = false;
ret = i2c_master_send(phy->i2c_dev, skb->data, skb->len);
}
- mutex_unlock(&phy->mutex);
+ mutex_unlock(&phy->common.mutex);
if (ret < 0)
return ret;
}
static const struct s3fwrn5_phy_ops i2c_phy_ops = {
- .set_wake = s3fwrn5_i2c_set_wake,
+ .set_wake = s3fwrn5_phy_set_wake,
.set_mode = s3fwrn5_i2c_set_mode,
- .get_mode = s3fwrn5_i2c_get_mode,
+ .get_mode = s3fwrn5_phy_get_mode,
.write = s3fwrn5_i2c_write,
};
char hdr[4];
int ret;
- hdr_size = (phy->mode == S3FWRN5_MODE_NCI) ?
+ hdr_size = (phy->common.mode == S3FWRN5_MODE_NCI) ?
NCI_CTRL_HDR_SIZE : S3FWRN5_FW_HDR_SIZE;
ret = i2c_master_recv(phy->i2c_dev, hdr, hdr_size);
if (ret < 0)
if (ret < hdr_size)
return -EBADMSG;
- data_len = (phy->mode == S3FWRN5_MODE_NCI) ?
+ data_len = (phy->common.mode == S3FWRN5_MODE_NCI) ?
((struct nci_ctrl_hdr *)hdr)->plen :
((struct s3fwrn5_fw_header *)hdr)->len;
}
out:
- return s3fwrn5_recv_frame(phy->ndev, skb, phy->mode);
+ return s3fwrn5_recv_frame(phy->common.ndev, skb, phy->common.mode);
}
static irqreturn_t s3fwrn5_i2c_irq_thread_fn(int irq, void *phy_id)
{
struct s3fwrn5_i2c_phy *phy = phy_id;
- if (!phy || !phy->ndev) {
+ if (!phy || !phy->common.ndev) {
WARN_ON_ONCE(1);
return IRQ_NONE;
}
- mutex_lock(&phy->mutex);
+ mutex_lock(&phy->common.mutex);
if (phy->irq_skip)
goto out;
- switch (phy->mode) {
+ switch (phy->common.mode) {
case S3FWRN5_MODE_NCI:
case S3FWRN5_MODE_FW:
s3fwrn5_i2c_read(phy);
}
out:
- mutex_unlock(&phy->mutex);
+ mutex_unlock(&phy->common.mutex);
return IRQ_HANDLED;
}
if (!np)
return -ENODEV;
- phy->gpio_en = of_get_named_gpio(np, "en-gpios", 0);
- if (!gpio_is_valid(phy->gpio_en)) {
+ phy->common.gpio_en = of_get_named_gpio(np, "en-gpios", 0);
+ if (!gpio_is_valid(phy->common.gpio_en)) {
/* Support also deprecated property */
- phy->gpio_en = of_get_named_gpio(np, "s3fwrn5,en-gpios", 0);
- if (!gpio_is_valid(phy->gpio_en))
+ phy->common.gpio_en = of_get_named_gpio(np,
+ "s3fwrn5,en-gpios",
+ 0);
+ if (!gpio_is_valid(phy->common.gpio_en))
return -ENODEV;
}
- phy->gpio_fw_wake = of_get_named_gpio(np, "wake-gpios", 0);
- if (!gpio_is_valid(phy->gpio_fw_wake)) {
+ phy->common.gpio_fw_wake = of_get_named_gpio(np, "wake-gpios", 0);
+ if (!gpio_is_valid(phy->common.gpio_fw_wake)) {
/* Support also deprecated property */
- phy->gpio_fw_wake = of_get_named_gpio(np, "s3fwrn5,fw-gpios", 0);
- if (!gpio_is_valid(phy->gpio_fw_wake))
+ phy->common.gpio_fw_wake = of_get_named_gpio(np,
+ "s3fwrn5,fw-gpios",
+ 0);
+ if (!gpio_is_valid(phy->common.gpio_fw_wake))
return -ENODEV;
}
if (!phy)
return -ENOMEM;
- mutex_init(&phy->mutex);
- phy->mode = S3FWRN5_MODE_COLD;
+ mutex_init(&phy->common.mutex);
+ phy->common.mode = S3FWRN5_MODE_COLD;
phy->irq_skip = true;
phy->i2c_dev = client;
if (ret < 0)
return ret;
- ret = devm_gpio_request_one(&phy->i2c_dev->dev, phy->gpio_en,
- GPIOF_OUT_INIT_HIGH, "s3fwrn5_en");
+ ret = devm_gpio_request_one(&phy->i2c_dev->dev, phy->common.gpio_en,
+ GPIOF_OUT_INIT_HIGH, "s3fwrn5_en");
if (ret < 0)
return ret;
- ret = devm_gpio_request_one(&phy->i2c_dev->dev, phy->gpio_fw_wake,
- GPIOF_OUT_INIT_LOW, "s3fwrn5_fw_wake");
+ ret = devm_gpio_request_one(&phy->i2c_dev->dev,
+ phy->common.gpio_fw_wake,
+ GPIOF_OUT_INIT_LOW, "s3fwrn5_fw_wake");
if (ret < 0)
return ret;
- ret = s3fwrn5_probe(&phy->ndev, phy, &phy->i2c_dev->dev, &i2c_phy_ops,
- S3FWRN5_I2C_MAX_PAYLOAD);
+ ret = s3fwrn5_probe(&phy->common.ndev, phy, &phy->i2c_dev->dev,
+ &i2c_phy_ops);
if (ret < 0)
return ret;
s3fwrn5_i2c_irq_thread_fn, IRQF_TRIGGER_HIGH | IRQF_ONESHOT,
S3FWRN5_I2C_DRIVER_NAME, phy);
if (ret)
- s3fwrn5_remove(phy->ndev);
+ s3fwrn5_remove(phy->common.ndev);
return ret;
}
{
struct s3fwrn5_i2c_phy *phy = i2c_get_clientdata(client);
- s3fwrn5_remove(phy->ndev);
+ s3fwrn5_remove(phy->common.ndev);
return 0;
}
--- /dev/null
+// SPDX-License-Identifier: GPL-2.0-or-later
+/*
+ * Link Layer for Samsung S3FWRN5 NCI based Driver
+ *
+ * Copyright (C) 2015 Samsung Electrnoics
+ * Robert Baldyga <r.baldyga@samsung.com>
+ * Copyright (C) 2020 Samsung Electrnoics
+ * Bongsu Jeon <bongsu.jeon@samsung.com>
+ */
+
+#include <linux/gpio.h>
+#include <linux/delay.h>
+#include <linux/module.h>
+
+#include "phy_common.h"
+
+void s3fwrn5_phy_set_wake(void *phy_id, bool wake)
+{
+ struct phy_common *phy = phy_id;
+
+ mutex_lock(&phy->mutex);
+ gpio_set_value(phy->gpio_fw_wake, wake);
+ msleep(S3FWRN5_EN_WAIT_TIME);
+ mutex_unlock(&phy->mutex);
+}
+EXPORT_SYMBOL(s3fwrn5_phy_set_wake);
+
+bool s3fwrn5_phy_power_ctrl(struct phy_common *phy, enum s3fwrn5_mode mode)
+{
+ if (phy->mode == mode)
+ return false;
+
+ phy->mode = mode;
+
+ gpio_set_value(phy->gpio_en, 1);
+ gpio_set_value(phy->gpio_fw_wake, 0);
+ if (mode == S3FWRN5_MODE_FW)
+ gpio_set_value(phy->gpio_fw_wake, 1);
+
+ if (mode != S3FWRN5_MODE_COLD) {
+ msleep(S3FWRN5_EN_WAIT_TIME);
+ gpio_set_value(phy->gpio_en, 0);
+ msleep(S3FWRN5_EN_WAIT_TIME);
+ }
+
+ return true;
+}
+EXPORT_SYMBOL(s3fwrn5_phy_power_ctrl);
+
+void s3fwrn5_phy_set_mode(void *phy_id, enum s3fwrn5_mode mode)
+{
+ struct phy_common *phy = phy_id;
+
+ mutex_lock(&phy->mutex);
+
+ s3fwrn5_phy_power_ctrl(phy, mode);
+
+ mutex_unlock(&phy->mutex);
+}
+EXPORT_SYMBOL(s3fwrn5_phy_set_mode);
+
+enum s3fwrn5_mode s3fwrn5_phy_get_mode(void *phy_id)
+{
+ struct phy_common *phy = phy_id;
+ enum s3fwrn5_mode mode;
+
+ mutex_lock(&phy->mutex);
+
+ mode = phy->mode;
+
+ mutex_unlock(&phy->mutex);
+
+ return mode;
+}
+EXPORT_SYMBOL(s3fwrn5_phy_get_mode);
--- /dev/null
+/* SPDX-License-Identifier: GPL-2.0-or-later
+ *
+ * Link Layer for Samsung S3FWRN5 NCI based Driver
+ *
+ * Copyright (C) 2015 Samsung Electrnoics
+ * Robert Baldyga <r.baldyga@samsung.com>
+ * Copyright (C) 2020 Samsung Electrnoics
+ * Bongsu Jeon <bongsu.jeon@samsung.com>
+ */
+
+#ifndef __NFC_S3FWRN5_PHY_COMMON_H
+#define __NFC_S3FWRN5_PHY_COMMON_H
+
+#include <linux/mutex.h>
+#include <net/nfc/nci_core.h>
+
+#include "s3fwrn5.h"
+
+#define S3FWRN5_EN_WAIT_TIME 20
+
+struct phy_common {
+ struct nci_dev *ndev;
+
+ int gpio_en;
+ int gpio_fw_wake;
+
+ struct mutex mutex;
+
+ enum s3fwrn5_mode mode;
+};
+
+void s3fwrn5_phy_set_wake(void *phy_id, bool wake);
+bool s3fwrn5_phy_power_ctrl(struct phy_common *phy, enum s3fwrn5_mode mode);
+void s3fwrn5_phy_set_mode(void *phy_id, enum s3fwrn5_mode mode);
+enum s3fwrn5_mode s3fwrn5_phy_get_mode(void *phy_id);
+
+#endif /* __NFC_S3FWRN5_PHY_COMMON_H */
struct device *pdev;
const struct s3fwrn5_phy_ops *phy_ops;
- unsigned int max_payload;
struct s3fwrn5_fw_info fw_info;
enum s3fwrn5_mode mode)
{
if (!info->phy_ops->set_mode)
- return -ENOTSUPP;
+ return -EOPNOTSUPP;
info->phy_ops->set_mode(info->phy_id, mode);
static inline enum s3fwrn5_mode s3fwrn5_get_mode(struct s3fwrn5_info *info)
{
if (!info->phy_ops->get_mode)
- return -ENOTSUPP;
+ return -EOPNOTSUPP;
return info->phy_ops->get_mode(info->phy_id);
}
static inline int s3fwrn5_set_wake(struct s3fwrn5_info *info, bool wake)
{
if (!info->phy_ops->set_wake)
- return -ENOTSUPP;
+ return -EOPNOTSUPP;
info->phy_ops->set_wake(info->phy_id, wake);
static inline int s3fwrn5_write(struct s3fwrn5_info *info, struct sk_buff *skb)
{
if (!info->phy_ops->write)
- return -ENOTSUPP;
+ return -EOPNOTSUPP;
return info->phy_ops->write(info->phy_id, skb);
}
int s3fwrn5_probe(struct nci_dev **ndev, void *phy_id, struct device *pdev,
- const struct s3fwrn5_phy_ops *phy_ops, unsigned int max_payload);
+ const struct s3fwrn5_phy_ops *phy_ops);
void s3fwrn5_remove(struct nci_dev *ndev);
int s3fwrn5_recv_frame(struct nci_dev *ndev, struct sk_buff *skb,
--- /dev/null
+// SPDX-License-Identifier: GPL-2.0+
+/*
+ * UART Link Layer for S3FWRN82 NCI based Driver
+ *
+ * Copyright (C) 2015 Samsung Electronics
+ * Robert Baldyga <r.baldyga@samsung.com>
+ * Copyright (C) 2020 Samsung Electronics
+ * Bongsu Jeon <bongsu.jeon@samsung.com>
+ */
+
+#include <linux/device.h>
+#include <linux/kernel.h>
+#include <linux/module.h>
+#include <linux/nfc.h>
+#include <linux/netdevice.h>
+#include <linux/of.h>
+#include <linux/serdev.h>
+#include <linux/gpio.h>
+#include <linux/of_gpio.h>
+
+#include "phy_common.h"
+
+#define S3FWRN82_NCI_HEADER 3
+#define S3FWRN82_NCI_IDX 2
+#define NCI_SKB_BUFF_LEN 258
+
+struct s3fwrn82_uart_phy {
+ struct phy_common common;
+ struct serdev_device *ser_dev;
+ struct sk_buff *recv_skb;
+};
+
+static int s3fwrn82_uart_write(void *phy_id, struct sk_buff *out)
+{
+ struct s3fwrn82_uart_phy *phy = phy_id;
+ int err;
+
+ err = serdev_device_write(phy->ser_dev,
+ out->data, out->len,
+ MAX_SCHEDULE_TIMEOUT);
+ if (err < 0)
+ return err;
+
+ return 0;
+}
+
+static const struct s3fwrn5_phy_ops uart_phy_ops = {
+ .set_wake = s3fwrn5_phy_set_wake,
+ .set_mode = s3fwrn5_phy_set_mode,
+ .get_mode = s3fwrn5_phy_get_mode,
+ .write = s3fwrn82_uart_write,
+};
+
+static int s3fwrn82_uart_read(struct serdev_device *serdev,
+ const unsigned char *data,
+ size_t count)
+{
+ struct s3fwrn82_uart_phy *phy = serdev_device_get_drvdata(serdev);
+ size_t i;
+
+ for (i = 0; i < count; i++) {
+ skb_put_u8(phy->recv_skb, *data++);
+
+ if (phy->recv_skb->len < S3FWRN82_NCI_HEADER)
+ continue;
+
+ if ((phy->recv_skb->len - S3FWRN82_NCI_HEADER)
+ < phy->recv_skb->data[S3FWRN82_NCI_IDX])
+ continue;
+
+ s3fwrn5_recv_frame(phy->common.ndev, phy->recv_skb,
+ phy->common.mode);
+ phy->recv_skb = alloc_skb(NCI_SKB_BUFF_LEN, GFP_KERNEL);
+ if (!phy->recv_skb)
+ return 0;
+ }
+
+ return i;
+}
+
+static const struct serdev_device_ops s3fwrn82_serdev_ops = {
+ .receive_buf = s3fwrn82_uart_read,
+ .write_wakeup = serdev_device_write_wakeup,
+};
+
+static const struct of_device_id s3fwrn82_uart_of_match[] = {
+ { .compatible = "samsung,s3fwrn82", },
+ {},
+};
+MODULE_DEVICE_TABLE(of, s3fwrn82_uart_of_match);
+
+static int s3fwrn82_uart_parse_dt(struct serdev_device *serdev)
+{
+ struct s3fwrn82_uart_phy *phy = serdev_device_get_drvdata(serdev);
+ struct device_node *np = serdev->dev.of_node;
+
+ if (!np)
+ return -ENODEV;
+
+ phy->common.gpio_en = of_get_named_gpio(np, "en-gpios", 0);
+ if (!gpio_is_valid(phy->common.gpio_en))
+ return -ENODEV;
+
+ phy->common.gpio_fw_wake = of_get_named_gpio(np, "wake-gpios", 0);
+ if (!gpio_is_valid(phy->common.gpio_fw_wake))
+ return -ENODEV;
+
+ return 0;
+}
+
+static int s3fwrn82_uart_probe(struct serdev_device *serdev)
+{
+ struct s3fwrn82_uart_phy *phy;
+ int ret = -ENOMEM;
+
+ phy = devm_kzalloc(&serdev->dev, sizeof(*phy), GFP_KERNEL);
+ if (!phy)
+ goto err_exit;
+
+ phy->recv_skb = alloc_skb(NCI_SKB_BUFF_LEN, GFP_KERNEL);
+ if (!phy->recv_skb)
+ goto err_exit;
+
+ mutex_init(&phy->common.mutex);
+ phy->common.mode = S3FWRN5_MODE_COLD;
+
+ phy->ser_dev = serdev;
+ serdev_device_set_drvdata(serdev, phy);
+ serdev_device_set_client_ops(serdev, &s3fwrn82_serdev_ops);
+ ret = serdev_device_open(serdev);
+ if (ret) {
+ dev_err(&serdev->dev, "Unable to open device\n");
+ goto err_skb;
+ }
+
+ ret = serdev_device_set_baudrate(serdev, 115200);
+ if (ret != 115200) {
+ ret = -EINVAL;
+ goto err_serdev;
+ }
+
+ serdev_device_set_flow_control(serdev, false);
+
+ ret = s3fwrn82_uart_parse_dt(serdev);
+ if (ret < 0)
+ goto err_serdev;
+
+ ret = devm_gpio_request_one(&phy->ser_dev->dev, phy->common.gpio_en,
+ GPIOF_OUT_INIT_HIGH, "s3fwrn82_en");
+ if (ret < 0)
+ goto err_serdev;
+
+ ret = devm_gpio_request_one(&phy->ser_dev->dev,
+ phy->common.gpio_fw_wake,
+ GPIOF_OUT_INIT_LOW, "s3fwrn82_fw_wake");
+ if (ret < 0)
+ goto err_serdev;
+
+ ret = s3fwrn5_probe(&phy->common.ndev, phy, &phy->ser_dev->dev,
+ &uart_phy_ops);
+ if (ret < 0)
+ goto err_serdev;
+
+ return ret;
+
+err_serdev:
+ serdev_device_close(serdev);
+err_skb:
+ kfree_skb(phy->recv_skb);
+err_exit:
+ return ret;
+}
+
+static void s3fwrn82_uart_remove(struct serdev_device *serdev)
+{
+ struct s3fwrn82_uart_phy *phy = serdev_device_get_drvdata(serdev);
+
+ s3fwrn5_remove(phy->common.ndev);
+ serdev_device_close(serdev);
+ kfree_skb(phy->recv_skb);
+}
+
+static struct serdev_device_driver s3fwrn82_uart_driver = {
+ .probe = s3fwrn82_uart_probe,
+ .remove = s3fwrn82_uart_remove,
+ .driver = {
+ .name = "s3fwrn82_uart",
+ .of_match_table = s3fwrn82_uart_of_match,
+ },
+};
+
+module_serdev_device_driver(s3fwrn82_uart_driver);
+
+MODULE_LICENSE("GPL");
+MODULE_DESCRIPTION("UART driver for Samsung NFC");
+MODULE_AUTHOR("Bongsu Jeon <bongsu.jeon@samsung.com>");
if (id->nsattr & NVME_NS_ATTR_RO)
set_disk_ro(disk, true);
- else
- set_disk_ro(disk, false);
}
static inline bool nvme_first_scan(struct gendisk *disk)
static int nvme_get_effects_log(struct nvme_ctrl *ctrl, u8 csi,
struct nvme_effects_log **log)
{
- struct nvme_cel *cel = xa_load(&ctrl->cels, csi);
+ struct nvme_effects_log *cel = xa_load(&ctrl->cels, csi);
int ret;
if (cel)
return -ENOMEM;
ret = nvme_get_log(ctrl, 0x00, NVME_LOG_CMD_EFFECTS, 0, csi,
- &cel->log, sizeof(cel->log), 0);
+ cel, sizeof(*cel), 0);
if (ret) {
kfree(cel);
return ret;
}
- cel->csi = csi;
- xa_store(&ctrl->cels, cel->csi, cel, GFP_KERNEL);
+ xa_store(&ctrl->cels, csi, cel, GFP_KERNEL);
out:
- *log = &cel->log;
+ *log = cel;
return 0;
}
}
EXPORT_SYMBOL_GPL(nvme_uninit_ctrl);
+static void nvme_free_cels(struct nvme_ctrl *ctrl)
+{
+ struct nvme_effects_log *cel;
+ unsigned long i;
+
+ xa_for_each (&ctrl->cels, i, cel) {
+ xa_erase(&ctrl->cels, i);
+ kfree(cel);
+ }
+
+ xa_destroy(&ctrl->cels);
+}
+
static void nvme_free_ctrl(struct device *dev)
{
struct nvme_ctrl *ctrl =
if (!subsys || ctrl->instance != subsys->instance)
ida_simple_remove(&nvme_instance_ida, ctrl->instance);
- xa_destroy(&ctrl->cels);
-
+ nvme_free_cels(ctrl);
nvme_mpath_uninit(ctrl);
__free_page(ctrl->discard_page);
#endif
};
-struct nvme_cel {
- struct list_head entry;
- struct nvme_effects_log log;
- u8 csi;
-};
-
struct nvme_ctrl {
bool comp_seen;
enum nvme_ctrl_state state;
nvmeq->dbbuf_cq_ei = &dev->dbbuf_eis[cq_idx(qid, dev->db_stride)];
}
+static void nvme_dbbuf_free(struct nvme_queue *nvmeq)
+{
+ if (!nvmeq->qid)
+ return;
+
+ nvmeq->dbbuf_sq_db = NULL;
+ nvmeq->dbbuf_cq_db = NULL;
+ nvmeq->dbbuf_sq_ei = NULL;
+ nvmeq->dbbuf_cq_ei = NULL;
+}
+
static void nvme_dbbuf_set(struct nvme_dev *dev)
{
struct nvme_command c;
+ unsigned int i;
if (!dev->dbbuf_dbs)
return;
dev_warn(dev->ctrl.device, "unable to set dbbuf\n");
/* Free memory and continue on */
nvme_dbbuf_dma_free(dev);
+
+ for (i = 1; i <= dev->online_queues; i++)
+ nvme_dbbuf_free(&dev->queues[i]);
}
}
*/
bool of_dma_is_coherent(struct device_node *np)
{
- struct device_node *node = of_node_get(np);
+ struct device_node *node;
if (IS_ENABLED(CONFIG_OF_DMA_DEFAULT_COHERENT))
return true;
+ node = of_node_get(np);
+
while (node) {
if (of_property_read_bool(node, "dma-coherent")) {
of_node_put(node);
reg |= params->mode << USB_PHY_UTMI_CTL_1_PHY_MODE_SHIFT;
brcm_usb_writel(reg, usb_phy + USB_PHY_UTMI_CTL_1);
- /* Fix the incorrect default */
- reg = brcm_usb_readl(ctrl + USB_CTRL_SETUP);
- reg &= ~USB_CTRL_SETUP_tca_drv_sel_MASK;
- brcm_usb_writel(reg, ctrl + USB_CTRL_SETUP);
-
usb_init_common(params);
/*
#
config PHY_INTEL_KEEMBAY_EMMC
tristate "Intel Keem Bay EMMC PHY driver"
- depends on (OF && ARM64) || COMPILE_TEST
+ depends on ARCH_KEEMBAY || COMPILE_TEST
depends on HAS_IOMEM
select GENERIC_PHY
select REGMAP_MMIO
it supports multiple usb2.0, usb3.0 ports, PCIe and
SATA, and meanwhile supports two version T-PHY which have
different banks layout, the T-PHY with shared banks between
- multi-ports is first version, otherwise is second veriosn,
+ multi-ports is first version, otherwise is second version,
so you can easily distinguish them by banks layout.
config PHY_MTK_UFS
error = devm_request_threaded_irq(ddata->dev, irq, NULL,
cpcap_phy_irq_thread,
- IRQF_SHARED,
+ IRQF_SHARED |
+ IRQF_ONESHOT,
name, ddata);
if (error) {
dev_err(ddata->dev, "could not get irq %s: %i\n",
config PHY_QCOM_USB_HS_28NM
tristate "Qualcomm 28nm High-Speed PHY"
- depends on ARCH_QCOM || COMPILE_TEST
+ depends on OF && (ARCH_QCOM || COMPILE_TEST)
depends on EXTCON || !EXTCON # if EXTCON=m, this cannot be built-in
select GENERIC_PHY
help
config PHY_QCOM_USB_SS
tristate "Qualcomm USB Super-Speed PHY driver"
- depends on ARCH_QCOM || COMPILE_TEST
+ depends on OF && (ARCH_QCOM || COMPILE_TEST)
depends on EXTCON || !EXTCON # if EXTCON=m, this cannot be built-in
select GENERIC_PHY
help
struct phy_provider *phy_provider;
void __iomem *serdes;
void __iomem *usb_serdes;
- void __iomem *dp_serdes;
+ void __iomem *dp_serdes = NULL;
const struct qmp_phy_combo_cfg *combo_cfg = NULL;
const struct qmp_phy_cfg *cfg = NULL;
const struct qmp_phy_cfg *usb_cfg = NULL;
reset:
reset_control_assert(padctl->rst);
remove:
+ platform_set_drvdata(pdev, NULL);
soc->ops->remove(padctl);
return err;
}
static bool aspeed_expr_is_gpio(const struct aspeed_sig_expr *expr)
{
/*
- * The signal type is GPIO if the signal name has "GPIO" as a prefix.
+ * The signal type is GPIO if the signal name has "GPI" as a prefix.
* strncmp (rather than strcmp) is used to implement the prefix
* requirement.
*
- * expr->signal might look like "GPIOT3" in the GPIO case.
+ * expr->signal might look like "GPIOB1" in the GPIO case.
+ * expr->signal might look like "GPIT0" in the GPI case.
*/
- return strncmp(expr->signal, "GPIO", 4) == 0;
+ return strncmp(expr->signal, "GPI", 3) == 0;
}
static bool aspeed_gpio_in_exprs(const struct aspeed_sig_expr **exprs)
#define PADCFG1_TERM_UP BIT(13)
#define PADCFG1_TERM_SHIFT 10
#define PADCFG1_TERM_MASK GENMASK(12, 10)
-#define PADCFG1_TERM_20K 4
-#define PADCFG1_TERM_2K 3
-#define PADCFG1_TERM_5K 2
-#define PADCFG1_TERM_1K 1
+#define PADCFG1_TERM_20K BIT(2)
+#define PADCFG1_TERM_5K BIT(1)
+#define PADCFG1_TERM_1K BIT(0)
+#define PADCFG1_TERM_833 (BIT(1) | BIT(0))
#define PADCFG2 0x008
#define PADCFG2_DEBEN BIT(0)
return -EINVAL;
switch (term) {
+ case PADCFG1_TERM_833:
+ *arg = 833;
+ break;
case PADCFG1_TERM_1K:
*arg = 1000;
break;
- case PADCFG1_TERM_2K:
- *arg = 2000;
- break;
case PADCFG1_TERM_5K:
*arg = 5000;
break;
return -EINVAL;
switch (term) {
+ case PADCFG1_TERM_833:
+ if (!(community->features & PINCTRL_FEATURE_1K_PD))
+ return -EINVAL;
+ *arg = 833;
+ break;
case PADCFG1_TERM_1K:
if (!(community->features & PINCTRL_FEATURE_1K_PD))
return -EINVAL;
value |= PADCFG1_TERM_UP;
+ /* Set default strength value in case none is given */
+ if (arg == 1)
+ arg = 5000;
+
switch (arg) {
case 20000:
value |= PADCFG1_TERM_20K << PADCFG1_TERM_SHIFT;
case 5000:
value |= PADCFG1_TERM_5K << PADCFG1_TERM_SHIFT;
break;
- case 2000:
- value |= PADCFG1_TERM_2K << PADCFG1_TERM_SHIFT;
- break;
case 1000:
value |= PADCFG1_TERM_1K << PADCFG1_TERM_SHIFT;
break;
+ case 833:
+ value |= PADCFG1_TERM_833 << PADCFG1_TERM_SHIFT;
+ break;
default:
ret = -EINVAL;
}
case PIN_CONFIG_BIAS_PULL_DOWN:
value &= ~(PADCFG1_TERM_UP | PADCFG1_TERM_MASK);
+ /* Set default strength value in case none is given */
+ if (arg == 1)
+ arg = 5000;
+
switch (arg) {
case 20000:
value |= PADCFG1_TERM_20K << PADCFG1_TERM_SHIFT;
}
value |= PADCFG1_TERM_1K << PADCFG1_TERM_SHIFT;
break;
+ case 833:
+ if (!(community->features & PINCTRL_FEATURE_1K_PD)) {
+ ret = -EINVAL;
+ break;
+ }
+ value |= PADCFG1_TERM_833 << PADCFG1_TERM_SHIFT;
+ break;
default:
ret = -EINVAL;
}
pin_reg |= BIT(DB_TMR_OUT_UNIT_OFF);
pin_reg &= ~BIT(DB_TMR_LARGE_OFF);
} else if (debounce < 250000) {
- time = debounce / 15600;
+ time = debounce / 15625;
pin_reg |= time & DB_TMR_OUT_MASK;
pin_reg &= ~BIT(DB_TMR_OUT_UNIT_OFF);
pin_reg |= BIT(DB_TMR_LARGE_OFF);
pin_reg |= BIT(DB_TMR_OUT_UNIT_OFF);
pin_reg |= BIT(DB_TMR_LARGE_OFF);
} else {
- pin_reg &= ~DB_CNTRl_MASK;
+ pin_reg &= ~(DB_CNTRl_MASK << DB_CNTRL_OFF);
ret = -EINVAL;
}
} else {
pin_reg &= ~BIT(DB_TMR_OUT_UNIT_OFF);
pin_reg &= ~BIT(DB_TMR_LARGE_OFF);
pin_reg &= ~DB_TMR_OUT_MASK;
- pin_reg &= ~DB_CNTRl_MASK;
+ pin_reg &= ~(DB_CNTRl_MASK << DB_CNTRL_OFF);
}
writel(pin_reg, gpio_dev->base + offset * 4);
raw_spin_unlock_irqrestore(&gpio_dev->lock, flags);
static int jz4770_uart3_hwflow_pins[] = { 0x88, 0x89, };
static int jz4770_ssi0_dt_a_pins[] = { 0x15, };
static int jz4770_ssi0_dt_b_pins[] = { 0x35, };
-static int jz4770_ssi0_dt_d_pins[] = { 0x55, };
-static int jz4770_ssi0_dt_e_pins[] = { 0x71, };
+static int jz4770_ssi0_dt_d_pins[] = { 0x75, };
+static int jz4770_ssi0_dt_e_pins[] = { 0x91, };
static int jz4770_ssi0_dr_a_pins[] = { 0x14, };
static int jz4770_ssi0_dr_b_pins[] = { 0x34, };
-static int jz4770_ssi0_dr_d_pins[] = { 0x54, };
-static int jz4770_ssi0_dr_e_pins[] = { 0x6e, };
+static int jz4770_ssi0_dr_d_pins[] = { 0x74, };
+static int jz4770_ssi0_dr_e_pins[] = { 0x8e, };
static int jz4770_ssi0_clk_a_pins[] = { 0x12, };
static int jz4770_ssi0_clk_b_pins[] = { 0x3c, };
-static int jz4770_ssi0_clk_d_pins[] = { 0x58, };
-static int jz4770_ssi0_clk_e_pins[] = { 0x6f, };
+static int jz4770_ssi0_clk_d_pins[] = { 0x78, };
+static int jz4770_ssi0_clk_e_pins[] = { 0x8f, };
static int jz4770_ssi0_gpc_b_pins[] = { 0x3e, };
-static int jz4770_ssi0_gpc_d_pins[] = { 0x56, };
-static int jz4770_ssi0_gpc_e_pins[] = { 0x73, };
+static int jz4770_ssi0_gpc_d_pins[] = { 0x76, };
+static int jz4770_ssi0_gpc_e_pins[] = { 0x93, };
static int jz4770_ssi0_ce0_a_pins[] = { 0x13, };
static int jz4770_ssi0_ce0_b_pins[] = { 0x3d, };
-static int jz4770_ssi0_ce0_d_pins[] = { 0x59, };
-static int jz4770_ssi0_ce0_e_pins[] = { 0x70, };
+static int jz4770_ssi0_ce0_d_pins[] = { 0x79, };
+static int jz4770_ssi0_ce0_e_pins[] = { 0x90, };
static int jz4770_ssi0_ce1_b_pins[] = { 0x3f, };
-static int jz4770_ssi0_ce1_d_pins[] = { 0x57, };
-static int jz4770_ssi0_ce1_e_pins[] = { 0x72, };
+static int jz4770_ssi0_ce1_d_pins[] = { 0x77, };
+static int jz4770_ssi0_ce1_e_pins[] = { 0x92, };
static int jz4770_ssi1_dt_b_pins[] = { 0x35, };
-static int jz4770_ssi1_dt_d_pins[] = { 0x55, };
-static int jz4770_ssi1_dt_e_pins[] = { 0x71, };
+static int jz4770_ssi1_dt_d_pins[] = { 0x75, };
+static int jz4770_ssi1_dt_e_pins[] = { 0x91, };
static int jz4770_ssi1_dr_b_pins[] = { 0x34, };
-static int jz4770_ssi1_dr_d_pins[] = { 0x54, };
-static int jz4770_ssi1_dr_e_pins[] = { 0x6e, };
+static int jz4770_ssi1_dr_d_pins[] = { 0x74, };
+static int jz4770_ssi1_dr_e_pins[] = { 0x8e, };
static int jz4770_ssi1_clk_b_pins[] = { 0x3c, };
-static int jz4770_ssi1_clk_d_pins[] = { 0x58, };
-static int jz4770_ssi1_clk_e_pins[] = { 0x6f, };
+static int jz4770_ssi1_clk_d_pins[] = { 0x78, };
+static int jz4770_ssi1_clk_e_pins[] = { 0x8f, };
static int jz4770_ssi1_gpc_b_pins[] = { 0x3e, };
-static int jz4770_ssi1_gpc_d_pins[] = { 0x56, };
-static int jz4770_ssi1_gpc_e_pins[] = { 0x73, };
+static int jz4770_ssi1_gpc_d_pins[] = { 0x76, };
+static int jz4770_ssi1_gpc_e_pins[] = { 0x93, };
static int jz4770_ssi1_ce0_b_pins[] = { 0x3d, };
-static int jz4770_ssi1_ce0_d_pins[] = { 0x59, };
-static int jz4770_ssi1_ce0_e_pins[] = { 0x70, };
+static int jz4770_ssi1_ce0_d_pins[] = { 0x79, };
+static int jz4770_ssi1_ce0_e_pins[] = { 0x90, };
static int jz4770_ssi1_ce1_b_pins[] = { 0x3f, };
-static int jz4770_ssi1_ce1_d_pins[] = { 0x57, };
-static int jz4770_ssi1_ce1_e_pins[] = { 0x72, };
+static int jz4770_ssi1_ce1_d_pins[] = { 0x77, };
+static int jz4770_ssi1_ce1_e_pins[] = { 0x92, };
static int jz4770_mmc0_1bit_a_pins[] = { 0x12, 0x13, 0x14, };
static int jz4770_mmc0_4bit_a_pins[] = { 0x15, 0x16, 0x17, };
static int jz4770_mmc0_1bit_e_pins[] = { 0x9c, 0x9d, 0x94, };
static int jz4780_ssi0_dt_a_21_pins[] = { 0x15, };
static int jz4780_ssi0_dt_a_28_pins[] = { 0x1c, };
static int jz4780_ssi0_dt_b_pins[] = { 0x3d, };
-static int jz4780_ssi0_dt_d_pins[] = { 0x59, };
+static int jz4780_ssi0_dt_d_pins[] = { 0x79, };
static int jz4780_ssi0_dr_a_20_pins[] = { 0x14, };
static int jz4780_ssi0_dr_a_27_pins[] = { 0x1b, };
static int jz4780_ssi0_dr_b_pins[] = { 0x34, };
-static int jz4780_ssi0_dr_d_pins[] = { 0x54, };
+static int jz4780_ssi0_dr_d_pins[] = { 0x74, };
static int jz4780_ssi0_clk_a_pins[] = { 0x12, };
static int jz4780_ssi0_clk_b_5_pins[] = { 0x25, };
static int jz4780_ssi0_clk_b_28_pins[] = { 0x3c, };
-static int jz4780_ssi0_clk_d_pins[] = { 0x58, };
+static int jz4780_ssi0_clk_d_pins[] = { 0x78, };
static int jz4780_ssi0_gpc_b_pins[] = { 0x3e, };
-static int jz4780_ssi0_gpc_d_pins[] = { 0x56, };
+static int jz4780_ssi0_gpc_d_pins[] = { 0x76, };
static int jz4780_ssi0_ce0_a_23_pins[] = { 0x17, };
static int jz4780_ssi0_ce0_a_25_pins[] = { 0x19, };
static int jz4780_ssi0_ce0_b_pins[] = { 0x3f, };
-static int jz4780_ssi0_ce0_d_pins[] = { 0x57, };
+static int jz4780_ssi0_ce0_d_pins[] = { 0x77, };
static int jz4780_ssi0_ce1_b_pins[] = { 0x35, };
-static int jz4780_ssi0_ce1_d_pins[] = { 0x55, };
+static int jz4780_ssi0_ce1_d_pins[] = { 0x75, };
static int jz4780_ssi1_dt_b_pins[] = { 0x3d, };
-static int jz4780_ssi1_dt_d_pins[] = { 0x59, };
+static int jz4780_ssi1_dt_d_pins[] = { 0x79, };
static int jz4780_ssi1_dr_b_pins[] = { 0x34, };
-static int jz4780_ssi1_dr_d_pins[] = { 0x54, };
+static int jz4780_ssi1_dr_d_pins[] = { 0x74, };
static int jz4780_ssi1_clk_b_pins[] = { 0x3c, };
-static int jz4780_ssi1_clk_d_pins[] = { 0x58, };
+static int jz4780_ssi1_clk_d_pins[] = { 0x78, };
static int jz4780_ssi1_gpc_b_pins[] = { 0x3e, };
-static int jz4780_ssi1_gpc_d_pins[] = { 0x56, };
+static int jz4780_ssi1_gpc_d_pins[] = { 0x76, };
static int jz4780_ssi1_ce0_b_pins[] = { 0x3f, };
-static int jz4780_ssi1_ce0_d_pins[] = { 0x57, };
+static int jz4780_ssi1_ce0_d_pins[] = { 0x77, };
static int jz4780_ssi1_ce1_b_pins[] = { 0x35, };
-static int jz4780_ssi1_ce1_d_pins[] = { 0x55, };
+static int jz4780_ssi1_ce1_d_pins[] = { 0x75, };
static int jz4780_mmc0_8bit_a_pins[] = { 0x04, 0x05, 0x06, 0x07, 0x18, };
static int jz4780_i2c3_pins[] = { 0x6a, 0x6b, };
static int jz4780_i2c4_e_pins[] = { 0x8c, 0x8d, };
return -EINVAL;
}
- copy = devm_kmemdup(dev, &config, sizeof(config), GFP_KERNEL);
+ copy = devm_kmemdup(dev, config, sizeof(*config), GFP_KERNEL);
if (!copy)
return -ENOMEM;
copy->name = name;
mcp->regmap = devm_regmap_init(dev, &mcp23sxx_spi_regmap, mcp, copy);
+ if (IS_ERR(mcp->regmap))
+ dev_err(dev, "regmap init failed for %s\n", mcp->chip.label);
return PTR_ERR_OR_ZERO(mcp->regmap);
}
if (!bank->domain)
return -ENXIO;
+ clk_enable(bank->clk);
virq = irq_create_mapping(bank->domain, offset);
+ clk_disable(bank->clk);
return (virq) ? : -ENXIO;
}
irq = __ffs(pend);
pend &= ~BIT(irq);
- virq = irq_linear_revmap(bank->domain, irq);
+ virq = irq_find_mapping(bank->domain, irq);
if (!virq) {
dev_err(bank->drvdata->dev, "unmapped irq %d\n", irq);
unsigned int clr = IRQ_NOREQUEST | IRQ_NOPROBE | IRQ_NOAUTOEN;
struct irq_chip_generic *gc;
int ret;
- int i, j;
+ int i;
for (i = 0; i < ctrl->nr_banks; ++i, ++bank) {
if (!bank->valid) {
ret = irq_alloc_domain_generic_chips(bank->domain, 32, 1,
"rockchip_gpio_irq", handle_level_irq,
- clr, 0, IRQ_GC_INIT_MASK_CACHE);
+ clr, 0, 0);
if (ret) {
dev_err(&pdev->dev, "could not alloc generic chips for bank %s\n",
bank->name);
continue;
}
- /*
- * Linux assumes that all interrupts start out disabled/masked.
- * Our driver only uses the concept of masked and always keeps
- * things enabled, so for us that's all masked and all enabled.
- */
- writel_relaxed(0xffffffff, bank->reg_base + GPIO_INTMASK);
- writel_relaxed(0xffffffff, bank->reg_base + GPIO_INTEN);
-
gc = irq_get_domain_generic_chip(bank->domain, 0);
gc->reg_base = bank->reg_base;
gc->private = bank;
gc->chip_types[0].chip.irq_set_type = rockchip_irq_set_type;
gc->wake_enabled = IRQ_MSK(bank->nr_pins);
+ /*
+ * Linux assumes that all interrupts start out disabled/masked.
+ * Our driver only uses the concept of masked and always keeps
+ * things enabled, so for us that's all masked and all enabled.
+ */
+ writel_relaxed(0xffffffff, bank->reg_base + GPIO_INTMASK);
+ writel_relaxed(0xffffffff, bank->reg_base + GPIO_INTEN);
+ gc->mask_cache = 0xffffffff;
+
irq_set_chained_handler_and_data(bank->irq,
rockchip_irq_demux, bank);
-
- /* map the gpio irqs here, when the clock is still running */
- for (j = 0 ; j < 32 ; j++)
- irq_create_mapping(bank->domain, j);
-
clk_disable(bank->clk);
}
static void msm_gpio_irq_enable(struct irq_data *d)
{
- /*
- * Clear the interrupt that may be pending before we enable
- * the line.
- * This is especially a problem with the GPIOs routed to the
- * PDC. These GPIOs are direct-connect interrupts to the GIC.
- * Disabling the interrupt line at the PDC does not prevent
- * the interrupt from being latched at the GIC. The state at
- * GIC needs to be cleared before enabling.
- */
- if (d->parent_data) {
- irq_chip_set_parent_state(d, IRQCHIP_STATE_PENDING, 0);
+ struct gpio_chip *gc = irq_data_get_irq_chip_data(d);
+ struct msm_pinctrl *pctrl = gpiochip_get_data(gc);
+
+ if (d->parent_data)
irq_chip_enable_parent(d);
- }
- msm_gpio_irq_clear_unmask(d, true);
+ if (!test_bit(d->hwirq, pctrl->skip_wake_irqs))
+ msm_gpio_irq_clear_unmask(d, true);
}
static void msm_gpio_irq_disable(struct irq_data *d)
ret = -EINVAL;
goto out;
}
+
+ /*
+ * Clear the interrupt that may be pending before we enable
+ * the line.
+ * This is especially a problem with the GPIOs routed to the
+ * PDC. These GPIOs are direct-connect interrupts to the GIC.
+ * Disabling the interrupt line at the PDC does not prevent
+ * the interrupt from being latched at the GIC. The state at
+ * GIC needs to be cleared before enabling.
+ */
+ if (d->parent_data && test_bit(d->hwirq, pctrl->skip_wake_irqs))
+ irq_chip_set_parent_state(d, IRQCHIP_STATE_PENDING, 0);
+
return 0;
out:
module_put(gc->owner);
[183] = SDC_PINGROUP(sdc2_data, 0xb7000, 9, 0),
};
+static const struct msm_gpio_wakeirq_map sm8250_pdc_map[] = {
+ { 0, 79 }, { 1, 84 }, { 2, 80 }, { 3, 82 }, { 4, 107 }, { 7, 43 },
+ { 11, 42 }, { 14, 44 }, { 15, 52 }, { 19, 67 }, { 23, 68 }, { 24, 105 },
+ { 27, 92 }, { 28, 106 }, { 31, 69 }, { 35, 70 }, { 39, 37 },
+ { 40, 108 }, { 43, 71 }, { 45, 72 }, { 47, 83 }, { 51, 74 }, { 55, 77 },
+ { 59, 78 }, { 63, 75 }, { 64, 81 }, { 65, 87 }, { 66, 88 }, { 67, 89 },
+ { 68, 54 }, { 70, 85 }, { 77, 46 }, { 80, 90 }, { 81, 91 }, { 83, 97 },
+ { 84, 98 }, { 86, 99 }, { 87, 100 }, { 88, 101 }, { 89, 102 },
+ { 92, 103 }, { 93, 104 }, { 100, 53 }, { 103, 47 }, { 104, 48 },
+ { 108, 49 }, { 109, 94 }, { 110, 95 }, { 111, 96 }, { 112, 55 },
+ { 113, 56 }, { 118, 50 }, { 121, 51 }, { 122, 57 }, { 123, 58 },
+ { 124, 45 }, { 126, 59 }, { 128, 76 }, { 129, 86 }, { 132, 93 },
+ { 133, 65 }, { 134, 66 }, { 136, 62 }, { 137, 63 }, { 138, 64 },
+ { 142, 60 }, { 143, 61 }
+};
+
static const struct msm_pinctrl_soc_data sm8250_pinctrl = {
.pins = sm8250_pins,
.npins = ARRAY_SIZE(sm8250_pins),
.ngpios = 181,
.tiles = sm8250_tiles,
.ntiles = ARRAY_SIZE(sm8250_tiles),
+ .wakeirq_map = sm8250_pdc_map,
+ .nwakeirq_map = ARRAY_SIZE(sm8250_pdc_map),
};
static int sm8250_pinctrl_probe(struct platform_device *pdev)
{KE_KEY, 0x64, {KEY_SWITCHVIDEOMODE} }, /* Display Switch */
{KE_IGNORE, 0x81, {KEY_SLEEP} },
{KE_KEY, 0x82, {KEY_TOUCHPAD_TOGGLE} }, /* Touch Pad Toggle */
+ {KE_IGNORE, 0x84, {KEY_KBDILLUMTOGGLE} }, /* Automatic Keyboard background light toggle */
{KE_KEY, KEY_TOUCHPAD_ON, {KEY_TOUCHPAD_ON} },
{KE_KEY, KEY_TOUCHPAD_OFF, {KEY_TOUCHPAD_OFF} },
{KE_IGNORE, 0x83, {KEY_TOUCHPAD_TOGGLE} },
DMI_MATCH(DMI_PRODUCT_NAME, "HP Stream x360 Convertible PC 11"),
},
},
+ {
+ .matches = {
+ DMI_MATCH(DMI_SYS_VENDOR, "Hewlett-Packard"),
+ DMI_MATCH(DMI_PRODUCT_NAME, "HP Pavilion 13 x360 PC"),
+ },
+ },
{} /* Array terminator */
};
in_tablet_mode = hotkey_gmms_get_tablet_mode(res,
&has_tablet_mode);
- if (has_tablet_mode)
+ /*
+ * The Yoga 11e series has 2 accelerometers described by a
+ * BOSC0200 ACPI node. This setup relies on a Windows service
+ * which calls special ACPI methods on this node to report
+ * the laptop/tent/tablet mode to the EC. The bmc150 iio driver
+ * does not support this, so skip the hotkey on these models.
+ */
+ if (has_tablet_mode && !acpi_dev_present("BOSC0200", "1", -1))
tp_features.hotkey_tablet = TP_HOTKEY_TABLET_USES_GMMS;
type = "GMMS";
} else if (acpi_evalf(hkey_handle, &res, "MHKG", "qd")) {
pr_err("error while attempting to reset the event firmware interface\n");
tpacpi_send_radiosw_update();
+ tpacpi_input_send_tabletsw();
hotkey_tablet_mode_notify_change();
hotkey_wakeup_reason_notify_change();
hotkey_wakeup_hotunplug_complete_notify_change();
TPACPI_Q_LNV3('N', '2', 'C', TPACPI_FAN_2CTL), /* P52 / P72 */
TPACPI_Q_LNV3('N', '2', 'E', TPACPI_FAN_2CTL), /* P1 / X1 Extreme (1st gen) */
TPACPI_Q_LNV3('N', '2', 'O', TPACPI_FAN_2CTL), /* P1 / X1 Extreme (2nd gen) */
+ TPACPI_Q_LNV3('N', '2', 'V', TPACPI_FAN_2CTL), /* P1 / X1 Extreme (3nd gen) */
+ TPACPI_Q_LNV3('N', '3', '0', TPACPI_FAN_2CTL), /* P15 (1st gen) / P15v (1st gen) */
};
static int __init fan_init(struct ibm_init_struct *iibm)
TPACPI_Q_LNV3('R', '0', 'B', true), /* Thinkpad 11e gen 3 */
TPACPI_Q_LNV3('R', '0', 'C', true), /* Thinkpad 13 */
TPACPI_Q_LNV3('R', '0', 'J', true), /* Thinkpad 13 gen 2 */
+ TPACPI_Q_LNV3('R', '0', 'K', true), /* Thinkpad 11e gen 4 celeron BIOS */
};
static int __init tpacpi_battery_init(struct ibm_init_struct *ibm)
struct toshiba_acpi_dev *dev = PDE_DATA(file_inode(file));
char *buffer;
char *cmd;
- int lcd_out, crt_out, tv_out;
+ int lcd_out = -1, crt_out = -1, tv_out = -1;
int remain = count;
int value;
int ret;
kfree(cmd);
- lcd_out = crt_out = tv_out = -1;
ret = get_video_status(dev, &video_out);
if (!ret) {
unsigned int new_video_out = video_out;
.properties = irbis_tw90_props,
};
+static const struct property_entry irbis_tw118_props[] = {
+ PROPERTY_ENTRY_U32("touchscreen-min-x", 20),
+ PROPERTY_ENTRY_U32("touchscreen-min-y", 30),
+ PROPERTY_ENTRY_U32("touchscreen-size-x", 1960),
+ PROPERTY_ENTRY_U32("touchscreen-size-y", 1510),
+ PROPERTY_ENTRY_STRING("firmware-name", "gsl1680-irbis-tw118.fw"),
+ PROPERTY_ENTRY_U32("silead,max-fingers", 10),
+ { }
+};
+
+static const struct ts_dmi_data irbis_tw118_data = {
+ .acpi_name = "MSSL1680:00",
+ .properties = irbis_tw118_props,
+};
+
static const struct property_entry itworks_tw891_props[] = {
PROPERTY_ENTRY_U32("touchscreen-min-x", 1),
PROPERTY_ENTRY_U32("touchscreen-min-y", 5),
.properties = pov_mobii_wintab_p1006w_v10_props,
};
+static const struct property_entry predia_basic_props[] = {
+ PROPERTY_ENTRY_U32("touchscreen-min-x", 3),
+ PROPERTY_ENTRY_U32("touchscreen-min-y", 10),
+ PROPERTY_ENTRY_U32("touchscreen-size-x", 1728),
+ PROPERTY_ENTRY_U32("touchscreen-size-y", 1144),
+ PROPERTY_ENTRY_BOOL("touchscreen-swapped-x-y"),
+ PROPERTY_ENTRY_STRING("firmware-name", "gsl3680-predia-basic.fw"),
+ PROPERTY_ENTRY_U32("silead,max-fingers", 10),
+ PROPERTY_ENTRY_BOOL("silead,home-button"),
+ { }
+};
+
+static const struct ts_dmi_data predia_basic_data = {
+ .acpi_name = "MSSL1680:00",
+ .properties = predia_basic_props,
+};
+
static const struct property_entry schneider_sct101ctm_props[] = {
PROPERTY_ENTRY_U32("touchscreen-size-x", 1715),
PROPERTY_ENTRY_U32("touchscreen-size-y", 1140),
DMI_MATCH(DMI_PRODUCT_NAME, "TW90"),
},
},
+ {
+ /* Irbis TW118 */
+ .driver_data = (void *)&irbis_tw118_data,
+ .matches = {
+ DMI_MATCH(DMI_SYS_VENDOR, "IRBIS"),
+ DMI_MATCH(DMI_PRODUCT_NAME, "TW118"),
+ },
+ },
{
/* I.T.Works TW891 */
.driver_data = (void *)&itworks_tw891_data,
DMI_MATCH(DMI_BIOS_DATE, "10/24/2014"),
},
},
+ {
+ /* Predia Basic tablet) */
+ .driver_data = (void *)&predia_basic_data,
+ .matches = {
+ DMI_MATCH(DMI_SYS_VENDOR, "Insyde"),
+ DMI_MATCH(DMI_PRODUCT_NAME, "CherryTrail"),
+ /* Above matches are too generic, add bios-version match */
+ DMI_MATCH(DMI_BIOS_VERSION, "Mx.WT107.KUBNGEA"),
+ },
+ },
{
/* Point of View mobii wintab p800w (v2.1) */
.driver_data = (void *)&pov_mobii_wintab_p800w_v21_data,
return 0;
}
-static int idtcm_strverscmp(const char *ver1, const char *ver2)
+static int idtcm_strverscmp(const char *version1, const char *version2)
{
- u8 num1;
- u8 num2;
- int result = 0;
-
- /* loop through each level of the version string */
- while (result == 0) {
- /* extract leading version numbers */
- if (kstrtou8(ver1, 10, &num1) < 0)
- return -1;
+ u8 ver1[3], ver2[3];
+ int i;
- if (kstrtou8(ver2, 10, &num2) < 0)
- return -1;
+ if (sscanf(version1, "%hhu.%hhu.%hhu",
+ &ver1[0], &ver1[1], &ver1[2]) != 3)
+ return -1;
+ if (sscanf(version2, "%hhu.%hhu.%hhu",
+ &ver2[0], &ver2[1], &ver2[2]) != 3)
+ return -1;
- /* if numbers differ, then set the result */
- if (num1 < num2)
- result = -1;
- else if (num1 > num2)
- result = 1;
- else {
- /* if numbers are the same, go to next level */
- ver1 = strchr(ver1, '.');
- ver2 = strchr(ver2, '.');
- if (!ver1 && !ver2)
- break;
- else if (!ver1)
- result = -1;
- else if (!ver2)
- result = 1;
- else {
- ver1++;
- ver2++;
- }
- }
+ for (i = 0; i < 3; i++) {
+ if (ver1[i] > ver2[i])
+ return 1;
+ if (ver1[i] < ver2[i])
+ return -1;
}
- return result;
+
+ return 0;
}
static int idtcm_xfer_read(struct idtcm *idtcm,
#define MESSAGE_TYPE_P_DELAY_RESP 3
#define MESSAGE_TYPE_DELAY_REQ 4
-#define SYNC 0x0
-#define DELAY_REQ 0x1
-#define PDELAY_REQ 0x2
-#define PDELAY_RESP 0x3
-
static LIST_HEAD(ines_clocks);
static DEFINE_MUTEX(ines_clocks_lock);
msgtype = ptp_get_msgtype(hdr, type);
- switch ((msgtype & 0xf)) {
- case SYNC:
- case PDELAY_RESP:
+ switch (msgtype) {
+ case PTP_MSGTYPE_SYNC:
+ case PTP_MSGTYPE_PDELAY_RESP:
return true;
default:
return false;
{
switch (tag) {
case MESSAGE_TYPE_SYNC:
- return SYNC;
+ return PTP_MSGTYPE_SYNC;
case MESSAGE_TYPE_P_DELAY_REQ:
- return PDELAY_REQ;
+ return PTP_MSGTYPE_PDELAY_REQ;
case MESSAGE_TYPE_P_DELAY_RESP:
- return PDELAY_RESP;
+ return PTP_MSGTYPE_PDELAY_RESP;
case MESSAGE_TYPE_DELAY_REQ:
- return DELAY_REQ;
+ return PTP_MSGTYPE_DELAY_REQ;
}
return 0xf;
}
struct regmap *regmap;
u32 offset;
};
+#define sl28cpld_pwm_from_chip(_chip) \
+ container_of(_chip, struct sl28cpld_pwm, pwm_chip)
static void sl28cpld_pwm_get_state(struct pwm_chip *chip,
struct pwm_device *pwm,
struct pwm_state *state)
{
- struct sl28cpld_pwm *priv = dev_get_drvdata(chip->dev);
+ struct sl28cpld_pwm *priv = sl28cpld_pwm_from_chip(chip);
unsigned int reg;
int prescaler;
static int sl28cpld_pwm_apply(struct pwm_chip *chip, struct pwm_device *pwm,
const struct pwm_state *state)
{
- struct sl28cpld_pwm *priv = dev_get_drvdata(chip->dev);
+ struct sl28cpld_pwm *priv = sl28cpld_pwm_from_chip(chip);
unsigned int cycle, prescaler;
bool write_duty_cycle_first;
int ret;
/**
* set_machine_constraints - sets regulator constraints
* @rdev: regulator source
- * @constraints: constraints to apply
*
* Allows platform initialisation code to define and constrain
* regulator circuits e.g. valid voltage/current ranges, etc. NOTE:
* regulator operations to proceed i.e. set_voltage, set_current_limit,
* set_mode.
*/
-static int set_machine_constraints(struct regulator_dev *rdev,
- const struct regulation_constraints *constraints)
+static int set_machine_constraints(struct regulator_dev *rdev)
{
int ret = 0;
const struct regulator_ops *ops = rdev->desc->ops;
- if (constraints)
- rdev->constraints = kmemdup(constraints, sizeof(*constraints),
- GFP_KERNEL);
- else
- rdev->constraints = kzalloc(sizeof(*constraints),
- GFP_KERNEL);
- if (!rdev->constraints)
- return -ENOMEM;
-
ret = machine_constraints_voltage(rdev, rdev->constraints);
if (ret != 0)
return ret;
}
}
+ if (r == rdev) {
+ dev_err(dev, "Supply for %s (%s) resolved to itself\n",
+ rdev->desc->name, rdev->supply_name);
+ if (!have_full_constraints())
+ return -EINVAL;
+ r = dummy_regulator_rdev;
+ get_device(&r->dev);
+ }
+
/*
* If the supply's parent device is not the same as the
* regulator's parent device, then ensure the parent device
regulator_register(const struct regulator_desc *regulator_desc,
const struct regulator_config *cfg)
{
- const struct regulation_constraints *constraints = NULL;
const struct regulator_init_data *init_data;
struct regulator_config *config = NULL;
static atomic_t regulator_no = ATOMIC_INIT(-1);
/* set regulator constraints */
if (init_data)
- constraints = &init_data->constraints;
+ rdev->constraints = kmemdup(&init_data->constraints,
+ sizeof(*rdev->constraints),
+ GFP_KERNEL);
+ else
+ rdev->constraints = kzalloc(sizeof(*rdev->constraints),
+ GFP_KERNEL);
+ if (!rdev->constraints) {
+ ret = -ENOMEM;
+ goto wash;
+ }
if (init_data && init_data->supply_regulator)
rdev->supply_name = init_data->supply_regulator;
else if (regulator_desc->supply_name)
rdev->supply_name = regulator_desc->supply_name;
- ret = set_machine_constraints(rdev, constraints);
+ ret = set_machine_constraints(rdev);
if (ret == -EPROBE_DEFER) {
/* Regulator might be in bypass mode and so needs its supply
* to set the constraints */
* that is just being created */
ret = regulator_resolve_supply(rdev);
if (!ret)
- ret = set_machine_constraints(rdev, constraints);
+ ret = set_machine_constraints(rdev);
else
rdev_dbg(rdev, "unable to resolve supply early: %pe\n",
ERR_PTR(ret));
if (rdev->use_count)
goto unlock;
- /* If we can't read the status assume it's on. */
+ /* If we can't read the status assume it's always on. */
if (ops->is_enabled)
enabled = ops->is_enabled(rdev);
else
enabled = 1;
- if (!enabled)
+ /* But if reading the status failed, assume that it's off. */
+ if (enabled <= 0)
goto unlock;
if (have_full_constraints()) {
* the switched regulator till yet.
*/
if (pfuze_chip->flags & PFUZE_FLAG_DISABLE_SW) {
- if (pfuze_chip->regulator_descs[i].sw_reg) {
- desc->ops = &pfuze100_sw_disable_regulator_ops;
- desc->enable_val = 0x8;
- desc->disable_val = 0x0;
- desc->enable_time = 500;
+ if (pfuze_chip->chip_id == PFUZE100 ||
+ pfuze_chip->chip_id == PFUZE200) {
+ if (pfuze_chip->regulator_descs[i].sw_reg) {
+ desc->ops = &pfuze100_sw_disable_regulator_ops;
+ desc->enable_val = 0x8;
+ desc->disable_val = 0x0;
+ desc->enable_time = 500;
+ }
}
}
return ret;
}
- /* If data is exactly the same, then just update index, no change */
info = &abb->info[sel];
+ /*
+ * When Linux kernel is starting up, we are'nt sure of the
+ * Bias configuration that bootloader has configured.
+ * So, we get to know the actual setting the first time
+ * we are asked to transition.
+ */
+ if (abb->current_info_idx == -EINVAL)
+ goto just_set_abb;
+
+ /* If data is exactly the same, then just update index, no change */
oinfo = &abb->info[abb->current_info_idx];
if (!memcmp(info, oinfo, sizeof(*info))) {
dev_dbg(dev, "%s: Same data new idx=%d, old idx=%d\n", __func__,
goto out;
}
+just_set_abb:
ret = ti_abb_set_opp(rdev, abb, info);
out:
if (!block)
return -EINVAL;
+ /*
+ * If the request is an ERP request there is nothing to requeue.
+ * This will be done with the remaining original request.
+ */
+ if (cqr->refers)
+ return 0;
spin_lock_irq(&cqr->dq->lock);
req = (struct request *) cqr->callback_data;
blk_mq_requeue_request(req, false);
/* p_header points to the last one we handled */
if (p_header)
p_header->pdu_flag |= PDU_LAST; /*Say it's the last one*/
- header = kzalloc(TH_HEADER_LENGTH, gfp_type());
- if (!header) {
- spin_unlock(&ch->collect_lock);
- fsm_event(priv->mpcg->fsm, MPCG_EVENT_INOP, dev);
- goto done;
- }
+
+ header = skb_push(ch->trans_skb, TH_HEADER_LENGTH);
+ memset(header, 0, TH_HEADER_LENGTH);
+
header->th_ch_flag = TH_HAS_PDU; /* Normal data */
ch->th_seq_num++;
header->th_seq_num = ch->th_seq_num;
CTCM_PR_DBGDATA("%s: ToVTAM_th_seq= %08x\n" ,
__func__, ch->th_seq_num);
- memcpy(skb_push(ch->trans_skb, TH_HEADER_LENGTH), header,
- TH_HEADER_LENGTH); /* put the TH on the packet */
-
- kfree(header);
-
CTCM_PR_DBGDATA("%s: trans_skb len:%04x \n",
__func__, ch->trans_skb->len);
CTCM_PR_DBGDATA("%s: up-to-50 bytes of trans_skb "
goto nomem;
}
- header = kmalloc(TH_SWEEP_LENGTH, gfp_type());
-
- if (!header) {
- dev_kfree_skb_any(sweep_skb);
- /* rc = -ENOMEM; */
- goto nomem;
- }
-
- header->th.th_seg = 0x00 ;
+ header = skb_put_zero(sweep_skb, TH_SWEEP_LENGTH);
header->th.th_ch_flag = TH_SWEEP_REQ; /* 0x0f */
- header->th.th_blk_flag = 0x00;
- header->th.th_is_xid = 0x00;
- header->th.th_seq_num = 0x00;
header->sw.th_last_seq = ch->th_seq_num;
- skb_put_data(sweep_skb, header, TH_SWEEP_LENGTH);
-
- kfree(header);
-
netif_trans_update(dev);
skb_queue_tail(&ch->sweep_queue, sweep_skb);
if ((fsm_getstate(ch->fsm) != CTC_STATE_TXIDLE) || grp->in_sweep) {
spin_lock_irqsave(&ch->collect_lock, saveflags);
refcount_inc(&skb->users);
- p_header = kmalloc(PDU_HEADER_LENGTH, gfp_type());
-
- if (!p_header) {
- spin_unlock_irqrestore(&ch->collect_lock, saveflags);
- goto nomem_exit;
- }
- p_header->pdu_offset = skb->len;
+ p_header = skb_push(skb, PDU_HEADER_LENGTH);
+ p_header->pdu_offset = skb->len - PDU_HEADER_LENGTH;
p_header->pdu_proto = 0x01;
- p_header->pdu_flag = 0x00;
if (be16_to_cpu(skb->protocol) == ETH_P_SNAP) {
- p_header->pdu_flag |= PDU_FIRST | PDU_CNTL;
+ p_header->pdu_flag = PDU_FIRST | PDU_CNTL;
} else {
- p_header->pdu_flag |= PDU_FIRST;
+ p_header->pdu_flag = PDU_FIRST;
}
p_header->pdu_seq = 0;
- memcpy(skb_push(skb, PDU_HEADER_LENGTH), p_header,
- PDU_HEADER_LENGTH);
CTCM_PR_DEBUG("%s(%s): Put on collect_q - skb len: %04x \n"
"pdu header and data for up to 32 bytes:\n",
skb_queue_tail(&ch->collect_queue, skb);
ch->collect_len += skb->len;
- kfree(p_header);
spin_unlock_irqrestore(&ch->collect_lock, saveflags);
goto done;
}
}
- p_header = kmalloc(PDU_HEADER_LENGTH, gfp_type());
-
- if (!p_header)
- goto nomem_exit;
-
- p_header->pdu_offset = skb->len;
+ p_header = skb_push(skb, PDU_HEADER_LENGTH);
+ p_header->pdu_offset = skb->len - PDU_HEADER_LENGTH;
p_header->pdu_proto = 0x01;
- p_header->pdu_flag = 0x00;
p_header->pdu_seq = 0;
if (be16_to_cpu(skb->protocol) == ETH_P_SNAP) {
- p_header->pdu_flag |= PDU_FIRST | PDU_CNTL;
+ p_header->pdu_flag = PDU_FIRST | PDU_CNTL;
} else {
- p_header->pdu_flag |= PDU_FIRST;
+ p_header->pdu_flag = PDU_FIRST;
}
- memcpy(skb_push(skb, PDU_HEADER_LENGTH), p_header, PDU_HEADER_LENGTH);
-
- kfree(p_header);
if (ch->collect_len > 0) {
spin_lock_irqsave(&ch->collect_lock, saveflags);
ch->prof.txlen += skb->len - PDU_HEADER_LENGTH;
- header = kmalloc(TH_HEADER_LENGTH, gfp_type());
- if (!header)
- goto nomem_exit;
+ /* put the TH on the packet */
+ header = skb_push(skb, TH_HEADER_LENGTH);
+ memset(header, 0, TH_HEADER_LENGTH);
- header->th_seg = 0x00;
header->th_ch_flag = TH_HAS_PDU; /* Normal data */
- header->th_blk_flag = 0x00;
- header->th_is_xid = 0x00; /* Just data here */
ch->th_seq_num++;
header->th_seq_num = ch->th_seq_num;
CTCM_PR_DBGDATA("%s(%s) ToVTAM_th_seq= %08x\n" ,
__func__, dev->name, ch->th_seq_num);
- /* put the TH on the packet */
- memcpy(skb_push(skb, TH_HEADER_LENGTH), header, TH_HEADER_LENGTH);
-
- kfree(header);
-
CTCM_PR_DBGDATA("%s(%s): skb len: %04x\n - pdu header and data for "
"up to 32 bytes sent to vtam:\n",
__func__, dev->name, skb->len);
CTCM_D3_DUMP((char *)skb->data, min_t(int, 32, skb->len));
len = skb->len + TH_HEADER_LENGTH + PDU_HEADER_LENGTH;
- newskb = __dev_alloc_skb(len, gfp_type() | GFP_DMA);
+ newskb = __dev_alloc_skb(len, GFP_ATOMIC | GFP_DMA);
if (!newskb) {
CTCM_DBF_TEXT_(MPC_TRACE, CTC_DBF_ERROR,
ch->protocol = priv->protocol;
if (IS_MPC(priv)) {
- ch->discontact_th = kzalloc(TH_HEADER_LENGTH, gfp_type());
+ ch->discontact_th = kzalloc(TH_HEADER_LENGTH, GFP_KERNEL);
if (ch->discontact_th == NULL)
goto nomem_return;
/* test if struct ctcm_priv of struct net_device has MPC protocol setting */
#define IS_MPCDEV(dev) IS_MPC((struct ctcm_priv *)dev->ml_priv)
-static inline gfp_t gfp_type(void)
-{
- return in_interrupt() ? GFP_ATOMIC : GFP_KERNEL;
-}
-
/*
* Definition of our link level header.
*/
goto done;
}
- header = kmalloc(sizeof(struct th_sweep), gfp_type());
-
- if (!header) {
- dev_kfree_skb_any(sweep_skb);
- goto done;
- }
-
- header->th.th_seg = 0x00 ;
+ header = skb_put_zero(sweep_skb, TH_SWEEP_LENGTH);
header->th.th_ch_flag = TH_SWEEP_RESP;
- header->th.th_blk_flag = 0x00;
- header->th.th_is_xid = 0x00;
- header->th.th_seq_num = 0x00;
header->sw.th_last_seq = ch->th_seq_num;
- skb_put_data(sweep_skb, header, TH_SWEEP_LENGTH);
-
- kfree(header);
-
netif_trans_update(dev);
skb_queue_tail(&ch->sweep_queue, sweep_skb);
skb_pull(pskb, new_len); /* point to next PDU */
}
} else {
- mpcginfo = kmalloc(sizeof(struct mpcg_info), gfp_type());
+ mpcginfo = kmalloc(sizeof(struct mpcg_info), GFP_ATOMIC);
if (mpcginfo == NULL)
goto done;
*/
static int mpc_send_qllc_discontact(struct net_device *dev)
{
- __u32 new_len = 0;
struct sk_buff *skb;
struct qllc *qllcptr;
struct ctcm_priv *priv = dev->ml_priv;
case MPCG_STATE_FLOWC:
case MPCG_STATE_READY:
grp->send_qllc_disc = 2;
- new_len = sizeof(struct qllc);
- qllcptr = kzalloc(new_len, gfp_type() | GFP_DMA);
- if (qllcptr == NULL) {
- CTCM_DBF_TEXT_(MPC_ERROR, CTC_DBF_ERROR,
- "%s(%s): qllcptr allocation error",
- CTCM_FUNTAIL, dev->name);
- return -ENOMEM;
- }
-
- qllcptr->qllc_address = 0xcc;
- qllcptr->qllc_commands = 0x03;
-
- skb = __dev_alloc_skb(new_len, GFP_ATOMIC);
+ skb = __dev_alloc_skb(sizeof(struct qllc), GFP_ATOMIC);
if (skb == NULL) {
CTCM_DBF_TEXT_(MPC_ERROR, CTC_DBF_ERROR,
"%s(%s): skb allocation error",
CTCM_FUNTAIL, dev->name);
priv->stats.rx_dropped++;
- kfree(qllcptr);
return -ENOMEM;
}
- skb_put_data(skb, qllcptr, new_len);
- kfree(qllcptr);
+ qllcptr = skb_put(skb, sizeof(struct qllc));
+ qllcptr->qllc_address = 0xcc;
+ qllcptr->qllc_commands = 0x03;
if (skb_headroom(skb) < 4) {
CTCM_DBF_TEXT_(MPC_ERROR, CTC_DBF_ERROR,
QETH_QDIO_BUF_EMPTY,
/* Filled by driver; owned by hardware in order to be sent. */
QETH_QDIO_BUF_PRIMED,
- /* Identified to be pending in TPQ. */
+ /* Discovered by the TX completion code: */
QETH_QDIO_BUF_PENDING,
- /* Found in completion queue. */
- QETH_QDIO_BUF_IN_CQ,
+ /* Finished by the TX completion code: */
+ QETH_QDIO_BUF_NEED_QAOB,
+ /* Received QAOB notification on CQ: */
+ QETH_QDIO_BUF_QAOB_OK,
+ QETH_QDIO_BUF_QAOB_ERROR,
/* Handled via transfer pending / completion queue. */
QETH_QDIO_BUF_HANDLED_DELAYED,
};
QETH_PNSO_ADDR_INFO,
};
+enum qeth_link_mode {
+ QETH_LINK_MODE_UNKNOWN,
+ QETH_LINK_MODE_FIBRE_SHORT,
+ QETH_LINK_MODE_FIBRE_LONG,
+};
+
+struct qeth_link_info {
+ u32 speed;
+ u8 duplex;
+ u8 port;
+ enum qeth_link_mode link_mode;
+};
+
#define QETH_BROADCAST_WITH_ECHO 0x01
#define QETH_BROADCAST_WITHOUT_ECHO 0x02
struct qeth_card_info {
struct qeth_card_blkt blkt;
__u32 diagass_support;
__u32 hwtrap;
+ struct qeth_link_info link_info;
};
enum qeth_discipline_id {
u8 bufs_refill;
};
-struct carrier_info {
- __u8 card_type;
- __u16 port_mode;
- __u32 port_speed;
-};
-
struct qeth_switch_info {
__u32 capabilities;
__u32 settings;
int qeth_query_switch_attributes(struct qeth_card *card,
struct qeth_switch_info *sw_info);
int qeth_query_card_info(struct qeth_card *card,
- struct carrier_info *carrier_info);
+ struct qeth_link_info *link_info);
int qeth_setadpparms_set_access_ctrl(struct qeth_card *card,
enum qeth_ipa_isolation_modes mode);
#include <net/iucv/af_iucv.h>
#include <net/dsfield.h>
+#include <net/sock.h>
#include <asm/ebcdic.h>
#include <asm/chpid.h>
}
}
- if (forced_cleanup && (atomic_read(&(q->bufs[bidx]->state)) ==
- QETH_QDIO_BUF_HANDLED_DELAYED)) {
- /* for recovery situations */
- qeth_init_qdio_out_buf(q, bidx);
- QETH_CARD_TEXT(q->card, 2, "clprecov");
- }
}
static void qeth_qdio_handle_aob(struct qeth_card *card,
unsigned long phys_aob_addr)
{
+ enum qeth_qdio_out_buffer_state new_state = QETH_QDIO_BUF_QAOB_OK;
struct qaob *aob;
struct qeth_qdio_out_buffer *buffer;
enum iucv_tx_notify notification;
buffer = (struct qeth_qdio_out_buffer *) aob->user1;
QETH_CARD_TEXT_(card, 5, "%lx", aob->user1);
- if (atomic_cmpxchg(&buffer->state, QETH_QDIO_BUF_PRIMED,
- QETH_QDIO_BUF_IN_CQ) == QETH_QDIO_BUF_PRIMED) {
- notification = TX_NOTIFY_OK;
- } else {
- WARN_ON_ONCE(atomic_read(&buffer->state) !=
- QETH_QDIO_BUF_PENDING);
- atomic_set(&buffer->state, QETH_QDIO_BUF_IN_CQ);
- notification = TX_NOTIFY_DELAYED_OK;
- }
-
- if (aob->aorc != 0) {
- QETH_CARD_TEXT_(card, 2, "aorc%02X", aob->aorc);
- notification = qeth_compute_cq_notification(aob->aorc, 1);
- }
- qeth_notify_skbs(buffer->q, buffer, notification);
-
/* Free dangling allocations. The attached skbs are handled by
* qeth_cleanup_handled_pending().
*/
if (data && buffer->is_header[i])
kmem_cache_free(qeth_core_header_cache, data);
}
- atomic_set(&buffer->state, QETH_QDIO_BUF_HANDLED_DELAYED);
+
+ if (aob->aorc) {
+ QETH_CARD_TEXT_(card, 2, "aorc%02X", aob->aorc);
+ new_state = QETH_QDIO_BUF_QAOB_ERROR;
+ }
+
+ switch (atomic_xchg(&buffer->state, new_state)) {
+ case QETH_QDIO_BUF_PRIMED:
+ /* Faster than TX completion code. */
+ notification = qeth_compute_cq_notification(aob->aorc, 0);
+ qeth_notify_skbs(buffer->q, buffer, notification);
+ atomic_set(&buffer->state, QETH_QDIO_BUF_HANDLED_DELAYED);
+ break;
+ case QETH_QDIO_BUF_PENDING:
+ /* TX completion code is active and will handle the async
+ * completion for us.
+ */
+ break;
+ case QETH_QDIO_BUF_NEED_QAOB:
+ /* TX completion code is already finished. */
+ notification = qeth_compute_cq_notification(aob->aorc, 1);
+ qeth_notify_skbs(buffer->q, buffer, notification);
+ atomic_set(&buffer->state, QETH_QDIO_BUF_HANDLED_DELAYED);
+ break;
+ default:
+ WARN_ON_ONCE(1);
+ }
qdio_release_aob(aob);
}
skb_queue_walk(&buf->skb_list, skb) {
QETH_CARD_TEXT_(q->card, 5, "skbn%d", notification);
QETH_CARD_TEXT_(q->card, 5, "%lx", (long) skb);
- if (skb->protocol == htons(ETH_P_AF_IUCV) && skb->sk)
+ if (skb->sk && skb->sk->sk_family == PF_IUCV)
iucv_sk(skb->sk)->sk_txnotify(skb, notification);
}
}
struct qeth_qdio_out_q *queue = buf->q;
struct sk_buff *skb;
- /* release may never happen from within CQ tasklet scope */
- WARN_ON_ONCE(atomic_read(&buf->state) == QETH_QDIO_BUF_IN_CQ);
-
if (atomic_read(&buf->state) == QETH_QDIO_BUF_PENDING)
qeth_notify_skbs(queue, buf, TX_NOTIFY_GENERALERROR);
static int qeth_query_card_info_cb(struct qeth_card *card,
struct qeth_reply *reply, unsigned long data)
{
- struct carrier_info *carrier_info = (struct carrier_info *)reply->param;
struct qeth_ipa_cmd *cmd = (struct qeth_ipa_cmd *)data;
+ struct qeth_link_info *link_info = reply->param;
struct qeth_query_card_info *card_info;
QETH_CARD_TEXT(card, 2, "qcrdincb");
return -EIO;
card_info = &cmd->data.setadapterparms.data.card_info;
- carrier_info->card_type = card_info->card_type;
- carrier_info->port_mode = card_info->port_mode;
- carrier_info->port_speed = card_info->port_speed;
+ netdev_dbg(card->dev,
+ "card info: card_type=0x%02x, port_mode=0x%04x, port_speed=0x%08x\n",
+ card_info->card_type, card_info->port_mode,
+ card_info->port_speed);
+
+ switch (card_info->port_mode) {
+ case CARD_INFO_PORTM_FULLDUPLEX:
+ link_info->duplex = DUPLEX_FULL;
+ break;
+ case CARD_INFO_PORTM_HALFDUPLEX:
+ link_info->duplex = DUPLEX_HALF;
+ break;
+ default:
+ link_info->duplex = DUPLEX_UNKNOWN;
+ }
+
+ switch (card_info->card_type) {
+ case CARD_INFO_TYPE_1G_COPPER_A:
+ case CARD_INFO_TYPE_1G_COPPER_B:
+ link_info->speed = SPEED_1000;
+ link_info->port = PORT_TP;
+ break;
+ case CARD_INFO_TYPE_1G_FIBRE_A:
+ case CARD_INFO_TYPE_1G_FIBRE_B:
+ link_info->speed = SPEED_1000;
+ link_info->port = PORT_FIBRE;
+ break;
+ case CARD_INFO_TYPE_10G_FIBRE_A:
+ case CARD_INFO_TYPE_10G_FIBRE_B:
+ link_info->speed = SPEED_10000;
+ link_info->port = PORT_FIBRE;
+ break;
+ default:
+ switch (card_info->port_speed) {
+ case CARD_INFO_PORTS_10M:
+ link_info->speed = SPEED_10;
+ break;
+ case CARD_INFO_PORTS_100M:
+ link_info->speed = SPEED_100;
+ break;
+ case CARD_INFO_PORTS_1G:
+ link_info->speed = SPEED_1000;
+ break;
+ case CARD_INFO_PORTS_10G:
+ link_info->speed = SPEED_10000;
+ break;
+ case CARD_INFO_PORTS_25G:
+ link_info->speed = SPEED_25000;
+ break;
+ default:
+ link_info->speed = SPEED_UNKNOWN;
+ }
+
+ link_info->port = PORT_OTHER;
+ }
+
return 0;
}
int qeth_query_card_info(struct qeth_card *card,
- struct carrier_info *carrier_info)
+ struct qeth_link_info *link_info)
{
struct qeth_cmd_buffer *iob;
iob = qeth_get_adapter_cmd(card, IPA_SETADP_QUERY_CARD_INFO, 0);
if (!iob)
return -ENOMEM;
- return qeth_send_ipa_cmd(card, iob, qeth_query_card_info_cb,
- (void *)carrier_info);
+
+ return qeth_send_ipa_cmd(card, iob, qeth_query_card_info_cb, link_info);
+}
+
+static int qeth_init_link_info_oat_cb(struct qeth_card *card,
+ struct qeth_reply *reply_priv,
+ unsigned long data)
+{
+ struct qeth_ipa_cmd *cmd = (struct qeth_ipa_cmd *)data;
+ struct qeth_link_info *link_info = reply_priv->param;
+ struct qeth_query_oat_physical_if *phys_if;
+ struct qeth_query_oat_reply *reply;
+
+ if (qeth_setadpparms_inspect_rc(cmd))
+ return -EIO;
+
+ /* Multi-part reply is unexpected, don't bother: */
+ if (cmd->data.setadapterparms.hdr.used_total > 1)
+ return -EINVAL;
+
+ /* Expect the reply to start with phys_if data: */
+ reply = &cmd->data.setadapterparms.data.query_oat.reply[0];
+ if (reply->type != QETH_QOAT_REPLY_TYPE_PHYS_IF ||
+ reply->length < sizeof(*reply))
+ return -EINVAL;
+
+ phys_if = &reply->phys_if;
+
+ switch (phys_if->speed_duplex) {
+ case QETH_QOAT_PHYS_SPEED_10M_HALF:
+ link_info->speed = SPEED_10;
+ link_info->duplex = DUPLEX_HALF;
+ break;
+ case QETH_QOAT_PHYS_SPEED_10M_FULL:
+ link_info->speed = SPEED_10;
+ link_info->duplex = DUPLEX_FULL;
+ break;
+ case QETH_QOAT_PHYS_SPEED_100M_HALF:
+ link_info->speed = SPEED_100;
+ link_info->duplex = DUPLEX_HALF;
+ break;
+ case QETH_QOAT_PHYS_SPEED_100M_FULL:
+ link_info->speed = SPEED_100;
+ link_info->duplex = DUPLEX_FULL;
+ break;
+ case QETH_QOAT_PHYS_SPEED_1000M_HALF:
+ link_info->speed = SPEED_1000;
+ link_info->duplex = DUPLEX_HALF;
+ break;
+ case QETH_QOAT_PHYS_SPEED_1000M_FULL:
+ link_info->speed = SPEED_1000;
+ link_info->duplex = DUPLEX_FULL;
+ break;
+ case QETH_QOAT_PHYS_SPEED_10G_FULL:
+ link_info->speed = SPEED_10000;
+ link_info->duplex = DUPLEX_FULL;
+ break;
+ case QETH_QOAT_PHYS_SPEED_25G_FULL:
+ link_info->speed = SPEED_25000;
+ link_info->duplex = DUPLEX_FULL;
+ break;
+ case QETH_QOAT_PHYS_SPEED_UNKNOWN:
+ default:
+ link_info->speed = SPEED_UNKNOWN;
+ link_info->duplex = DUPLEX_UNKNOWN;
+ break;
+ }
+
+ switch (phys_if->media_type) {
+ case QETH_QOAT_PHYS_MEDIA_COPPER:
+ link_info->port = PORT_TP;
+ link_info->link_mode = QETH_LINK_MODE_UNKNOWN;
+ break;
+ case QETH_QOAT_PHYS_MEDIA_FIBRE_SHORT:
+ link_info->port = PORT_FIBRE;
+ link_info->link_mode = QETH_LINK_MODE_FIBRE_SHORT;
+ break;
+ case QETH_QOAT_PHYS_MEDIA_FIBRE_LONG:
+ link_info->port = PORT_FIBRE;
+ link_info->link_mode = QETH_LINK_MODE_FIBRE_LONG;
+ break;
+ default:
+ link_info->port = PORT_OTHER;
+ link_info->link_mode = QETH_LINK_MODE_UNKNOWN;
+ break;
+ }
+
+ return 0;
+}
+
+static void qeth_init_link_info(struct qeth_card *card)
+{
+ card->info.link_info.duplex = DUPLEX_FULL;
+
+ if (IS_IQD(card) || IS_VM_NIC(card)) {
+ card->info.link_info.speed = SPEED_10000;
+ card->info.link_info.port = PORT_FIBRE;
+ card->info.link_info.link_mode = QETH_LINK_MODE_FIBRE_SHORT;
+ } else {
+ switch (card->info.link_type) {
+ case QETH_LINK_TYPE_FAST_ETH:
+ case QETH_LINK_TYPE_LANE_ETH100:
+ card->info.link_info.speed = SPEED_100;
+ card->info.link_info.port = PORT_TP;
+ break;
+ case QETH_LINK_TYPE_GBIT_ETH:
+ case QETH_LINK_TYPE_LANE_ETH1000:
+ card->info.link_info.speed = SPEED_1000;
+ card->info.link_info.port = PORT_FIBRE;
+ break;
+ case QETH_LINK_TYPE_10GBIT_ETH:
+ card->info.link_info.speed = SPEED_10000;
+ card->info.link_info.port = PORT_FIBRE;
+ break;
+ case QETH_LINK_TYPE_25GBIT_ETH:
+ card->info.link_info.speed = SPEED_25000;
+ card->info.link_info.port = PORT_FIBRE;
+ break;
+ default:
+ dev_info(&card->gdev->dev, "Unknown link type %x\n",
+ card->info.link_type);
+ card->info.link_info.speed = SPEED_UNKNOWN;
+ card->info.link_info.port = PORT_OTHER;
+ }
+
+ card->info.link_info.link_mode = QETH_LINK_MODE_UNKNOWN;
+ }
+
+ /* Get more accurate data via QUERY OAT: */
+ if (qeth_adp_supported(card, IPA_SETADP_QUERY_OAT)) {
+ struct qeth_link_info link_info;
+ struct qeth_cmd_buffer *iob;
+
+ iob = qeth_get_adapter_cmd(card, IPA_SETADP_QUERY_OAT,
+ SETADP_DATA_SIZEOF(query_oat));
+ if (iob) {
+ struct qeth_ipa_cmd *cmd = __ipa_cmd(iob);
+ struct qeth_query_oat *oat_req;
+
+ oat_req = &cmd->data.setadapterparms.data.query_oat;
+ oat_req->subcmd_code = QETH_QOAT_SCOPE_INTERFACE;
+
+ if (!qeth_send_ipa_cmd(card, iob,
+ qeth_init_link_info_oat_cb,
+ &link_info)) {
+ if (link_info.speed != SPEED_UNKNOWN)
+ card->info.link_info.speed = link_info.speed;
+ if (link_info.duplex != DUPLEX_UNKNOWN)
+ card->info.link_info.duplex = link_info.duplex;
+ if (link_info.port != PORT_OTHER)
+ card->info.link_info.port = link_info.port;
+ if (link_info.link_mode != QETH_LINK_MODE_UNKNOWN)
+ card->info.link_info.link_mode = link_info.link_mode;
+ }
+ }
+ }
}
/**
goto out;
}
+ qeth_init_link_info(card);
+
rc = qeth_init_qdio_queues(card);
if (rc) {
QETH_CARD_TEXT_(card, 2, "9err%d", rc);
if (atomic_cmpxchg(&buffer->state, QETH_QDIO_BUF_PRIMED,
QETH_QDIO_BUF_PENDING) ==
- QETH_QDIO_BUF_PRIMED)
+ QETH_QDIO_BUF_PRIMED) {
qeth_notify_skbs(queue, buffer, TX_NOTIFY_PENDING);
+ /* Handle race with qeth_qdio_handle_aob(): */
+ switch (atomic_xchg(&buffer->state,
+ QETH_QDIO_BUF_NEED_QAOB)) {
+ case QETH_QDIO_BUF_PENDING:
+ /* No concurrent QAOB notification. */
+ break;
+ case QETH_QDIO_BUF_QAOB_OK:
+ qeth_notify_skbs(queue, buffer,
+ TX_NOTIFY_DELAYED_OK);
+ atomic_set(&buffer->state,
+ QETH_QDIO_BUF_HANDLED_DELAYED);
+ break;
+ case QETH_QDIO_BUF_QAOB_ERROR:
+ qeth_notify_skbs(queue, buffer,
+ TX_NOTIFY_DELAYED_GENERALERROR);
+ atomic_set(&buffer->state,
+ QETH_QDIO_BUF_HANDLED_DELAYED);
+ break;
+ default:
+ WARN_ON_ONCE(1);
+ }
+ }
+
QETH_CARD_TEXT_(card, 5, "pel%u", bidx);
/* prepare the queue slot for re-use: */
__u8 reserved[8];
} __attribute__((packed));
+#define QETH_QOAT_PHYS_SPEED_UNKNOWN 0x00
+#define QETH_QOAT_PHYS_SPEED_10M_HALF 0x01
+#define QETH_QOAT_PHYS_SPEED_10M_FULL 0x02
+#define QETH_QOAT_PHYS_SPEED_100M_HALF 0x03
+#define QETH_QOAT_PHYS_SPEED_100M_FULL 0x04
+#define QETH_QOAT_PHYS_SPEED_1000M_HALF 0x05
+#define QETH_QOAT_PHYS_SPEED_1000M_FULL 0x06
+// n/a 0x07
+#define QETH_QOAT_PHYS_SPEED_10G_FULL 0x08
+// n/a 0x09
+#define QETH_QOAT_PHYS_SPEED_25G_FULL 0x0A
+
+#define QETH_QOAT_PHYS_MEDIA_COPPER 0x01
+#define QETH_QOAT_PHYS_MEDIA_FIBRE_SHORT 0x02
+#define QETH_QOAT_PHYS_MEDIA_FIBRE_LONG 0x04
+
+struct qeth_query_oat_physical_if {
+ u8 res_head[33];
+ u8 speed_duplex;
+ u8 media_type;
+ u8 res_tail[29];
+};
+
+#define QETH_QOAT_REPLY_TYPE_PHYS_IF 0x0004
+
+struct qeth_query_oat_reply {
+ u16 type;
+ u16 length;
+ u16 version;
+ u8 res[10];
+ struct qeth_query_oat_physical_if phys_if;
+};
+
+#define QETH_QOAT_SCOPE_INTERFACE 0x00000001
+
struct qeth_query_oat {
- __u32 subcmd_code;
- __u8 reserved[12];
+ u32 subcmd_code;
+ u8 reserved[12];
+ struct qeth_query_oat_reply reply[];
} __packed;
struct qeth_qoat_priv {
/* Autoneg and full-duplex are supported and advertised unconditionally. */
/* Always advertise and support all speeds up to specified, and only one */
/* specified port type. */
-static void qeth_set_cmd_adv_sup(struct ethtool_link_ksettings *cmd,
- int maxspeed, int porttype)
+static void qeth_set_ethtool_link_modes(struct ethtool_link_ksettings *cmd,
+ enum qeth_link_mode link_mode)
{
ethtool_link_ksettings_zero_link_mode(cmd, supported);
ethtool_link_ksettings_zero_link_mode(cmd, advertising);
ethtool_link_ksettings_add_link_mode(cmd, supported, Autoneg);
ethtool_link_ksettings_add_link_mode(cmd, advertising, Autoneg);
- switch (porttype) {
+ switch (cmd->base.port) {
case PORT_TP:
ethtool_link_ksettings_add_link_mode(cmd, supported, TP);
ethtool_link_ksettings_add_link_mode(cmd, advertising, TP);
+
+ switch (cmd->base.speed) {
+ case SPEED_10000:
+ ethtool_link_ksettings_add_link_mode(cmd, supported,
+ 10000baseT_Full);
+ ethtool_link_ksettings_add_link_mode(cmd, advertising,
+ 10000baseT_Full);
+ fallthrough;
+ case SPEED_1000:
+ ethtool_link_ksettings_add_link_mode(cmd, supported,
+ 1000baseT_Full);
+ ethtool_link_ksettings_add_link_mode(cmd, advertising,
+ 1000baseT_Full);
+ ethtool_link_ksettings_add_link_mode(cmd, supported,
+ 1000baseT_Half);
+ ethtool_link_ksettings_add_link_mode(cmd, advertising,
+ 1000baseT_Half);
+ fallthrough;
+ case SPEED_100:
+ ethtool_link_ksettings_add_link_mode(cmd, supported,
+ 100baseT_Full);
+ ethtool_link_ksettings_add_link_mode(cmd, advertising,
+ 100baseT_Full);
+ ethtool_link_ksettings_add_link_mode(cmd, supported,
+ 100baseT_Half);
+ ethtool_link_ksettings_add_link_mode(cmd, advertising,
+ 100baseT_Half);
+ fallthrough;
+ case SPEED_10:
+ ethtool_link_ksettings_add_link_mode(cmd, supported,
+ 10baseT_Full);
+ ethtool_link_ksettings_add_link_mode(cmd, advertising,
+ 10baseT_Full);
+ ethtool_link_ksettings_add_link_mode(cmd, supported,
+ 10baseT_Half);
+ ethtool_link_ksettings_add_link_mode(cmd, advertising,
+ 10baseT_Half);
+ break;
+ default:
+ break;
+ }
+
break;
case PORT_FIBRE:
ethtool_link_ksettings_add_link_mode(cmd, supported, FIBRE);
ethtool_link_ksettings_add_link_mode(cmd, advertising, FIBRE);
- break;
- default:
- ethtool_link_ksettings_add_link_mode(cmd, supported, TP);
- ethtool_link_ksettings_add_link_mode(cmd, advertising, TP);
- WARN_ON_ONCE(1);
- }
- /* partially does fall through, to also select lower speeds */
- switch (maxspeed) {
- case SPEED_25000:
- ethtool_link_ksettings_add_link_mode(cmd, supported,
- 25000baseSR_Full);
- ethtool_link_ksettings_add_link_mode(cmd, advertising,
- 25000baseSR_Full);
- break;
- case SPEED_10000:
- ethtool_link_ksettings_add_link_mode(cmd, supported,
- 10000baseT_Full);
- ethtool_link_ksettings_add_link_mode(cmd, advertising,
- 10000baseT_Full);
- fallthrough;
- case SPEED_1000:
- ethtool_link_ksettings_add_link_mode(cmd, supported,
- 1000baseT_Full);
- ethtool_link_ksettings_add_link_mode(cmd, advertising,
- 1000baseT_Full);
- ethtool_link_ksettings_add_link_mode(cmd, supported,
- 1000baseT_Half);
- ethtool_link_ksettings_add_link_mode(cmd, advertising,
- 1000baseT_Half);
- fallthrough;
- case SPEED_100:
- ethtool_link_ksettings_add_link_mode(cmd, supported,
- 100baseT_Full);
- ethtool_link_ksettings_add_link_mode(cmd, advertising,
- 100baseT_Full);
- ethtool_link_ksettings_add_link_mode(cmd, supported,
- 100baseT_Half);
- ethtool_link_ksettings_add_link_mode(cmd, advertising,
- 100baseT_Half);
- fallthrough;
- case SPEED_10:
- ethtool_link_ksettings_add_link_mode(cmd, supported,
- 10baseT_Full);
- ethtool_link_ksettings_add_link_mode(cmd, advertising,
- 10baseT_Full);
- ethtool_link_ksettings_add_link_mode(cmd, supported,
- 10baseT_Half);
- ethtool_link_ksettings_add_link_mode(cmd, advertising,
- 10baseT_Half);
+ switch (cmd->base.speed) {
+ case SPEED_25000:
+ ethtool_link_ksettings_add_link_mode(cmd, supported,
+ 25000baseSR_Full);
+ ethtool_link_ksettings_add_link_mode(cmd, advertising,
+ 25000baseSR_Full);
+ break;
+ case SPEED_10000:
+ if (link_mode == QETH_LINK_MODE_FIBRE_LONG) {
+ ethtool_link_ksettings_add_link_mode(cmd, supported,
+ 10000baseLR_Full);
+ ethtool_link_ksettings_add_link_mode(cmd, advertising,
+ 10000baseLR_Full);
+ } else if (link_mode == QETH_LINK_MODE_FIBRE_SHORT) {
+ ethtool_link_ksettings_add_link_mode(cmd, supported,
+ 10000baseSR_Full);
+ ethtool_link_ksettings_add_link_mode(cmd, advertising,
+ 10000baseSR_Full);
+ }
+ break;
+ case SPEED_1000:
+ ethtool_link_ksettings_add_link_mode(cmd, supported,
+ 1000baseX_Full);
+ ethtool_link_ksettings_add_link_mode(cmd, advertising,
+ 1000baseX_Full);
+ break;
+ default:
+ break;
+ }
+
break;
default:
- ethtool_link_ksettings_add_link_mode(cmd, supported,
- 10baseT_Full);
- ethtool_link_ksettings_add_link_mode(cmd, advertising,
- 10baseT_Full);
- ethtool_link_ksettings_add_link_mode(cmd, supported,
- 10baseT_Half);
- ethtool_link_ksettings_add_link_mode(cmd, advertising,
- 10baseT_Half);
- WARN_ON_ONCE(1);
+ break;
}
}
-
static int qeth_get_link_ksettings(struct net_device *netdev,
struct ethtool_link_ksettings *cmd)
{
struct qeth_card *card = netdev->ml_priv;
- enum qeth_link_types link_type;
- struct carrier_info carrier_info;
- int rc;
+ struct qeth_link_info link_info;
- if (IS_IQD(card) || IS_VM_NIC(card))
- link_type = QETH_LINK_TYPE_10GBIT_ETH;
- else
- link_type = card->info.link_type;
-
- cmd->base.duplex = DUPLEX_FULL;
+ cmd->base.speed = card->info.link_info.speed;
+ cmd->base.duplex = card->info.link_info.duplex;
+ cmd->base.port = card->info.link_info.port;
cmd->base.autoneg = AUTONEG_ENABLE;
cmd->base.phy_address = 0;
cmd->base.mdio_support = 0;
cmd->base.eth_tp_mdix = ETH_TP_MDI_INVALID;
cmd->base.eth_tp_mdix_ctrl = ETH_TP_MDI_INVALID;
- switch (link_type) {
- case QETH_LINK_TYPE_FAST_ETH:
- case QETH_LINK_TYPE_LANE_ETH100:
- cmd->base.speed = SPEED_100;
- cmd->base.port = PORT_TP;
- break;
- case QETH_LINK_TYPE_GBIT_ETH:
- case QETH_LINK_TYPE_LANE_ETH1000:
- cmd->base.speed = SPEED_1000;
- cmd->base.port = PORT_FIBRE;
- break;
- case QETH_LINK_TYPE_10GBIT_ETH:
- cmd->base.speed = SPEED_10000;
- cmd->base.port = PORT_FIBRE;
- break;
- case QETH_LINK_TYPE_25GBIT_ETH:
- cmd->base.speed = SPEED_25000;
- cmd->base.port = PORT_FIBRE;
- break;
- default:
- cmd->base.speed = SPEED_10;
- cmd->base.port = PORT_TP;
- }
- qeth_set_cmd_adv_sup(cmd, cmd->base.speed, cmd->base.port);
-
/* Check if we can obtain more accurate information. */
- /* If QUERY_CARD_INFO command is not supported or fails, */
- /* just return the heuristics that was filled above. */
- rc = qeth_query_card_info(card, &carrier_info);
- if (rc == -EOPNOTSUPP) /* for old hardware, return heuristic */
- return 0;
- if (rc) /* report error from the hardware operation */
- return rc;
- /* on success, fill in the information got from the hardware */
-
- netdev_dbg(netdev,
- "card info: card_type=0x%02x, port_mode=0x%04x, port_speed=0x%08x\n",
- carrier_info.card_type,
- carrier_info.port_mode,
- carrier_info.port_speed);
-
- /* Update attributes for which we've obtained more authoritative */
- /* information, leave the rest the way they where filled above. */
- switch (carrier_info.card_type) {
- case CARD_INFO_TYPE_1G_COPPER_A:
- case CARD_INFO_TYPE_1G_COPPER_B:
- cmd->base.port = PORT_TP;
- qeth_set_cmd_adv_sup(cmd, SPEED_1000, cmd->base.port);
- break;
- case CARD_INFO_TYPE_1G_FIBRE_A:
- case CARD_INFO_TYPE_1G_FIBRE_B:
- cmd->base.port = PORT_FIBRE;
- qeth_set_cmd_adv_sup(cmd, SPEED_1000, cmd->base.port);
- break;
- case CARD_INFO_TYPE_10G_FIBRE_A:
- case CARD_INFO_TYPE_10G_FIBRE_B:
- cmd->base.port = PORT_FIBRE;
- qeth_set_cmd_adv_sup(cmd, SPEED_10000, cmd->base.port);
- break;
- }
-
- switch (carrier_info.port_mode) {
- case CARD_INFO_PORTM_FULLDUPLEX:
- cmd->base.duplex = DUPLEX_FULL;
- break;
- case CARD_INFO_PORTM_HALFDUPLEX:
- cmd->base.duplex = DUPLEX_HALF;
- break;
+ if (!qeth_query_card_info(card, &link_info)) {
+ if (link_info.speed != SPEED_UNKNOWN)
+ cmd->base.speed = link_info.speed;
+ if (link_info.duplex != DUPLEX_UNKNOWN)
+ cmd->base.duplex = link_info.duplex;
+ if (link_info.port != PORT_OTHER)
+ cmd->base.port = link_info.port;
}
- switch (carrier_info.port_speed) {
- case CARD_INFO_PORTS_10M:
- cmd->base.speed = SPEED_10;
- break;
- case CARD_INFO_PORTS_100M:
- cmd->base.speed = SPEED_100;
- break;
- case CARD_INFO_PORTS_1G:
- cmd->base.speed = SPEED_1000;
- break;
- case CARD_INFO_PORTS_10G:
- cmd->base.speed = SPEED_10000;
- break;
- case CARD_INFO_PORTS_25G:
- cmd->base.speed = SPEED_25000;
- break;
- }
+ qeth_set_ethtool_link_modes(cmd, card->info.link_info.link_mode);
return 0;
}
*
* On enable, emits a series of address notifications for all
* currently registered hosts.
- *
- * Must be called under rtnl_lock
*/
static int qeth_l2_dev2br_an_set(struct qeth_card *card, bool enable)
{
* change notification' and thus can support the learning_sync bridgeport
* attribute
* @card: qeth_card structure pointer
- *
- * This is a destructive test and must be called before dev2br or
- * bridgeport address notification is enabled!
*/
static void qeth_l2_detect_dev2br_support(struct qeth_card *card)
{
struct qeth_priv *priv = netdev_priv(card->dev);
bool dev2br_supported;
- int rc;
QETH_CARD_TEXT(card, 2, "d2brsup");
if (!IS_IQD(card))
return;
/* dev2br requires valid cssid,iid,chid */
- if (!card->info.ids_valid) {
- dev2br_supported = false;
- } else if (css_general_characteristics.enarf) {
- dev2br_supported = true;
- } else {
- /* Old machines don't have the feature bit:
- * Probe by testing whether a disable succeeds
- */
- rc = qeth_l2_pnso(card, PNSO_OC_NET_ADDR_INFO, 0, NULL, NULL);
- dev2br_supported = !rc;
- }
+ dev2br_supported = card->info.ids_valid &&
+ css_general_characteristics.enarf;
QETH_CARD_TEXT_(card, 2, "D2Bsup%02x", dev2br_supported);
if (dev2br_supported)
if (READ_ONCE(card->info.pnso_mode) == QETH_PNSO_NONE)
goto free;
- /* Potential re-config in progress, try again later: */
- if (!rtnl_trylock()) {
- queue_delayed_work(card->event_wq, dwork,
- msecs_to_jiffies(100));
- return;
- }
- if (!netif_device_present(card->dev))
- goto out_unlock;
-
if (data->ac_event.lost_event_mask) {
+ /* Potential re-config in progress, try again later: */
+ if (!rtnl_trylock()) {
+ queue_delayed_work(card->event_wq, dwork,
+ msecs_to_jiffies(100));
+ return;
+ }
+
+ if (!netif_device_present(card->dev)) {
+ rtnl_unlock();
+ goto free;
+ }
+
QETH_DBF_MESSAGE(3,
"Address change notification overflow on device %x\n",
CARD_DEVID(card));
"Address Notification resynced on device %x\n",
CARD_DEVID(card));
}
+
+ rtnl_unlock();
} else {
for (i = 0; i < data->ac_event.num_entries; i++) {
struct qeth_ipacmd_addr_change_entry *entry =
}
}
-out_unlock:
- rtnl_unlock();
-
free:
kfree(data);
}
struct net_device *dev = card->dev;
int rc = 0;
- /* query before bridgeport_notification may be enabled */
qeth_l2_detect_dev2br_support(card);
mutex_lock(&card->sbp_lock);
card->state = CARD_STATE_DOWN;
qeth_l2_set_pnso_mode(card, QETH_PNSO_NONE);
- if (priv->brport_features & BR_LEARNING_SYNC) {
- rtnl_lock();
+ if (priv->brport_features & BR_LEARNING_SYNC)
qeth_l2_dev2br_fdb_flush(card);
- rtnl_unlock();
- }
}
/* Returns zero if the command is successfully "consumed" */
qeth_l3_convert_addr_to_bits(ipatoe->addr, ipatoe_bits,
(ipatoe->proto == QETH_PROT_IPV4) ?
4 : 16);
- if (addr->proto == QETH_PROT_IPV4)
- rc = !memcmp(addr_bits, ipatoe_bits, ipatoe->mask_bits);
- else
- rc = !memcmp(addr_bits, ipatoe_bits, ipatoe->mask_bits);
+ rc = !memcmp(addr_bits, ipatoe_bits, ipatoe->mask_bits);
if (rc)
break;
}
#include "bnx2fc.h"
+#include <linux/ethtool.h>
+
static struct list_head adapter_list;
static struct list_head if_list;
static u32 adapter_count;
#include <linux/kernel.h>
#include <linux/list.h>
#include <linux/netdevice.h>
+#include <linux/ethtool.h>
#include <linux/errno.h>
#include <linux/crc32.h>
#include <scsi/libfcoe.h>
if (conn->task == task)
conn->task = NULL;
- if (conn->ping_task == task)
- conn->ping_task = NULL;
+ if (READ_ONCE(conn->ping_task) == task)
+ WRITE_ONCE(conn->ping_task, NULL);
/* release get from queueing */
__iscsi_put_task(task);
task->conn->session->age);
}
+ if (unlikely(READ_ONCE(conn->ping_task) == INVALID_SCSI_TASK))
+ WRITE_ONCE(conn->ping_task, task);
+
if (!ihost->workq) {
if (iscsi_prep_mgmt_task(conn, task))
goto free_task;
struct iscsi_nopout hdr;
struct iscsi_task *task;
- if (!rhdr && conn->ping_task)
- return -EINVAL;
+ if (!rhdr) {
+ if (READ_ONCE(conn->ping_task))
+ return -EINVAL;
+ WRITE_ONCE(conn->ping_task, INVALID_SCSI_TASK);
+ }
memset(&hdr, 0, sizeof(struct iscsi_nopout));
hdr.opcode = ISCSI_OP_NOOP_OUT | ISCSI_OP_IMMEDIATE;
task = __iscsi_conn_send_pdu(conn, (struct iscsi_hdr *)&hdr, NULL, 0);
if (!task) {
+ if (!rhdr)
+ WRITE_ONCE(conn->ping_task, NULL);
iscsi_conn_printk(KERN_ERR, conn, "Could not send nopout\n");
return -EIO;
} else if (!rhdr) {
/* only track our nops */
- conn->ping_task = task;
conn->last_ping = jiffies;
}
struct iscsi_conn *conn = task->conn;
int rc = 0;
- if (conn->ping_task != task) {
+ if (READ_ONCE(conn->ping_task) != task) {
/*
* If this is not in response to one of our
* nops then it must be from userspace.
*/
static int iscsi_has_ping_timed_out(struct iscsi_conn *conn)
{
- if (conn->ping_task &&
+ if (READ_ONCE(conn->ping_task) &&
time_before_eq(conn->last_recv + (conn->recv_timeout * HZ) +
(conn->ping_timeout * HZ), jiffies))
return 1;
* Checking the transport already or nop from a cmd timeout still
* running
*/
- if (conn->ping_task) {
+ if (READ_ONCE(conn->ping_task)) {
task->have_checked_conn = true;
rc = BLK_EH_RESET_TIMER;
goto done;
}
spin_unlock_irqrestore(hba->host->host_lock, irq_flags);
+ pm_runtime_get_noresume(hba->dev);
+ if (!pm_runtime_active(hba->dev)) {
+ pm_runtime_put_noidle(hba->dev);
+ ret = -EAGAIN;
+ goto out;
+ }
start = ktime_get();
ret = ufshcd_devfreq_scale(hba, scale_up);
+ pm_runtime_put(hba->dev);
trace_ufshcd_profile_clk_scaling(dev_name(hba->dev),
(scale_up ? "up" : "down"),
*/
fallthrough;
case CLKS_OFF:
- ufshcd_scsi_block_requests(hba);
hba->clk_gating.state = REQ_CLKS_ON;
trace_ufshcd_clk_gating(dev_name(hba->dev),
hba->clk_gating.state);
- queue_work(hba->clk_gating.clk_gating_workq,
- &hba->clk_gating.ungate_work);
+ if (queue_work(hba->clk_gating.clk_gating_workq,
+ &hba->clk_gating.ungate_work))
+ ufshcd_scsi_block_requests(hba);
/*
* fall through to check if we should wait for this
* work to be done or not.
unsigned long flags;
if (wait_for_completion_timeout(&uic_cmd->done,
- msecs_to_jiffies(UIC_CMD_TIMEOUT)))
+ msecs_to_jiffies(UIC_CMD_TIMEOUT))) {
ret = uic_cmd->argument2 & MASK_UIC_COMMAND_RESULT;
- else
+ } else {
ret = -ETIMEDOUT;
+ dev_err(hba->dev,
+ "uic cmd 0x%x with arg3 0x%x completion timeout\n",
+ uic_cmd->command, uic_cmd->argument3);
+
+ if (!uic_cmd->cmd_active) {
+ dev_err(hba->dev, "%s: UIC cmd has been completed, return the result\n",
+ __func__);
+ ret = uic_cmd->argument2 & MASK_UIC_COMMAND_RESULT;
+ }
+ }
spin_lock_irqsave(hba->host->host_lock, flags);
hba->active_uic_cmd = NULL;
if (completion)
init_completion(&uic_cmd->done);
+ uic_cmd->cmd_active = 1;
ufshcd_dispatch_uic_cmd(hba, uic_cmd);
return 0;
/* Get the length of descriptor */
ufshcd_map_desc_id_to_length(hba, desc_id, &buff_len);
if (!buff_len) {
- dev_err(hba->dev, "%s: Failed to get desc length", __func__);
+ dev_err(hba->dev, "%s: Failed to get desc length\n", __func__);
+ return -EINVAL;
+ }
+
+ if (param_offset >= buff_len) {
+ dev_err(hba->dev, "%s: Invalid offset 0x%x in descriptor IDN 0x%x, length 0x%x\n",
+ __func__, param_offset, desc_id, buff_len);
return -EINVAL;
}
/* Check whether we need temp memory */
if (param_offset != 0 || param_size < buff_len) {
- desc_buf = kmalloc(buff_len, GFP_KERNEL);
+ desc_buf = kzalloc(buff_len, GFP_KERNEL);
if (!desc_buf)
return -ENOMEM;
} else {
desc_buf, &buff_len);
if (ret) {
- dev_err(hba->dev, "%s: Failed reading descriptor. desc_id %d, desc_index %d, param_offset %d, ret %d",
+ dev_err(hba->dev, "%s: Failed reading descriptor. desc_id %d, desc_index %d, param_offset %d, ret %d\n",
__func__, desc_id, desc_index, param_offset, ret);
goto out;
}
/* Sanity check */
if (desc_buf[QUERY_DESC_DESC_TYPE_OFFSET] != desc_id) {
- dev_err(hba->dev, "%s: invalid desc_id %d in descriptor header",
+ dev_err(hba->dev, "%s: invalid desc_id %d in descriptor header\n",
__func__, desc_buf[QUERY_DESC_DESC_TYPE_OFFSET]);
ret = -EINVAL;
goto out;
buff_len = desc_buf[QUERY_DESC_LENGTH_OFFSET];
ufshcd_update_desc_length(hba, desc_id, desc_index, buff_len);
- /* Check wherher we will not copy more data, than available */
- if (is_kmalloc && (param_offset + param_size) > buff_len)
- param_size = buff_len - param_offset;
-
- if (is_kmalloc)
+ if (is_kmalloc) {
+ /* Make sure we don't copy more data than available */
+ if (param_offset + param_size > buff_len)
+ param_size = buff_len - param_offset;
memcpy(param_read_buf, &desc_buf[param_offset], param_size);
+ }
out:
if (is_kmalloc)
kfree(desc_buf);
dev_err(hba->dev,
"pwr ctrl cmd 0x%x with mode 0x%x completion timeout\n",
cmd->command, cmd->argument3);
+
+ if (!cmd->cmd_active) {
+ dev_err(hba->dev, "%s: Power Mode Change operation has been completed, go check UPMCRS\n",
+ __func__);
+ goto check_upmcrs;
+ }
+
ret = -ETIMEDOUT;
goto out;
}
+check_upmcrs:
status = ufshcd_get_upmcrs(hba);
if (status != PWR_LOCAL) {
dev_err(hba->dev,
ufshcd_get_uic_cmd_result(hba);
hba->active_uic_cmd->argument3 =
ufshcd_get_dme_attr_val(hba);
+ if (!hba->uic_async_done)
+ hba->active_uic_cmd->cmd_active = 0;
complete(&hba->active_uic_cmd->done);
retval = IRQ_HANDLED;
}
if ((intr_status & UFSHCD_UIC_PWR_MASK) && hba->uic_async_done) {
+ hba->active_uic_cmd->cmd_active = 0;
complete(hba->uic_async_done);
retval = IRQ_HANDLED;
}
if (ufshcd_is_ufs_dev_poweroff(hba) && ufshcd_is_link_off(hba))
goto out;
- if (pm_runtime_suspended(hba->dev)) {
- ret = ufshcd_runtime_resume(hba);
- if (ret)
- goto out;
- }
+ pm_runtime_get_sync(hba->dev);
ret = ufshcd_suspend(hba, UFS_SHUTDOWN_PM);
out:
blk_mq_free_tag_set(&hba->tmf_tag_set);
blk_cleanup_queue(hba->cmd_queue);
scsi_remove_host(hba->host);
+ destroy_workqueue(hba->eh_wq);
/* disable interrupts */
ufshcd_disable_intr(hba, hba->intr_mask);
ufshcd_hba_stop(hba);
exit_gating:
ufshcd_exit_clk_scaling(hba);
ufshcd_exit_clk_gating(hba);
+ destroy_workqueue(hba->eh_wq);
out_disable:
hba->is_irq_enabled = false;
ufshcd_hba_exit(hba);
* @argument1: UIC command argument 1
* @argument2: UIC command argument 2
* @argument3: UIC command argument 3
+ * @cmd_active: Indicate if UIC command is outstanding
* @done: UIC command completion
*/
struct uic_command {
u32 argument1;
u32 argument2;
u32 argument3;
+ int cmd_active;
struct completion done;
};
{
int error;
struct fsl_mc_device_irq *irq;
- cpumask_t mask;
irq = dpio_dev->irqs[0];
error = devm_request_irq(&dpio_dev->dev,
}
/* set the affinity hint */
- cpumask_clear(&mask);
- cpumask_set_cpu(cpu, &mask);
- if (irq_set_affinity_hint(irq->msi_desc->irq, &mask))
+ if (irq_set_affinity_hint(irq->msi_desc->irq, cpumask_of(cpu)))
dev_err(&dpio_dev->dev,
"irq_set_affinity failed irq %d cpu %d\n",
irq->msi_desc->irq, cpu);
data = of_id->data;
- master = spi_alloc_master(dev, sizeof(struct bcm_qspi));
+ master = devm_spi_alloc_master(dev, sizeof(struct bcm_qspi));
if (!master) {
dev_err(dev, "error allocating spi_master\n");
return -ENOMEM;
if (res) {
qspi->base[MSPI] = devm_ioremap_resource(dev, res);
- if (IS_ERR(qspi->base[MSPI])) {
- ret = PTR_ERR(qspi->base[MSPI]);
- goto qspi_resource_err;
- }
+ if (IS_ERR(qspi->base[MSPI]))
+ return PTR_ERR(qspi->base[MSPI]);
} else {
- goto qspi_resource_err;
+ return 0;
}
res = platform_get_resource_byname(pdev, IORESOURCE_MEM, "bspi");
if (res) {
qspi->base[BSPI] = devm_ioremap_resource(dev, res);
- if (IS_ERR(qspi->base[BSPI])) {
- ret = PTR_ERR(qspi->base[BSPI]);
- goto qspi_resource_err;
- }
+ if (IS_ERR(qspi->base[BSPI]))
+ return PTR_ERR(qspi->base[BSPI]);
qspi->bspi_mode = true;
} else {
qspi->bspi_mode = false;
res = platform_get_resource_byname(pdev, IORESOURCE_MEM, "cs_reg");
if (res) {
qspi->base[CHIP_SELECT] = devm_ioremap_resource(dev, res);
- if (IS_ERR(qspi->base[CHIP_SELECT])) {
- ret = PTR_ERR(qspi->base[CHIP_SELECT]);
- goto qspi_resource_err;
- }
+ if (IS_ERR(qspi->base[CHIP_SELECT]))
+ return PTR_ERR(qspi->base[CHIP_SELECT]);
}
qspi->dev_ids = kcalloc(num_irqs, sizeof(struct bcm_qspi_dev_id),
GFP_KERNEL);
- if (!qspi->dev_ids) {
- ret = -ENOMEM;
- goto qspi_resource_err;
- }
+ if (!qspi->dev_ids)
+ return -ENOMEM;
for (val = 0; val < num_irqs; val++) {
irq = -1;
qspi->xfer_mode.addrlen = -1;
qspi->xfer_mode.hp = -1;
- ret = devm_spi_register_master(&pdev->dev, master);
+ ret = spi_register_master(master);
if (ret < 0) {
dev_err(dev, "can't register master\n");
goto qspi_reg_err;
clk_disable_unprepare(qspi->clk);
qspi_probe_err:
kfree(qspi->dev_ids);
-qspi_resource_err:
- spi_master_put(master);
return ret;
}
/* probe function to be called by SoC specific platform driver probe */
{
struct bcm_qspi *qspi = platform_get_drvdata(pdev);
+ spi_unregister_master(qspi->master);
bcm_qspi_hw_uninit(qspi);
clk_disable_unprepare(qspi->clk);
kfree(qspi->dev_ids);
- spi_unregister_master(qspi->master);
return 0;
}
struct bcm2835_spi *bs;
int err;
- ctlr = spi_alloc_master(&pdev->dev, ALIGN(sizeof(*bs),
+ ctlr = devm_spi_alloc_master(&pdev->dev, ALIGN(sizeof(*bs),
dma_get_cache_alignment()));
if (!ctlr)
return -ENOMEM;
bs->ctlr = ctlr;
bs->regs = devm_platform_ioremap_resource(pdev, 0);
- if (IS_ERR(bs->regs)) {
- err = PTR_ERR(bs->regs);
- goto out_controller_put;
- }
+ if (IS_ERR(bs->regs))
+ return PTR_ERR(bs->regs);
bs->clk = devm_clk_get(&pdev->dev, NULL);
- if (IS_ERR(bs->clk)) {
- err = dev_err_probe(&pdev->dev, PTR_ERR(bs->clk),
- "could not get clk\n");
- goto out_controller_put;
- }
+ if (IS_ERR(bs->clk))
+ return dev_err_probe(&pdev->dev, PTR_ERR(bs->clk),
+ "could not get clk\n");
bs->irq = platform_get_irq(pdev, 0);
- if (bs->irq <= 0) {
- err = bs->irq ? bs->irq : -ENODEV;
- goto out_controller_put;
- }
+ if (bs->irq <= 0)
+ return bs->irq ? bs->irq : -ENODEV;
clk_prepare_enable(bs->clk);
bcm2835_dma_release(ctlr, bs);
out_clk_disable:
clk_disable_unprepare(bs->clk);
-out_controller_put:
- spi_controller_put(ctlr);
return err;
}
unsigned long clk_hz;
int err;
- master = spi_alloc_master(&pdev->dev, sizeof(*bs));
+ master = devm_spi_alloc_master(&pdev->dev, sizeof(*bs));
if (!master)
return -ENOMEM;
/* the main area */
bs->regs = devm_platform_ioremap_resource(pdev, 0);
- if (IS_ERR(bs->regs)) {
- err = PTR_ERR(bs->regs);
- goto out_master_put;
- }
+ if (IS_ERR(bs->regs))
+ return PTR_ERR(bs->regs);
bs->clk = devm_clk_get(&pdev->dev, NULL);
if (IS_ERR(bs->clk)) {
err = PTR_ERR(bs->clk);
dev_err(&pdev->dev, "could not get clk: %d\n", err);
- goto out_master_put;
+ return err;
}
bs->irq = platform_get_irq(pdev, 0);
- if (bs->irq <= 0) {
- err = bs->irq ? bs->irq : -ENODEV;
- goto out_master_put;
- }
+ if (bs->irq <= 0)
+ return bs->irq ? bs->irq : -ENODEV;
/* this also enables the HW block */
err = clk_prepare_enable(bs->clk);
if (err) {
dev_err(&pdev->dev, "could not prepare clock: %d\n", err);
- goto out_master_put;
+ return err;
}
/* just checking if the clock returns a sane value */
out_clk_disable:
clk_disable_unprepare(bs->clk);
-out_master_put:
- spi_master_put(master);
return err;
}
/* Obtain QSPI reset control */
rstc = devm_reset_control_get_optional_exclusive(dev, "qspi");
if (IS_ERR(rstc)) {
+ ret = PTR_ERR(rstc);
dev_err(dev, "Cannot get QSPI reset.\n");
goto probe_reset_failed;
}
rstc_ocp = devm_reset_control_get_optional_exclusive(dev, "qspi-ocp");
if (IS_ERR(rstc_ocp)) {
+ ret = PTR_ERR(rstc_ocp);
dev_err(dev, "Cannot get QSPI OCP reset.\n");
goto probe_reset_failed;
}
dw_writel(dws, DW_SPI_TXFTLR, level);
dw_writel(dws, DW_SPI_RXFTLR, level - 1);
+ dws->transfer_handler = dw_spi_transfer_handler;
+
imask = SPI_INT_TXEI | SPI_INT_TXOI | SPI_INT_RXUI | SPI_INT_RXOI |
SPI_INT_RXFI;
spi_umask_intr(dws, imask);
-
- dws->transfer_handler = dw_spi_transfer_handler;
}
/*
master->set_cs = dw_spi_set_cs;
master->transfer_one = dw_spi_transfer_one;
master->handle_err = dw_spi_handle_err;
- master->mem_ops = &dws->mem_ops;
+ if (dws->mem_ops.exec_op)
+ master->mem_ops = &dws->mem_ops;
master->max_speed_hz = dws->max_freq;
master->dev.of_node = dev->of_node;
master->dev.fwnode = dev->fwnode;
rc = fsi_spi_check_mux(ctx->fsi, ctx->dev);
if (rc)
- return rc;
+ goto error;
list_for_each_entry(transfer, &mesg->transfers, transfer_list) {
struct fsi_spi_sequence seq;
spi_controller_get_devdata(controller);
pm_runtime_disable(fsl_lpspi->dev);
-
- spi_master_put(controller);
-
return 0;
}
pm_runtime_set_autosuspend_delay(spi_imx->dev, MXC_RPM_TIMEOUT);
pm_runtime_use_autosuspend(spi_imx->dev);
+ pm_runtime_get_noresume(spi_imx->dev);
pm_runtime_set_active(spi_imx->dev);
pm_runtime_enable(spi_imx->dev);
struct resource *res;
int id;
- ctrl = spi_alloc_master(dev, sizeof(*fiu));
+ ctrl = devm_spi_alloc_master(dev, sizeof(*fiu));
if (!ctrl)
return -ENOMEM;
struct resource *res;
struct nxp_fspi *f;
int ret;
+ u32 reg;
ctlr = spi_alloc_master(&pdev->dev, sizeof(*f));
if (!ctlr)
goto err_put_ctrl;
}
+ /* Clear potential interrupts */
+ reg = fspi_readl(f, f->iobase + FSPI_INTR);
+ if (reg)
+ fspi_writel(f, reg, f->iobase + FSPI_INTR);
+
+
/* find the resources - controller memory mapped space */
if (is_acpi_node(f->dev->fwnode))
res = platform_get_resource(pdev, IORESOURCE_MEM, 1);
enable = !enable;
if (spi->cs_gpiod || gpio_is_valid(spi->cs_gpio)) {
- /*
- * Honour the SPI_NO_CS flag and invert the enable line, as
- * active low is default for SPI. Execution paths that handle
- * polarity inversion in gpiolib (such as device tree) will
- * enforce active high using the SPI_CS_HIGH resulting in a
- * double inversion through the code above.
- */
if (!(spi->mode & SPI_NO_CS)) {
if (spi->cs_gpiod)
+ /* polarity handled by gpiolib */
gpiod_set_value_cansleep(spi->cs_gpiod,
- !enable);
+ enable1);
else
+ /*
+ * invert the enable line, as active low is
+ * default for SPI.
+ */
gpio_set_value_cansleep(spi->cs_gpio, !enable);
}
/* Some SPI masters need both GPIO CS & slave_select */
}
spi->chip_select = value;
- /*
- * For descriptors associated with the device, polarity inversion is
- * handled in the gpiolib, so all gpio chip selects are "active high"
- * in the logical sense, the gpiolib will invert the line if need be.
- */
- if ((ctlr->use_gpio_descriptors) && ctlr->cs_gpiods &&
- ctlr->cs_gpiods[spi->chip_select])
- spi->mode |= SPI_CS_HIGH;
-
/* Device speed */
if (!of_property_read_u32(nc, "spi-max-frequency", &value))
spi->max_speed_hz = value;
}
EXPORT_SYMBOL_GPL(__spi_alloc_controller);
+static void devm_spi_release_controller(struct device *dev, void *ctlr)
+{
+ spi_controller_put(*(struct spi_controller **)ctlr);
+}
+
+/**
+ * __devm_spi_alloc_controller - resource-managed __spi_alloc_controller()
+ * @dev: physical device of SPI controller
+ * @size: how much zeroed driver-private data to allocate
+ * @slave: whether to allocate an SPI master (false) or SPI slave (true)
+ * Context: can sleep
+ *
+ * Allocate an SPI controller and automatically release a reference on it
+ * when @dev is unbound from its driver. Drivers are thus relieved from
+ * having to call spi_controller_put().
+ *
+ * The arguments to this function are identical to __spi_alloc_controller().
+ *
+ * Return: the SPI controller structure on success, else NULL.
+ */
+struct spi_controller *__devm_spi_alloc_controller(struct device *dev,
+ unsigned int size,
+ bool slave)
+{
+ struct spi_controller **ptr, *ctlr;
+
+ ptr = devres_alloc(devm_spi_release_controller, sizeof(*ptr),
+ GFP_KERNEL);
+ if (!ptr)
+ return NULL;
+
+ ctlr = __spi_alloc_controller(dev, size, slave);
+ if (ctlr) {
+ *ptr = ctlr;
+ devres_add(dev, ptr);
+ } else {
+ devres_free(ptr);
+ }
+
+ return ctlr;
+}
+EXPORT_SYMBOL_GPL(__devm_spi_alloc_controller);
+
#ifdef CONFIG_OF
static int of_spi_get_gpio_numbers(struct spi_controller *ctlr)
{
}
EXPORT_SYMBOL_GPL(devm_spi_register_controller);
+static int devm_spi_match_controller(struct device *dev, void *res, void *ctlr)
+{
+ return *(struct spi_controller **)res == ctlr;
+}
+
static int __unregister(struct device *dev, void *null)
{
spi_unregister_device(to_spi_device(dev));
list_del(&ctlr->list);
mutex_unlock(&board_lock);
- device_unregister(&ctlr->dev);
+ device_del(&ctlr->dev);
+
+ /* Release the last reference on the controller if its driver
+ * has not yet been converted to devm_spi_alloc_master/slave().
+ */
+ if (!devres_find(ctlr->dev.parent, devm_spi_release_controller,
+ devm_spi_match_controller, ctlr))
+ put_device(&ctlr->dev);
+
/* free bus id */
mutex_lock(&board_lock);
if (found == ctlr)
if (!spi->max_speed_hz)
spi->max_speed_hz = spi->controller->max_speed_hz;
+ mutex_lock(&spi->controller->io_mutex);
+
if (spi->controller->setup)
status = spi->controller->setup(spi);
if (spi->controller->auto_runtime_pm && spi->controller->set_cs) {
status = pm_runtime_get_sync(spi->controller->dev.parent);
if (status < 0) {
+ mutex_unlock(&spi->controller->io_mutex);
pm_runtime_put_noidle(spi->controller->dev.parent);
dev_err(&spi->controller->dev, "Failed to power device: %d\n",
status);
spi_set_cs(spi, false);
}
+ mutex_unlock(&spi->controller->io_mutex);
+
if (spi->rt && !spi->controller->rt) {
spi->controller->rt = true;
spi_set_thread_rt(spi->controller);
*
*/
+#include <linux/ethtool.h>
+
#include "ethsw.h"
static struct {
reg |= (pps->second_chroma_qp_index_offset & 0x3f) << 16;
reg |= (pps->chroma_qp_index_offset & 0x3f) << 8;
reg |= (pps->pic_init_qp_minus26 + 26 + slice->slice_qp_delta) & 0x3f;
- if (pps->flags & V4L2_H264_PPS_FLAG_SCALING_MATRIX_PRESENT)
+ if (!(pps->flags & V4L2_H264_PPS_FLAG_SCALING_MATRIX_PRESENT))
reg |= VE_H264_SHS_QP_SCALING_MATRIX_DEFAULT;
cedrus_write(dev, VE_H264_SHS_QP, reg);
return 0;
}
-static int mt7621_pcie_request_resources(struct mt7621_pcie *pcie,
- struct list_head *res)
+static void mt7621_pcie_add_resources(struct mt7621_pcie *pcie,
+ struct list_head *res)
{
- struct device *dev = pcie->dev;
-
pci_add_resource_offset(res, &pcie->io, pcie->offset.io);
pci_add_resource_offset(res, &pcie->mem, pcie->offset.mem);
- pci_add_resource(res, &pcie->busn);
-
- return devm_request_pci_bus_resources(dev, res);
}
static int mt7621_pcie_register_host(struct pci_host_bridge *host,
setup_cm_memory_region(pcie);
- err = mt7621_pcie_request_resources(pcie, &res);
- if (err) {
- dev_err(dev, "Error requesting resources\n");
- return err;
- }
+ mt7621_pcie_add_resources(pcie, &res);
err = mt7621_pcie_register_host(bridge, &res);
if (err) {
config DMA_RALINK
tristate "RALINK DMA support"
depends on RALINK && !SOC_RT288X
+ depends on DMADEVICES
select DMA_ENGINE
select DMA_VIRTUAL_CHANNELS
{ SDIO_DEVICE(0x024c, 0x0525), },
{ SDIO_DEVICE(0x024c, 0x0623), },
{ SDIO_DEVICE(0x024c, 0x0626), },
+ { SDIO_DEVICE(0x024c, 0x0627), },
{ SDIO_DEVICE(0x024c, 0xb723), },
{ /* end: all zeroes */ },
};
#include "i2400m-usb.h"
#include "linux-wimax-i2400m.h"
#include <linux/debugfs.h>
+#include <linux/ethtool.h>
#include <linux/slab.h>
#include <linux/module.h>
void iscsit_aborted_task(struct iscsi_conn *conn, struct iscsi_cmd *cmd)
{
spin_lock_bh(&conn->cmd_lock);
- if (!list_empty(&cmd->i_conn_node) &&
- !(cmd->se_cmd.transport_state & CMD_T_FABRIC_STOP))
+ if (!list_empty(&cmd->i_conn_node))
list_del_init(&cmd->i_conn_node);
spin_unlock_bh(&conn->cmd_lock);
spin_lock_bh(&conn->cmd_lock);
list_splice_init(&conn->conn_cmd_list, &tmp_list);
- list_for_each_entry(cmd, &tmp_list, i_conn_node) {
+ list_for_each_entry_safe(cmd, cmd_tmp, &tmp_list, i_conn_node) {
struct se_cmd *se_cmd = &cmd->se_cmd;
if (se_cmd->se_tfo != NULL) {
spin_lock_irq(&se_cmd->t_state_lock);
- se_cmd->transport_state |= CMD_T_FABRIC_STOP;
+ if (se_cmd->transport_state & CMD_T_ABORTED) {
+ /*
+ * LIO's abort path owns the cleanup for this,
+ * so put it back on the list and let
+ * aborted_task handle it.
+ */
+ list_move_tail(&cmd->i_conn_node,
+ &conn->conn_cmd_list);
+ } else {
+ se_cmd->transport_state |= CMD_T_FABRIC_STOP;
+ }
spin_unlock_irq(&se_cmd->t_state_lock);
}
}
/**
* struct amdtee_context_data - AMD-TEE driver context data
* @sess_list: Keeps track of sessions opened in current TEE context
+ * @shm_list: Keeps track of buffers allocated and mapped in current TEE
+ * context
*/
struct amdtee_context_data {
struct list_head sess_list;
+ struct list_head shm_list;
+ struct mutex shm_mutex; /* synchronizes access to @shm_list */
};
struct amdtee_driver_data {
u32 buf_id;
};
-struct amdtee_shm_context {
- struct list_head shmdata_list;
-};
-
#define LOWER_TWO_BYTE_MASK 0x0000FFFF
/**
static struct amdtee_driver_data *drv_data;
static DEFINE_MUTEX(session_list_mutex);
-static struct amdtee_shm_context shmctx;
static void amdtee_get_version(struct tee_device *teedev,
struct tee_ioctl_version_data *vers)
return -ENOMEM;
INIT_LIST_HEAD(&ctxdata->sess_list);
- INIT_LIST_HEAD(&shmctx.shmdata_list);
+ INIT_LIST_HEAD(&ctxdata->shm_list);
+ mutex_init(&ctxdata->shm_mutex);
ctx->data = ctxdata;
return 0;
list_del(&sess->list_node);
release_session(sess);
}
+ mutex_destroy(&ctxdata->shm_mutex);
kfree(ctxdata);
ctx->data = NULL;
u32 get_buffer_id(struct tee_shm *shm)
{
- u32 buf_id = 0;
+ struct amdtee_context_data *ctxdata = shm->ctx->data;
struct amdtee_shm_data *shmdata;
+ u32 buf_id = 0;
- list_for_each_entry(shmdata, &shmctx.shmdata_list, shm_node)
+ mutex_lock(&ctxdata->shm_mutex);
+ list_for_each_entry(shmdata, &ctxdata->shm_list, shm_node)
if (shmdata->kaddr == shm->kaddr) {
buf_id = shmdata->buf_id;
break;
}
+ mutex_unlock(&ctxdata->shm_mutex);
return buf_id;
}
int amdtee_map_shmem(struct tee_shm *shm)
{
- struct shmem_desc shmem;
+ struct amdtee_context_data *ctxdata;
struct amdtee_shm_data *shmnode;
+ struct shmem_desc shmem;
int rc, count;
u32 buf_id;
shmnode->kaddr = shm->kaddr;
shmnode->buf_id = buf_id;
- list_add(&shmnode->shm_node, &shmctx.shmdata_list);
+ ctxdata = shm->ctx->data;
+ mutex_lock(&ctxdata->shm_mutex);
+ list_add(&shmnode->shm_node, &ctxdata->shm_list);
+ mutex_unlock(&ctxdata->shm_mutex);
pr_debug("buf_id :[%x] kaddr[%p]\n", shmnode->buf_id, shmnode->kaddr);
void amdtee_unmap_shmem(struct tee_shm *shm)
{
+ struct amdtee_context_data *ctxdata;
struct amdtee_shm_data *shmnode;
u32 buf_id;
/* Unmap the shared memory from TEE */
handle_unmap_shmem(buf_id);
- list_for_each_entry(shmnode, &shmctx.shmdata_list, shm_node)
+ ctxdata = shm->ctx->data;
+ mutex_lock(&ctxdata->shm_mutex);
+ list_for_each_entry(shmnode, &ctxdata->shm_list, shm_node)
if (buf_id == shmnode->buf_id) {
list_del(&shmnode->shm_node);
kfree(shmnode);
break;
}
+ mutex_unlock(&ctxdata->shm_mutex);
}
int amdtee_invoke_func(struct tee_context *ctx,
static bool is_normal_memory(pgprot_t p)
{
#if defined(CONFIG_ARM)
- return (pgprot_val(p) & L_PTE_MT_MASK) == L_PTE_MT_WRITEALLOC;
+ return (((pgprot_val(p) & L_PTE_MT_MASK) == L_PTE_MT_WRITEALLOC) ||
+ ((pgprot_val(p) & L_PTE_MT_MASK) == L_PTE_MT_WRITEBACK));
#elif defined(CONFIG_ARM64)
return (pgprot_val(p) & PTE_ATTRINDX_MASK) == PTE_ATTRINDX(MT_NORMAL);
#else
#include <linux/err.h>
#include <linux/types.h>
#include <linux/spinlock.h>
+#include <linux/sys_soc.h>
#include <linux/reboot.h>
#include <linux/of_device.h>
#include <linux/of_platform.h>
return bgp;
}
+/*
+ * List of SoCs on which the CPU PM notifier can cause erros on the DTEMP
+ * readout.
+ * Enabled notifier on these machines results in erroneous, random values which
+ * could trigger unexpected thermal shutdown.
+ */
+static const struct soc_device_attribute soc_no_cpu_notifier[] = {
+ { .machine = "OMAP4430" },
+ { /* sentinel */ },
+};
+
/*** Device driver call backs ***/
static
#ifdef CONFIG_PM_SLEEP
bgp->nb.notifier_call = bandgap_omap_cpu_notifier;
- cpu_pm_register_notifier(&bgp->nb);
+ if (!soc_device_match(soc_no_cpu_notifier))
+ cpu_pm_register_notifier(&bgp->nb);
#endif
return 0;
struct ti_bandgap *bgp = platform_get_drvdata(pdev);
int i;
- cpu_pm_unregister_notifier(&bgp->nb);
+ if (!soc_device_match(soc_no_cpu_notifier))
+ cpu_pm_unregister_notifier(&bgp->nb);
/* Remove sensor interfaces */
for (i = 0; i < bgp->conf->sensor_count; i++) {
#include <linux/debugfs.h>
#include <linux/pm_runtime.h>
+#include <linux/uaccess.h>
#include "tb.h"
case PCI_DEVICE_ID_INTEL_TGL_NHI0:
case PCI_DEVICE_ID_INTEL_TGL_NHI1:
+ case PCI_DEVICE_ID_INTEL_TGL_H_NHI0:
+ case PCI_DEVICE_ID_INTEL_TGL_H_NHI1:
icm->is_supported = icm_tgl_is_supported;
icm->driver_ready = icm_icl_driver_ready;
icm->set_uuid = icm_icl_set_uuid;
ring->vector = ret;
- ring->irq = pci_irq_vector(ring->nhi->pdev, ring->vector);
- if (ring->irq < 0)
- return ring->irq;
+ ret = pci_irq_vector(ring->nhi->pdev, ring->vector);
+ if (ret < 0)
+ goto err_ida_remove;
+
+ ring->irq = ret;
irqflags = no_suspend ? IRQF_NO_SUSPEND : 0;
- return request_irq(ring->irq, ring_msix, irqflags, "thunderbolt", ring);
+ ret = request_irq(ring->irq, ring_msix, irqflags, "thunderbolt", ring);
+ if (ret)
+ goto err_ida_remove;
+
+ return 0;
+
+err_ida_remove:
+ ida_simple_remove(&nhi->msix_ida, ring->vector);
+
+ return ret;
}
static void ring_release_msix(struct tb_ring *ring)
.driver_data = (kernel_ulong_t)&icl_nhi_ops },
{ PCI_VDEVICE(INTEL, PCI_DEVICE_ID_INTEL_TGL_NHI1),
.driver_data = (kernel_ulong_t)&icl_nhi_ops },
+ { PCI_VDEVICE(INTEL, PCI_DEVICE_ID_INTEL_TGL_H_NHI0),
+ .driver_data = (kernel_ulong_t)&icl_nhi_ops },
+ { PCI_VDEVICE(INTEL, PCI_DEVICE_ID_INTEL_TGL_H_NHI1),
+ .driver_data = (kernel_ulong_t)&icl_nhi_ops },
/* Any USB4 compliant host */
{ PCI_DEVICE_CLASS(PCI_CLASS_SERIAL_USB_USB4, ~0) },
#define PCI_DEVICE_ID_INTEL_ICL_NHI0 0x8a17
#define PCI_DEVICE_ID_INTEL_TGL_NHI0 0x9a1b
#define PCI_DEVICE_ID_INTEL_TGL_NHI1 0x9a1d
+#define PCI_DEVICE_ID_INTEL_TGL_H_NHI0 0x9a1f
+#define PCI_DEVICE_ID_INTEL_TGL_H_NHI1 0x9a21
#define PCI_CLASS_SERIAL_USB_USB4 0x0c0340
switch (sw->config.device_id) {
case PCI_DEVICE_ID_INTEL_TGL_NHI0:
case PCI_DEVICE_ID_INTEL_TGL_NHI1:
+ case PCI_DEVICE_ID_INTEL_TGL_H_NHI0:
+ case PCI_DEVICE_ID_INTEL_TGL_H_NHI1:
return true;
}
}
* upstream USB4 port.
*/
tb_switch_for_each_port(sw, port) {
+ if (!tb_port_is_null(port))
+ continue;
if (!route && tb_is_upstream_port(port))
continue;
+ if (!port->cap_usb4)
+ continue;
ret = tb_port_read(port, &val, TB_CFG_PORT,
port->cap_usb4 + PORT_CS_19, 1);
id = ida_simple_get(&xd->service_ids, 0, 0, GFP_KERNEL);
if (id < 0) {
+ kfree(svc->key);
kfree(svc);
break;
}
goto err_disable_clk;
up->gpios = mctrl_gpio_init(port, 0);
- if (IS_ERR(up->gpios) && PTR_ERR(up->gpios) != -ENOSYS)
- return PTR_ERR(up->gpios);
+ if (IS_ERR(up->gpios) && PTR_ERR(up->gpios) != -ENOSYS) {
+ ret = PTR_ERR(up->gpios);
+ goto err_disable_clk;
+ }
up->rts_gpiod = mctrl_gpio_to_gpiod(up->gpios, UART_GPIO_RTS);
struct imx_port *sport = dev_id;
unsigned int usr1, usr2, ucr1, ucr2, ucr3, ucr4;
irqreturn_t ret = IRQ_NONE;
+ unsigned long flags = 0;
- spin_lock(&sport->port.lock);
+ /*
+ * IRQs might not be disabled upon entering this interrupt handler,
+ * e.g. when interrupt handlers are forced to be threaded. To support
+ * this scenario as well, disable IRQs when acquiring the spinlock.
+ */
+ spin_lock_irqsave(&sport->port.lock, flags);
usr1 = imx_uart_readl(sport, USR1);
usr2 = imx_uart_readl(sport, USR2);
ret = IRQ_HANDLED;
}
- spin_unlock(&sport->port.lock);
+ spin_unlock_irqrestore(&sport->port.lock, flags);
return ret;
}
unsigned int ucr1;
unsigned long flags = 0;
int locked = 1;
- int retval;
-
- retval = clk_enable(sport->clk_per);
- if (retval)
- return;
- retval = clk_enable(sport->clk_ipg);
- if (retval) {
- clk_disable(sport->clk_per);
- return;
- }
if (sport->port.sysrq)
locked = 0;
if (locked)
spin_unlock_irqrestore(&sport->port.lock, flags);
-
- clk_disable(sport->clk_ipg);
- clk_disable(sport->clk_per);
}
/*
retval = uart_set_options(&sport->port, co, baud, parity, bits, flow);
- clk_disable(sport->clk_ipg);
if (retval) {
- clk_unprepare(sport->clk_ipg);
+ clk_disable_unprepare(sport->clk_ipg);
goto error_console;
}
- retval = clk_prepare(sport->clk_per);
+ retval = clk_prepare_enable(sport->clk_per);
if (retval)
- clk_unprepare(sport->clk_ipg);
+ clk_disable_unprepare(sport->clk_ipg);
error_console:
return retval;
return retval;
}
-static void uio_free_minor(struct uio_device *idev)
+static void uio_free_minor(unsigned long minor)
{
mutex_lock(&minor_lock);
- idr_remove(&uio_idr, idev->minor);
+ idr_remove(&uio_idr, minor);
mutex_unlock(&minor_lock);
}
err_uio_dev_add_attributes:
device_del(&idev->dev);
err_device_create:
- uio_free_minor(idev);
+ uio_free_minor(idev->minor);
put_device(&idev->dev);
return ret;
}
void uio_unregister_device(struct uio_info *info)
{
struct uio_device *idev;
+ unsigned long minor;
if (!info || !info->uio_dev)
return;
idev = info->uio_dev;
+ minor = idev->minor;
mutex_lock(&idev->info_lock);
uio_dev_del_attributes(idev);
device_unregister(&idev->dev);
- uio_free_minor(idev);
+ uio_free_minor(minor);
return;
}
struct cdns3_device *priv_dev = priv_ep->cdns3_dev;
struct cdns3_request *priv_req;
struct cdns3_trb *trb;
- struct cdns3_trb *link_trb;
+ struct cdns3_trb *link_trb = NULL;
dma_addr_t trb_dma;
u32 togle_pcs = 1;
int sg_iter = 0;
/* set incorrect Cycle Bit for first trb*/
control = priv_ep->pcs ? 0 : TRB_CYCLE;
+ trb->length = 0;
+ if (priv_dev->dev_ver >= DEV_VER_V2) {
+ u16 td_size;
+
+ td_size = DIV_ROUND_UP(request->length,
+ priv_ep->endpoint.maxpacket);
+ if (priv_dev->gadget.speed == USB_SPEED_SUPER)
+ trb->length = TRB_TDL_SS_SIZE(td_size);
+ else
+ control |= TRB_TDL_HS_SIZE(td_size);
+ }
do {
u32 length;
- u16 td_size = 0;
/* fill TRB */
control |= TRB_TYPE(TRB_NORMAL);
length = request->length;
}
- if (likely(priv_dev->dev_ver >= DEV_VER_V2))
- td_size = DIV_ROUND_UP(length,
- priv_ep->endpoint.maxpacket);
- else if (priv_ep->flags & EP_TDLCHK_EN)
+ if (priv_ep->flags & EP_TDLCHK_EN)
total_tdl += DIV_ROUND_UP(length,
priv_ep->endpoint.maxpacket);
- trb->length = cpu_to_le32(TRB_BURST_LEN(priv_ep->trb_burst_size) |
+ trb->length |= cpu_to_le32(TRB_BURST_LEN(priv_ep->trb_burst_size) |
TRB_LEN(length));
- if (priv_dev->gadget.speed == USB_SPEED_SUPER)
- trb->length |= cpu_to_le32(TRB_TDL_SS_SIZE(td_size));
- else
- control |= TRB_TDL_HS_SIZE(td_size);
-
pcs = priv_ep->pcs ? TRB_CYCLE : 0;
/*
{ USB_DEVICE(0x0870, 0x0001), /* Metricom GS Modem */
.driver_info = NO_UNION_NORMAL, /* has no union descriptor */
},
+ { USB_DEVICE(0x045b, 0x023c), /* Renesas USB Download mode */
+ .driver_info = DISABLE_ECHO, /* Don't echo banner */
+ },
+ { USB_DEVICE(0x045b, 0x0248), /* Renesas USB Download mode */
+ .driver_info = DISABLE_ECHO, /* Don't echo banner */
+ },
+ { USB_DEVICE(0x045b, 0x024D), /* Renesas USB Download mode */
+ .driver_info = DISABLE_ECHO, /* Don't echo banner */
+ },
{ USB_DEVICE(0x0e8d, 0x0003), /* FIREFLY, MediaTek Inc; andrey.arapov@gmail.com */
.driver_info = NO_UNION_NORMAL, /* has no union descriptor */
},
if (userurb) { /* Async */
if (when == SUBMIT)
- dev_info(&udev->dev, "userurb %pK, ep%d %s-%s, "
+ dev_info(&udev->dev, "userurb %px, ep%d %s-%s, "
"length %u\n",
userurb, ep, t, d, length);
else
- dev_info(&udev->dev, "userurb %pK, ep%d %s-%s, "
+ dev_info(&udev->dev, "userurb %px, ep%d %s-%s, "
"actual_length %u status %d\n",
userurb, ep, t, d, length,
timeout_or_status);
if (as) {
int retval;
- snoop(&ps->dev->dev, "reap %pK\n", as->userurb);
+ snoop(&ps->dev->dev, "reap %px\n", as->userurb);
retval = processcompl(as, (void __user * __user *)arg);
free_async(as);
return retval;
as = async_getcompleted(ps);
if (as) {
- snoop(&ps->dev->dev, "reap %pK\n", as->userurb);
+ snoop(&ps->dev->dev, "reap %px\n", as->userurb);
retval = processcompl(as, (void __user * __user *)arg);
free_async(as);
} else {
if (as) {
int retval;
- snoop(&ps->dev->dev, "reap %pK\n", as->userurb);
+ snoop(&ps->dev->dev, "reap %px\n", as->userurb);
retval = processcompl_compat(as, (void __user * __user *)arg);
free_async(as);
return retval;
as = async_getcompleted(ps);
if (as) {
- snoop(&ps->dev->dev, "reap %pK\n", as->userurb);
+ snoop(&ps->dev->dev, "reap %px\n", as->userurb);
retval = processcompl_compat(as, (void __user * __user *)arg);
free_async(as);
} else {
#endif
case USBDEVFS_DISCARDURB:
- snoop(&dev->dev, "%s: DISCARDURB %pK\n", __func__, p);
+ snoop(&dev->dev, "%s: DISCARDURB %px\n", __func__, p);
ret = proc_unlinkurb(ps, p);
break;
/* Guillemot Webcam Hercules Dualpix Exchange*/
{ USB_DEVICE(0x06f8, 0x3005), .driver_info = USB_QUIRK_RESET_RESUME },
+ /* Guillemot Hercules DJ Console audio card (BZ 208357) */
+ { USB_DEVICE(0x06f8, 0xb000), .driver_info =
+ USB_QUIRK_ENDPOINT_IGNORE },
+
/* Midiman M-Audio Keystation 88es */
{ USB_DEVICE(0x0763, 0x0192), .driver_info = USB_QUIRK_RESET_RESUME },
{ USB_DEVICE(0x1532, 0x0116), .driver_info =
USB_QUIRK_LINEAR_UFRAME_INTR_BINTERVAL },
+ /* Lenovo ThinkCenter A630Z TI024Gen3 usb-audio */
+ { USB_DEVICE(0x17ef, 0xa012), .driver_info =
+ USB_QUIRK_DISCONNECT_SUSPEND },
+
/* BUILDWIN Photo Frame */
{ USB_DEVICE(0x1908, 0x1315), .driver_info =
USB_QUIRK_HONOR_BNUMINTERFACES },
* Matched for devices with USB_QUIRK_ENDPOINT_IGNORE.
*/
static const struct usb_device_id usb_endpoint_ignore[] = {
+ { USB_DEVICE_INTERFACE_NUMBER(0x06f8, 0xb000, 5), .driver_info = 0x01 },
+ { USB_DEVICE_INTERFACE_NUMBER(0x06f8, 0xb000, 5), .driver_info = 0x81 },
{ USB_DEVICE_INTERFACE_NUMBER(0x0926, 0x0202, 1), .driver_info = 0x85 },
{ USB_DEVICE_INTERFACE_NUMBER(0x0926, 0x0208, 1), .driver_info = 0x85 },
{ }
midi->id = kstrdup(opts->id, GFP_KERNEL);
if (opts->id && !midi->id) {
status = -ENOMEM;
- goto setup_fail;
+ goto midi_free;
}
midi->in_ports = opts->in_ports;
midi->out_ports = opts->out_ports;
status = kfifo_alloc(&midi->in_req_fifo, midi->qlen, GFP_KERNEL);
if (status)
- goto setup_fail;
+ goto midi_free;
spin_lock_init(&midi->transmit_lock);
return &midi->func;
+midi_free:
+ if (midi)
+ kfree(midi->id);
+ kfree(midi);
setup_fail:
mutex_unlock(&opts->lock);
- kfree(midi);
+
return ERR_PTR(status);
}
return 0;
Enomem:
+ kfree(CHIP);
+ CHIP = NULL;
+
return -ENOMEM;
}
/* Initialize dma_mask and coherent_dma_mask to 32-bits */
ret = dma_set_mask_and_coherent(dev, DMA_BIT_MASK(32));
if (ret)
- return ret;
+ goto disable_pm;
hcd = usb_create_hcd(driver, dev, dev_name(dev));
if (!hcd) {
struct platform_device *parent = to_platform_device(dev->parent);
const struct dsps_musb_wrapper *wrp = glue->wrp;
void __iomem *reg_base;
+ struct resource *r;
u32 rev, val;
int ret;
- reg_base = devm_platform_ioremap_resource_byname(parent, "control");
+ r = platform_get_resource_byname(parent, IORESOURCE_MEM, "control");
+ reg_base = devm_ioremap_resource(dev, r);
if (IS_ERR(reg_base))
return PTR_ERR(reg_base);
musb->ctrl_base = reg_base;
config TYPEC_QCOM_PMIC
tristate "Qualcomm PMIC USB Type-C driver"
depends on ARCH_QCOM || COMPILE_TEST
+ depends on USB_ROLE_SWITCH || !USB_ROLE_SWITCH
help
Driver for supporting role switch over the Qualcomm PMIC. This will
handle the USB Type-C role and orientation detection reported by the
* Supported power operation mode can be configured through device tree
* else it is read from chip registers in stusb160x_get_caps.
*/
- ret = fwnode_property_read_string(fwnode, "power-opmode", &cap_str);
+ ret = fwnode_property_read_string(fwnode, "typec-power-opmode", &cap_str);
if (!ret) {
ret = typec_find_pwr_opmode(cap_str);
/* Power delivery not yet supported */
return;
power_supply_unregister(con->psy);
+ con->psy = NULL;
+}
+
+void ucsi_port_psy_changed(struct ucsi_connector *con)
+{
+ if (IS_ERR_OR_NULL(con->psy))
+ return;
+
+ power_supply_changed(con->psy);
}
role = !!(con->status.flags & UCSI_CONSTAT_PWR_DIR);
if (con->status.change & UCSI_CONSTAT_POWER_OPMODE_CHANGE ||
- con->status.change & UCSI_CONSTAT_POWER_LEVEL_CHANGE)
+ con->status.change & UCSI_CONSTAT_POWER_LEVEL_CHANGE) {
ucsi_pwr_opmode_change(con);
+ ucsi_port_psy_changed(con);
+ }
if (con->status.change & UCSI_CONSTAT_POWER_DIR_CHANGE) {
typec_set_pwr_role(con->port, role);
ucsi_register_partner(con);
else
ucsi_unregister_partner(con);
+
+ ucsi_port_psy_changed(con);
}
if (con->status.change & UCSI_CONSTAT_CAM_CHANGE) {
!!(con->status.flags & UCSI_CONSTAT_PWR_DIR));
ucsi_pwr_opmode_change(con);
ucsi_register_partner(con);
+ ucsi_port_psy_changed(con);
}
if (con->partner) {
#if IS_ENABLED(CONFIG_POWER_SUPPLY)
int ucsi_register_port_psy(struct ucsi_connector *con);
void ucsi_unregister_port_psy(struct ucsi_connector *con);
+void ucsi_port_psy_changed(struct ucsi_connector *con);
#else
static inline int ucsi_register_port_psy(struct ucsi_connector *con) { return 0; }
static inline void ucsi_unregister_port_psy(struct ucsi_connector *con) { }
+static inline void ucsi_port_psy_changed(struct ucsi_connector *con) { }
#endif /* CONFIG_POWER_SUPPLY */
#if IS_ENABLED(CONFIG_TYPEC_DP_ALTMODE)
depends on RUNTIME_TESTING_MENU && HAS_DMA
select DMA_OPS
select VHOST_RING
+ select GENERIC_NET_UTILS
default n
help
vDPA networking device simulator which loop TX traffic back
config MLX5_VDPA
bool
+ select VHOST_IOTLB
help
Support library for Mellanox VDPA drivers. Provides code that is
common for all types of VDPA drivers. The following drivers are planned:
#define VHOST_SCSI_VERSION "v0.1"
#define VHOST_SCSI_NAMELEN 256
#define VHOST_SCSI_MAX_CDB_SIZE 32
-#define VHOST_SCSI_DEFAULT_TAGS 256
#define VHOST_SCSI_PREALLOC_SGLS 2048
#define VHOST_SCSI_PREALLOC_UPAGES 2048
#define VHOST_SCSI_PREALLOC_PROT_SGLS 2048
struct se_portal_group se_tpg;
/* Pointer back to vhost_scsi, protected by tv_tpg_mutex */
struct vhost_scsi *vhost_scsi;
+ struct list_head tmf_queue;
};
struct vhost_scsi_tport {
* Writers must also take dev mutex and flush under it.
*/
int inflight_idx;
+ struct vhost_scsi_cmd *scsi_cmds;
+ struct sbitmap scsi_tags;
+ int max_cmds;
};
struct vhost_scsi {
int vs_events_nr; /* num of pending events, protected by vq->mutex */
};
+struct vhost_scsi_tmf {
+ struct vhost_work vwork;
+ struct vhost_scsi_tpg *tpg;
+ struct vhost_scsi *vhost;
+ struct vhost_scsi_virtqueue *svq;
+ struct list_head queue_entry;
+
+ struct se_cmd se_cmd;
+ u8 scsi_resp;
+ struct vhost_scsi_inflight *inflight;
+ struct iovec resp_iov;
+ int in_iovs;
+ int vq_desc;
+};
+
/*
* Context for processing request and control queue operations.
*/
return 1;
}
-static void vhost_scsi_release_cmd(struct se_cmd *se_cmd)
+static void vhost_scsi_release_cmd_res(struct se_cmd *se_cmd)
{
struct vhost_scsi_cmd *tv_cmd = container_of(se_cmd,
struct vhost_scsi_cmd, tvc_se_cmd);
- struct se_session *se_sess = tv_cmd->tvc_nexus->tvn_se_sess;
+ struct vhost_scsi_virtqueue *svq = container_of(tv_cmd->tvc_vq,
+ struct vhost_scsi_virtqueue, vq);
+ struct vhost_scsi_inflight *inflight = tv_cmd->inflight;
int i;
if (tv_cmd->tvc_sgl_count) {
put_page(sg_page(&tv_cmd->tvc_prot_sgl[i]));
}
- vhost_scsi_put_inflight(tv_cmd->inflight);
- target_free_tag(se_sess, se_cmd);
+ sbitmap_clear_bit(&svq->scsi_tags, se_cmd->map_tag);
+ vhost_scsi_put_inflight(inflight);
+}
+
+static void vhost_scsi_release_tmf_res(struct vhost_scsi_tmf *tmf)
+{
+ struct vhost_scsi_tpg *tpg = tmf->tpg;
+ struct vhost_scsi_inflight *inflight = tmf->inflight;
+
+ mutex_lock(&tpg->tv_tpg_mutex);
+ list_add_tail(&tpg->tmf_queue, &tmf->queue_entry);
+ mutex_unlock(&tpg->tv_tpg_mutex);
+ vhost_scsi_put_inflight(inflight);
+}
+
+static void vhost_scsi_release_cmd(struct se_cmd *se_cmd)
+{
+ if (se_cmd->se_cmd_flags & SCF_SCSI_TMR_CDB) {
+ struct vhost_scsi_tmf *tmf = container_of(se_cmd,
+ struct vhost_scsi_tmf, se_cmd);
+
+ vhost_work_queue(&tmf->vhost->dev, &tmf->vwork);
+ } else {
+ struct vhost_scsi_cmd *cmd = container_of(se_cmd,
+ struct vhost_scsi_cmd, tvc_se_cmd);
+ struct vhost_scsi *vs = cmd->tvc_vhost;
+
+ llist_add(&cmd->tvc_completion_list, &vs->vs_completion_list);
+ vhost_work_queue(&vs->dev, &vs->vs_completion_work);
+ }
}
static u32 vhost_scsi_sess_get_index(struct se_session *se_sess)
return 0;
}
-static void vhost_scsi_complete_cmd(struct vhost_scsi_cmd *cmd)
-{
- struct vhost_scsi *vs = cmd->tvc_vhost;
-
- llist_add(&cmd->tvc_completion_list, &vs->vs_completion_list);
-
- vhost_work_queue(&vs->dev, &vs->vs_completion_work);
-}
-
static int vhost_scsi_queue_data_in(struct se_cmd *se_cmd)
{
- struct vhost_scsi_cmd *cmd = container_of(se_cmd,
- struct vhost_scsi_cmd, tvc_se_cmd);
- vhost_scsi_complete_cmd(cmd);
+ transport_generic_free_cmd(se_cmd, 0);
return 0;
}
static int vhost_scsi_queue_status(struct se_cmd *se_cmd)
{
- struct vhost_scsi_cmd *cmd = container_of(se_cmd,
- struct vhost_scsi_cmd, tvc_se_cmd);
- vhost_scsi_complete_cmd(cmd);
+ transport_generic_free_cmd(se_cmd, 0);
return 0;
}
static void vhost_scsi_queue_tm_rsp(struct se_cmd *se_cmd)
{
- return;
+ struct vhost_scsi_tmf *tmf = container_of(se_cmd, struct vhost_scsi_tmf,
+ se_cmd);
+
+ tmf->scsi_resp = se_cmd->se_tmr_req->response;
+ transport_generic_free_cmd(&tmf->se_cmd, 0);
}
static void vhost_scsi_aborted_task(struct se_cmd *se_cmd)
return evt;
}
-static void vhost_scsi_free_cmd(struct vhost_scsi_cmd *cmd)
-{
- struct se_cmd *se_cmd = &cmd->tvc_se_cmd;
-
- /* TODO locking against target/backend threads? */
- transport_generic_free_cmd(se_cmd, 0);
-
-}
-
static int vhost_scsi_check_stop_free(struct se_cmd *se_cmd)
{
return target_put_sess_cmd(se_cmd);
} else
pr_err("Faulted on virtio_scsi_cmd_resp\n");
- vhost_scsi_free_cmd(cmd);
+ vhost_scsi_release_cmd_res(se_cmd);
}
vq = -1;
}
static struct vhost_scsi_cmd *
-vhost_scsi_get_tag(struct vhost_virtqueue *vq, struct vhost_scsi_tpg *tpg,
+vhost_scsi_get_cmd(struct vhost_virtqueue *vq, struct vhost_scsi_tpg *tpg,
unsigned char *cdb, u64 scsi_tag, u16 lun, u8 task_attr,
u32 exp_data_len, int data_direction)
{
+ struct vhost_scsi_virtqueue *svq = container_of(vq,
+ struct vhost_scsi_virtqueue, vq);
struct vhost_scsi_cmd *cmd;
struct vhost_scsi_nexus *tv_nexus;
- struct se_session *se_sess;
struct scatterlist *sg, *prot_sg;
struct page **pages;
- int tag, cpu;
+ int tag;
tv_nexus = tpg->tpg_nexus;
if (!tv_nexus) {
pr_err("Unable to locate active struct vhost_scsi_nexus\n");
return ERR_PTR(-EIO);
}
- se_sess = tv_nexus->tvn_se_sess;
- tag = sbitmap_queue_get(&se_sess->sess_tag_pool, &cpu);
+ tag = sbitmap_get(&svq->scsi_tags, 0, false);
if (tag < 0) {
pr_err("Unable to obtain tag for vhost_scsi_cmd\n");
return ERR_PTR(-ENOMEM);
}
- cmd = &((struct vhost_scsi_cmd *)se_sess->sess_cmd_map)[tag];
+ cmd = &svq->scsi_cmds[tag];
sg = cmd->tvc_sgl;
prot_sg = cmd->tvc_prot_sgl;
pages = cmd->tvc_upages;
cmd->tvc_prot_sgl = prot_sg;
cmd->tvc_upages = pages;
cmd->tvc_se_cmd.map_tag = tag;
- cmd->tvc_se_cmd.map_cpu = cpu;
cmd->tvc_tag = scsi_tag;
cmd->tvc_lun = lun;
cmd->tvc_task_attr = task_attr;
return ret;
}
+static u16 vhost_buf_to_lun(u8 *lun_buf)
+{
+ return ((lun_buf[2] << 8) | lun_buf[3]) & 0x3FFF;
+}
+
static void
vhost_scsi_handle_vq(struct vhost_scsi *vs, struct vhost_virtqueue *vq)
{
tag = vhost64_to_cpu(vq, v_req_pi.tag);
task_attr = v_req_pi.task_attr;
cdb = &v_req_pi.cdb[0];
- lun = ((v_req_pi.lun[2] << 8) | v_req_pi.lun[3]) & 0x3FFF;
+ lun = vhost_buf_to_lun(v_req_pi.lun);
} else {
tag = vhost64_to_cpu(vq, v_req.tag);
task_attr = v_req.task_attr;
cdb = &v_req.cdb[0];
- lun = ((v_req.lun[2] << 8) | v_req.lun[3]) & 0x3FFF;
+ lun = vhost_buf_to_lun(v_req.lun);
}
/*
* Check that the received CDB size does not exceeded our
scsi_command_size(cdb), VHOST_SCSI_MAX_CDB_SIZE);
goto err;
}
- cmd = vhost_scsi_get_tag(vq, tpg, cdb, tag, lun, task_attr,
+ cmd = vhost_scsi_get_cmd(vq, tpg, cdb, tag, lun, task_attr,
exp_data_len + prot_bytes,
data_direction);
if (IS_ERR(cmd)) {
- vq_err(vq, "vhost_scsi_get_tag failed %ld\n",
+ vq_err(vq, "vhost_scsi_get_cmd failed %ld\n",
PTR_ERR(cmd));
goto err;
}
&prot_iter, exp_data_len,
&data_iter))) {
vq_err(vq, "Failed to map iov to sgl\n");
- vhost_scsi_release_cmd(&cmd->tvc_se_cmd);
+ vhost_scsi_release_cmd_res(&cmd->tvc_se_cmd);
goto err;
}
}
}
static void
-vhost_scsi_send_tmf_reject(struct vhost_scsi *vs,
- struct vhost_virtqueue *vq,
- struct vhost_scsi_ctx *vc)
+vhost_scsi_send_tmf_resp(struct vhost_scsi *vs, struct vhost_virtqueue *vq,
+ int in_iovs, int vq_desc, struct iovec *resp_iov,
+ int tmf_resp_code)
{
struct virtio_scsi_ctrl_tmf_resp rsp;
struct iov_iter iov_iter;
pr_debug("%s\n", __func__);
memset(&rsp, 0, sizeof(rsp));
- rsp.response = VIRTIO_SCSI_S_FUNCTION_REJECTED;
+ rsp.response = tmf_resp_code;
- iov_iter_init(&iov_iter, READ, &vq->iov[vc->out], vc->in, sizeof(rsp));
+ iov_iter_init(&iov_iter, READ, resp_iov, in_iovs, sizeof(rsp));
ret = copy_to_iter(&rsp, sizeof(rsp), &iov_iter);
if (likely(ret == sizeof(rsp)))
- vhost_add_used_and_signal(&vs->dev, vq, vc->head, 0);
+ vhost_add_used_and_signal(&vs->dev, vq, vq_desc, 0);
else
pr_err("Faulted on virtio_scsi_ctrl_tmf_resp\n");
}
+static void vhost_scsi_tmf_resp_work(struct vhost_work *work)
+{
+ struct vhost_scsi_tmf *tmf = container_of(work, struct vhost_scsi_tmf,
+ vwork);
+ int resp_code;
+
+ if (tmf->scsi_resp == TMR_FUNCTION_COMPLETE)
+ resp_code = VIRTIO_SCSI_S_FUNCTION_SUCCEEDED;
+ else
+ resp_code = VIRTIO_SCSI_S_FUNCTION_REJECTED;
+
+ vhost_scsi_send_tmf_resp(tmf->vhost, &tmf->svq->vq, tmf->in_iovs,
+ tmf->vq_desc, &tmf->resp_iov, resp_code);
+ vhost_scsi_release_tmf_res(tmf);
+}
+
+static void
+vhost_scsi_handle_tmf(struct vhost_scsi *vs, struct vhost_scsi_tpg *tpg,
+ struct vhost_virtqueue *vq,
+ struct virtio_scsi_ctrl_tmf_req *vtmf,
+ struct vhost_scsi_ctx *vc)
+{
+ struct vhost_scsi_virtqueue *svq = container_of(vq,
+ struct vhost_scsi_virtqueue, vq);
+ struct vhost_scsi_tmf *tmf;
+
+ if (vhost32_to_cpu(vq, vtmf->subtype) !=
+ VIRTIO_SCSI_T_TMF_LOGICAL_UNIT_RESET)
+ goto send_reject;
+
+ if (!tpg->tpg_nexus || !tpg->tpg_nexus->tvn_se_sess) {
+ pr_err("Unable to locate active struct vhost_scsi_nexus for LUN RESET.\n");
+ goto send_reject;
+ }
+
+ mutex_lock(&tpg->tv_tpg_mutex);
+ if (list_empty(&tpg->tmf_queue)) {
+ pr_err("Missing reserve TMF. Could not handle LUN RESET.\n");
+ mutex_unlock(&tpg->tv_tpg_mutex);
+ goto send_reject;
+ }
+
+ tmf = list_first_entry(&tpg->tmf_queue, struct vhost_scsi_tmf,
+ queue_entry);
+ list_del_init(&tmf->queue_entry);
+ mutex_unlock(&tpg->tv_tpg_mutex);
+
+ tmf->tpg = tpg;
+ tmf->vhost = vs;
+ tmf->svq = svq;
+ tmf->resp_iov = vq->iov[vc->out];
+ tmf->vq_desc = vc->head;
+ tmf->in_iovs = vc->in;
+ tmf->inflight = vhost_scsi_get_inflight(vq);
+
+ if (target_submit_tmr(&tmf->se_cmd, tpg->tpg_nexus->tvn_se_sess, NULL,
+ vhost_buf_to_lun(vtmf->lun), NULL,
+ TMR_LUN_RESET, GFP_KERNEL, 0,
+ TARGET_SCF_ACK_KREF) < 0) {
+ vhost_scsi_release_tmf_res(tmf);
+ goto send_reject;
+ }
+
+ return;
+
+send_reject:
+ vhost_scsi_send_tmf_resp(vs, vq, vc->in, vc->head, &vq->iov[vc->out],
+ VIRTIO_SCSI_S_FUNCTION_REJECTED);
+}
+
static void
vhost_scsi_send_an_resp(struct vhost_scsi *vs,
struct vhost_virtqueue *vq,
static void
vhost_scsi_ctl_handle_vq(struct vhost_scsi *vs, struct vhost_virtqueue *vq)
{
+ struct vhost_scsi_tpg *tpg;
union {
__virtio32 type;
struct virtio_scsi_ctrl_an_req an;
vc.req += typ_size;
vc.req_size -= typ_size;
- ret = vhost_scsi_get_req(vq, &vc, NULL);
+ ret = vhost_scsi_get_req(vq, &vc, &tpg);
if (ret)
goto err;
if (v_req.type == VIRTIO_SCSI_T_TMF)
- vhost_scsi_send_tmf_reject(vs, vq, &vc);
+ vhost_scsi_handle_tmf(vs, tpg, vq, &v_req.tmf, &vc);
else
vhost_scsi_send_an_resp(vs, vq, &vc);
err:
wait_for_completion(&old_inflight[i]->comp);
}
+static void vhost_scsi_destroy_vq_cmds(struct vhost_virtqueue *vq)
+{
+ struct vhost_scsi_virtqueue *svq = container_of(vq,
+ struct vhost_scsi_virtqueue, vq);
+ struct vhost_scsi_cmd *tv_cmd;
+ unsigned int i;
+
+ if (!svq->scsi_cmds)
+ return;
+
+ for (i = 0; i < svq->max_cmds; i++) {
+ tv_cmd = &svq->scsi_cmds[i];
+
+ kfree(tv_cmd->tvc_sgl);
+ kfree(tv_cmd->tvc_prot_sgl);
+ kfree(tv_cmd->tvc_upages);
+ }
+
+ sbitmap_free(&svq->scsi_tags);
+ kfree(svq->scsi_cmds);
+ svq->scsi_cmds = NULL;
+}
+
+static int vhost_scsi_setup_vq_cmds(struct vhost_virtqueue *vq, int max_cmds)
+{
+ struct vhost_scsi_virtqueue *svq = container_of(vq,
+ struct vhost_scsi_virtqueue, vq);
+ struct vhost_scsi_cmd *tv_cmd;
+ unsigned int i;
+
+ if (svq->scsi_cmds)
+ return 0;
+
+ if (sbitmap_init_node(&svq->scsi_tags, max_cmds, -1, GFP_KERNEL,
+ NUMA_NO_NODE))
+ return -ENOMEM;
+ svq->max_cmds = max_cmds;
+
+ svq->scsi_cmds = kcalloc(max_cmds, sizeof(*tv_cmd), GFP_KERNEL);
+ if (!svq->scsi_cmds) {
+ sbitmap_free(&svq->scsi_tags);
+ return -ENOMEM;
+ }
+
+ for (i = 0; i < max_cmds; i++) {
+ tv_cmd = &svq->scsi_cmds[i];
+
+ tv_cmd->tvc_sgl = kcalloc(VHOST_SCSI_PREALLOC_SGLS,
+ sizeof(struct scatterlist),
+ GFP_KERNEL);
+ if (!tv_cmd->tvc_sgl) {
+ pr_err("Unable to allocate tv_cmd->tvc_sgl\n");
+ goto out;
+ }
+
+ tv_cmd->tvc_upages = kcalloc(VHOST_SCSI_PREALLOC_UPAGES,
+ sizeof(struct page *),
+ GFP_KERNEL);
+ if (!tv_cmd->tvc_upages) {
+ pr_err("Unable to allocate tv_cmd->tvc_upages\n");
+ goto out;
+ }
+
+ tv_cmd->tvc_prot_sgl = kcalloc(VHOST_SCSI_PREALLOC_PROT_SGLS,
+ sizeof(struct scatterlist),
+ GFP_KERNEL);
+ if (!tv_cmd->tvc_prot_sgl) {
+ pr_err("Unable to allocate tv_cmd->tvc_prot_sgl\n");
+ goto out;
+ }
+ }
+ return 0;
+out:
+ vhost_scsi_destroy_vq_cmds(vq);
+ return -ENOMEM;
+}
+
/*
* Called from vhost_scsi_ioctl() context to walk the list of available
* vhost_scsi_tpg with an active struct vhost_scsi_nexus
if (!strcmp(tv_tport->tport_name, t->vhost_wwpn)) {
if (vs->vs_tpg && vs->vs_tpg[tpg->tport_tpgt]) {
- kfree(vs_tpg);
mutex_unlock(&tpg->tv_tpg_mutex);
ret = -EEXIST;
- goto out;
+ goto undepend;
}
/*
* In order to ensure individual vhost-scsi configfs
ret = target_depend_item(&se_tpg->tpg_group.cg_item);
if (ret) {
pr_warn("target_depend_item() failed: %d\n", ret);
- kfree(vs_tpg);
mutex_unlock(&tpg->tv_tpg_mutex);
- goto out;
+ goto undepend;
}
tpg->tv_tpg_vhost_count++;
tpg->vhost_scsi = vs;
if (match) {
memcpy(vs->vs_vhost_wwpn, t->vhost_wwpn,
sizeof(vs->vs_vhost_wwpn));
+
+ for (i = VHOST_SCSI_VQ_IO; i < VHOST_SCSI_MAX_VQ; i++) {
+ vq = &vs->vqs[i].vq;
+ if (!vhost_vq_is_setup(vq))
+ continue;
+
+ if (vhost_scsi_setup_vq_cmds(vq, vq->num))
+ goto destroy_vq_cmds;
+ }
+
for (i = 0; i < VHOST_SCSI_MAX_VQ; i++) {
vq = &vs->vqs[i].vq;
mutex_lock(&vq->mutex);
vhost_scsi_flush(vs);
kfree(vs->vs_tpg);
vs->vs_tpg = vs_tpg;
+ goto out;
+destroy_vq_cmds:
+ for (i--; i >= VHOST_SCSI_VQ_IO; i--) {
+ if (!vhost_vq_get_backend(&vs->vqs[i].vq))
+ vhost_scsi_destroy_vq_cmds(&vs->vqs[i].vq);
+ }
+undepend:
+ for (i = 0; i < VHOST_SCSI_MAX_TARGET; i++) {
+ tpg = vs_tpg[i];
+ if (tpg) {
+ tpg->tv_tpg_vhost_count--;
+ target_undepend_item(&tpg->se_tpg.tpg_group.cg_item);
+ }
+ }
+ kfree(vs_tpg);
out:
mutex_unlock(&vs->dev.mutex);
mutex_unlock(&vhost_scsi_mutex);
mutex_lock(&vq->mutex);
vhost_vq_set_backend(vq, NULL);
mutex_unlock(&vq->mutex);
+ /*
+ * Make sure cmds are not running before tearing them
+ * down.
+ */
+ vhost_scsi_flush(vs);
+ vhost_scsi_destroy_vq_cmds(vq);
}
}
/*
{
struct vhost_scsi_tpg *tpg = container_of(se_tpg,
struct vhost_scsi_tpg, se_tpg);
+ struct vhost_scsi_tmf *tmf;
+
+ tmf = kzalloc(sizeof(*tmf), GFP_KERNEL);
+ if (!tmf)
+ return -ENOMEM;
+ INIT_LIST_HEAD(&tmf->queue_entry);
+ vhost_work_init(&tmf->vwork, vhost_scsi_tmf_resp_work);
mutex_lock(&vhost_scsi_mutex);
mutex_lock(&tpg->tv_tpg_mutex);
tpg->tv_tpg_port_count++;
+ list_add_tail(&tmf->queue_entry, &tpg->tmf_queue);
mutex_unlock(&tpg->tv_tpg_mutex);
vhost_scsi_hotplug(tpg, lun);
{
struct vhost_scsi_tpg *tpg = container_of(se_tpg,
struct vhost_scsi_tpg, se_tpg);
+ struct vhost_scsi_tmf *tmf;
mutex_lock(&vhost_scsi_mutex);
mutex_lock(&tpg->tv_tpg_mutex);
tpg->tv_tpg_port_count--;
+ tmf = list_first_entry(&tpg->tmf_queue, struct vhost_scsi_tmf,
+ queue_entry);
+ list_del(&tmf->queue_entry);
+ kfree(tmf);
mutex_unlock(&tpg->tv_tpg_mutex);
vhost_scsi_hotunplug(tpg, lun);
mutex_unlock(&vhost_scsi_mutex);
}
-static void vhost_scsi_free_cmd_map_res(struct se_session *se_sess)
-{
- struct vhost_scsi_cmd *tv_cmd;
- unsigned int i;
-
- if (!se_sess->sess_cmd_map)
- return;
-
- for (i = 0; i < VHOST_SCSI_DEFAULT_TAGS; i++) {
- tv_cmd = &((struct vhost_scsi_cmd *)se_sess->sess_cmd_map)[i];
-
- kfree(tv_cmd->tvc_sgl);
- kfree(tv_cmd->tvc_prot_sgl);
- kfree(tv_cmd->tvc_upages);
- }
-}
-
static ssize_t vhost_scsi_tpg_attrib_fabric_prot_type_store(
struct config_item *item, const char *page, size_t count)
{
NULL,
};
-static int vhost_scsi_nexus_cb(struct se_portal_group *se_tpg,
- struct se_session *se_sess, void *p)
-{
- struct vhost_scsi_cmd *tv_cmd;
- unsigned int i;
-
- for (i = 0; i < VHOST_SCSI_DEFAULT_TAGS; i++) {
- tv_cmd = &((struct vhost_scsi_cmd *)se_sess->sess_cmd_map)[i];
-
- tv_cmd->tvc_sgl = kcalloc(VHOST_SCSI_PREALLOC_SGLS,
- sizeof(struct scatterlist),
- GFP_KERNEL);
- if (!tv_cmd->tvc_sgl) {
- pr_err("Unable to allocate tv_cmd->tvc_sgl\n");
- goto out;
- }
-
- tv_cmd->tvc_upages = kcalloc(VHOST_SCSI_PREALLOC_UPAGES,
- sizeof(struct page *),
- GFP_KERNEL);
- if (!tv_cmd->tvc_upages) {
- pr_err("Unable to allocate tv_cmd->tvc_upages\n");
- goto out;
- }
-
- tv_cmd->tvc_prot_sgl = kcalloc(VHOST_SCSI_PREALLOC_PROT_SGLS,
- sizeof(struct scatterlist),
- GFP_KERNEL);
- if (!tv_cmd->tvc_prot_sgl) {
- pr_err("Unable to allocate tv_cmd->tvc_prot_sgl\n");
- goto out;
- }
- }
- return 0;
-out:
- vhost_scsi_free_cmd_map_res(se_sess);
- return -ENOMEM;
-}
-
static int vhost_scsi_make_nexus(struct vhost_scsi_tpg *tpg,
const char *name)
{
* struct se_node_acl for the vhost_scsi struct se_portal_group with
* the SCSI Initiator port name of the passed configfs group 'name'.
*/
- tv_nexus->tvn_se_sess = target_setup_session(&tpg->se_tpg,
- VHOST_SCSI_DEFAULT_TAGS,
- sizeof(struct vhost_scsi_cmd),
+ tv_nexus->tvn_se_sess = target_setup_session(&tpg->se_tpg, 0, 0,
TARGET_PROT_DIN_PASS | TARGET_PROT_DOUT_PASS,
- (unsigned char *)name, tv_nexus,
- vhost_scsi_nexus_cb);
+ (unsigned char *)name, tv_nexus, NULL);
if (IS_ERR(tv_nexus->tvn_se_sess)) {
mutex_unlock(&tpg->tv_tpg_mutex);
kfree(tv_nexus);
" %s Initiator Port: %s\n", vhost_scsi_dump_proto_id(tpg->tport),
tv_nexus->tvn_se_sess->se_node_acl->initiatorname);
- vhost_scsi_free_cmd_map_res(se_sess);
/*
* Release the SCSI I_T Nexus to the emulated vhost Target Port
*/
}
mutex_init(&tpg->tv_tpg_mutex);
INIT_LIST_HEAD(&tpg->tv_tpg_list);
+ INIT_LIST_HEAD(&tpg->tmf_queue);
tpg->tport = tport;
tpg->tport_tpgt = tpgt;
.last = v->range.last,
};
- return copy_to_user(argp, &range, sizeof(range));
+ if (copy_to_user(argp, &range, sizeof(range)))
+ return -EFAULT;
+ return 0;
}
static long vhost_vdpa_vring_ioctl(struct vhost_vdpa *v, unsigned int cmd,
if (r)
vhost_iotlb_del_range(dev->iotlb, iova, iova + size - 1);
+ else
+ atomic64_add(size >> PAGE_SHIFT, &dev->mm->pinned_vm);
return r;
}
unsigned long list_size = PAGE_SIZE / sizeof(struct page *);
unsigned int gup_flags = FOLL_LONGTERM;
unsigned long npages, cur_base, map_pfn, last_pfn = 0;
- unsigned long locked, lock_limit, pinned, i;
+ unsigned long lock_limit, sz2pin, nchunks, i;
u64 iova = msg->iova;
+ long pinned;
int ret = 0;
if (msg->iova < v->range.first ||
msg->iova + msg->size - 1))
return -EEXIST;
+ /* Limit the use of memory for bookkeeping */
page_list = (struct page **) __get_free_page(GFP_KERNEL);
if (!page_list)
return -ENOMEM;
gup_flags |= FOLL_WRITE;
npages = PAGE_ALIGN(msg->size + (iova & ~PAGE_MASK)) >> PAGE_SHIFT;
- if (!npages)
- return -EINVAL;
+ if (!npages) {
+ ret = -EINVAL;
+ goto free;
+ }
mmap_read_lock(dev->mm);
- locked = atomic64_add_return(npages, &dev->mm->pinned_vm);
lock_limit = rlimit(RLIMIT_MEMLOCK) >> PAGE_SHIFT;
-
- if (locked > lock_limit) {
+ if (npages + atomic64_read(&dev->mm->pinned_vm) > lock_limit) {
ret = -ENOMEM;
- goto out;
+ goto unlock;
}
cur_base = msg->uaddr & PAGE_MASK;
iova &= PAGE_MASK;
+ nchunks = 0;
while (npages) {
- pinned = min_t(unsigned long, npages, list_size);
- ret = pin_user_pages(cur_base, pinned,
- gup_flags, page_list, NULL);
- if (ret != pinned)
+ sz2pin = min_t(unsigned long, npages, list_size);
+ pinned = pin_user_pages(cur_base, sz2pin,
+ gup_flags, page_list, NULL);
+ if (sz2pin != pinned) {
+ if (pinned < 0) {
+ ret = pinned;
+ } else {
+ unpin_user_pages(page_list, pinned);
+ ret = -ENOMEM;
+ }
goto out;
+ }
+ nchunks++;
if (!last_pfn)
map_pfn = page_to_pfn(page_list[0]);
- for (i = 0; i < ret; i++) {
+ for (i = 0; i < pinned; i++) {
unsigned long this_pfn = page_to_pfn(page_list[i]);
u64 csize;
if (last_pfn && (this_pfn != last_pfn + 1)) {
/* Pin a contiguous chunk of memory */
csize = (last_pfn - map_pfn + 1) << PAGE_SHIFT;
- if (vhost_vdpa_map(v, iova, csize,
- map_pfn << PAGE_SHIFT,
- msg->perm))
+ ret = vhost_vdpa_map(v, iova, csize,
+ map_pfn << PAGE_SHIFT,
+ msg->perm);
+ if (ret) {
+ /*
+ * Unpin the pages that are left unmapped
+ * from this point on in the current
+ * page_list. The remaining outstanding
+ * ones which may stride across several
+ * chunks will be covered in the common
+ * error path subsequently.
+ */
+ unpin_user_pages(&page_list[i],
+ pinned - i);
goto out;
+ }
+
map_pfn = this_pfn;
iova += csize;
+ nchunks = 0;
}
last_pfn = this_pfn;
}
- cur_base += ret << PAGE_SHIFT;
- npages -= ret;
+ cur_base += pinned << PAGE_SHIFT;
+ npages -= pinned;
}
/* Pin the rest chunk */
map_pfn << PAGE_SHIFT, msg->perm);
out:
if (ret) {
+ if (nchunks) {
+ unsigned long pfn;
+
+ /*
+ * Unpin the outstanding pages which are yet to be
+ * mapped but haven't due to vdpa_map() or
+ * pin_user_pages() failure.
+ *
+ * Mapped pages are accounted in vdpa_map(), hence
+ * the corresponding unpinning will be handled by
+ * vdpa_unmap().
+ */
+ WARN_ON(!last_pfn);
+ for (pfn = map_pfn; pfn <= last_pfn; pfn++)
+ unpin_user_page(pfn_to_page(pfn));
+ }
vhost_vdpa_unmap(v, msg->iova, msg->size);
- atomic64_sub(npages, &dev->mm->pinned_vm);
}
+unlock:
mmap_read_unlock(dev->mm);
+free:
free_page((unsigned long)page_list);
return ret;
}
memset(&call_ctx->producer, 0x0, sizeof(struct irq_bypass_producer));
}
+bool vhost_vq_is_setup(struct vhost_virtqueue *vq)
+{
+ return vq->avail && vq->desc && vq->used && vhost_vq_access_ok(vq);
+}
+EXPORT_SYMBOL_GPL(vhost_vq_is_setup);
+
static void vhost_vq_reset(struct vhost_dev *dev,
struct vhost_virtqueue *vq)
{
struct vhost_log *log, unsigned int *log_num);
void vhost_discard_vq_desc(struct vhost_virtqueue *, int n);
+bool vhost_vq_is_setup(struct vhost_virtqueue *vq);
int vhost_vq_init_access(struct vhost_virtqueue *);
int vhost_add_used(struct vhost_virtqueue *, unsigned int head, int len);
int vhost_add_used_n(struct vhost_virtqueue *, struct vring_used_elem *heads,
/**
* vringh_iov_push_user - copy bytes into vring_iov.
* @wiov: the wiov as passed to vringh_getdesc_user() (updated as we consume)
- * @dst: the place to copy.
+ * @src: the place to copy from.
* @len: the maximum length to copy.
*
* Returns the bytes copied <= len or a negative errno.
/**
* vringh_iov_push_kern - copy bytes into vring_iov.
* @wiov: the wiov as passed to vringh_getdesc_kern() (updated as we consume)
- * @dst: the place to copy.
+ * @src: the place to copy from.
* @len: the maximum length to copy.
*
* Returns the bytes copied <= len or a negative errno.
* vringh_iov_push_iotlb - copy bytes into vring_iov.
* @vrh: the vring.
* @wiov: the wiov as passed to vringh_getdesc_iotlb() (updated as we consume)
- * @dst: the place to copy.
+ * @src: the place to copy from.
* @len: the maximum length to copy.
*
* Returns the bytes copied <= len or a negative errno.
#include <linux/module.h>
#include <linux/kernel.h>
+#include <linux/vmalloc.h>
#include <linux/init.h>
#include <linux/completion.h>
#include <linux/fb.h>
goto err1;
}
- fb_virt = ioremap(par->mem->start, screen_fb_size);
+ /*
+ * Map the VRAM cacheable for performance. This is also required for
+ * VM Connect to display properly for ARM64 Linux VM, as the host also
+ * maps the VRAM cacheable.
+ */
+ fb_virt = ioremap_cache(par->mem->start, screen_fb_size);
if (!fb_virt)
goto err2;
poll_wait(file, &ne_enclave->eventq, wait);
- if (!ne_enclave->has_event)
- return mask;
-
- mask = POLLHUP;
+ if (ne_enclave->has_event)
+ mask |= EPOLLHUP;
return mask;
}
.release = v9fs_dir_release,
.lock = v9fs_file_lock,
.mmap = v9fs_file_mmap,
+ .splice_read = generic_file_splice_read,
+ .splice_write = iter_file_splice_write,
.fsync = v9fs_file_fsync,
};
.lock = v9fs_file_lock_dotl,
.flock = v9fs_file_flock_dotl,
.mmap = v9fs_file_mmap,
+ .splice_read = generic_file_splice_read,
+ .splice_write = iter_file_splice_write,
.fsync = v9fs_file_fsync_dotl,
};
.release = v9fs_dir_release,
.lock = v9fs_file_lock,
.mmap = generic_file_readonly_mmap,
+ .splice_read = generic_file_splice_read,
+ .splice_write = iter_file_splice_write,
.fsync = v9fs_file_fsync,
};
.lock = v9fs_file_lock_dotl,
.flock = v9fs_file_flock_dotl,
.mmap = generic_file_readonly_mmap,
+ .splice_read = generic_file_splice_read,
+ .splice_write = iter_file_splice_write,
.fsync = v9fs_file_fsync_dotl,
};
.release = v9fs_dir_release,
.lock = v9fs_file_lock,
.mmap = v9fs_mmap_file_mmap,
+ .splice_read = generic_file_splice_read,
+ .splice_write = iter_file_splice_write,
.fsync = v9fs_file_fsync,
};
.lock = v9fs_file_lock_dotl,
.flock = v9fs_file_flock_dotl,
.mmap = v9fs_mmap_file_mmap,
+ .splice_read = generic_file_splice_read,
+ .splice_write = iter_file_splice_write,
.fsync = v9fs_file_fsync_dotl,
};
vp->cb_break_before = afs_calc_vnode_cb_break(vnode);
vp->vnode = vnode;
vp->put_vnode = true;
+ vp->speculative = true; /* vnode not locked */
}
}
}
op->flags &= ~AFS_OPERATION_DIR_CONFLICT;
}
} else if (vp->scb.have_status) {
+ if (vp->dv_before + vp->dv_delta != vp->scb.status.data_version &&
+ vp->speculative)
+ /* Ignore the result of a speculative bulk status fetch
+ * if it splits around a modification op, thereby
+ * appearing to regress the data version.
+ */
+ goto out;
afs_apply_status(op, vp);
if (vp->scb.have_cb)
afs_apply_callback(op, vp);
}
}
+out:
write_sequnlock(&vnode->cb_lock);
if (vp->scb.have_status)
bool update_ctime:1; /* Need to update the ctime */
bool set_size:1; /* Must update i_size */
bool op_unlinked:1; /* True if file was unlinked by op */
+ bool speculative:1; /* T if speculative status fetch (no vnode lock) */
};
/*
unsigned int f, from = pos & (PAGE_SIZE - 1);
unsigned int t, to = from + copied;
loff_t i_size, maybe_i_size;
- int ret;
+ int ret = 0;
_enter("{%llx:%llu},{%lx}",
vnode->fid.vid, vnode->fid.vnode, page->index);
+ if (copied == 0)
+ goto out;
+
maybe_i_size = pos + copied;
i_size = i_size_read(&vnode->vfs_inode);
* we return to userspace.
*/
if (S_ISREG(file_inode(file)->i_mode)) {
- __sb_start_write(file_inode(file)->i_sb, SB_FREEZE_WRITE, true);
+ sb_start_write(file_inode(file)->i_sb);
__sb_writers_release(file_inode(file)->i_sb, SB_FREEZE_WRITE);
}
req->ki_flags |= IOCB_WRITE;
*/
struct ulist *qgroup_ulist;
- /* protect user change for quota operations */
+ /*
+ * Protect user change for quota operations. If a transaction is needed,
+ * it must be started before locking this lock.
+ */
struct mutex qgroup_ioctl_lock;
/* list of dirty qgroups to be written at next commit */
}
}
-static int btrfs_find_new_delalloc_bytes(struct btrfs_inode *inode,
- const u64 start,
- const u64 len,
- struct extent_state **cached_state)
-{
- u64 search_start = start;
- const u64 end = start + len - 1;
-
- while (search_start < end) {
- const u64 search_len = end - search_start + 1;
- struct extent_map *em;
- u64 em_len;
- int ret = 0;
-
- em = btrfs_get_extent(inode, NULL, 0, search_start, search_len);
- if (IS_ERR(em))
- return PTR_ERR(em);
-
- if (em->block_start != EXTENT_MAP_HOLE)
- goto next;
-
- em_len = em->len;
- if (em->start < search_start)
- em_len -= search_start - em->start;
- if (em_len > search_len)
- em_len = search_len;
-
- ret = set_extent_bit(&inode->io_tree, search_start,
- search_start + em_len - 1,
- EXTENT_DELALLOC_NEW,
- NULL, cached_state, GFP_NOFS);
-next:
- search_start = extent_map_end(em);
- free_extent_map(em);
- if (ret)
- return ret;
- }
- return 0;
-}
-
/*
* after copy_from_user, pages need to be dirtied and we need to make
* sure holes are created between the current EOF and the start of
EXTENT_DELALLOC | EXTENT_DO_ACCOUNTING | EXTENT_DEFRAG,
0, 0, cached);
- if (!btrfs_is_free_space_inode(inode)) {
- if (start_pos >= isize &&
- !(inode->flags & BTRFS_INODE_PREALLOC)) {
- /*
- * There can't be any extents following eof in this case
- * so just set the delalloc new bit for the range
- * directly.
- */
- extra_bits |= EXTENT_DELALLOC_NEW;
- } else {
- err = btrfs_find_new_delalloc_bytes(inode, start_pos,
- num_bytes, cached);
- if (err)
- return err;
- }
- }
-
err = btrfs_set_extent_delalloc(inode, start_pos, end_of_last_block,
extra_bits, cached);
if (err)
return 0;
}
+static int btrfs_find_new_delalloc_bytes(struct btrfs_inode *inode,
+ const u64 start,
+ const u64 len,
+ struct extent_state **cached_state)
+{
+ u64 search_start = start;
+ const u64 end = start + len - 1;
+
+ while (search_start < end) {
+ const u64 search_len = end - search_start + 1;
+ struct extent_map *em;
+ u64 em_len;
+ int ret = 0;
+
+ em = btrfs_get_extent(inode, NULL, 0, search_start, search_len);
+ if (IS_ERR(em))
+ return PTR_ERR(em);
+
+ if (em->block_start != EXTENT_MAP_HOLE)
+ goto next;
+
+ em_len = em->len;
+ if (em->start < search_start)
+ em_len -= search_start - em->start;
+ if (em_len > search_len)
+ em_len = search_len;
+
+ ret = set_extent_bit(&inode->io_tree, search_start,
+ search_start + em_len - 1,
+ EXTENT_DELALLOC_NEW,
+ NULL, cached_state, GFP_NOFS);
+next:
+ search_start = extent_map_end(em);
+ free_extent_map(em);
+ if (ret)
+ return ret;
+ }
+ return 0;
+}
+
int btrfs_set_extent_delalloc(struct btrfs_inode *inode, u64 start, u64 end,
unsigned int extra_bits,
struct extent_state **cached_state)
{
WARN_ON(PAGE_ALIGNED(end));
+
+ if (start >= i_size_read(&inode->vfs_inode) &&
+ !(inode->flags & BTRFS_INODE_PREALLOC)) {
+ /*
+ * There can't be any extents following eof in this case so just
+ * set the delalloc new bit for the range directly.
+ */
+ extra_bits |= EXTENT_DELALLOC_NEW;
+ } else {
+ int ret;
+
+ ret = btrfs_find_new_delalloc_bytes(inode, start,
+ end + 1 - start,
+ cached_state);
+ if (ret)
+ return ret;
+ }
+
return set_extent_delalloc(&inode->io_tree, start, end, extra_bits,
cached_state);
}
#include <linux/slab.h>
#include <linux/workqueue.h>
#include <linux/btrfs.h>
+#include <linux/sched/mm.h>
#include "ctree.h"
#include "transaction.h"
break;
}
out:
+ btrfs_free_path(path);
fs_info->qgroup_flags |= flags;
if (!(fs_info->qgroup_flags & BTRFS_QGROUP_STATUS_FLAG_ON))
clear_bit(BTRFS_FS_QUOTA_ENABLED, &fs_info->flags);
else if (fs_info->qgroup_flags & BTRFS_QGROUP_STATUS_FLAG_RESCAN &&
ret >= 0)
ret = qgroup_rescan_init(fs_info, rescan_progress, 0);
- btrfs_free_path(path);
if (ret < 0) {
ulist_free(fs_info->qgroup_ulist);
struct btrfs_key found_key;
struct btrfs_qgroup *qgroup = NULL;
struct btrfs_trans_handle *trans = NULL;
+ struct ulist *ulist = NULL;
int ret = 0;
int slot;
if (fs_info->quota_root)
goto out;
- fs_info->qgroup_ulist = ulist_alloc(GFP_KERNEL);
- if (!fs_info->qgroup_ulist) {
+ ulist = ulist_alloc(GFP_KERNEL);
+ if (!ulist) {
ret = -ENOMEM;
goto out;
}
ret = btrfs_sysfs_add_qgroups(fs_info);
if (ret < 0)
goto out;
+
+ /*
+ * Unlock qgroup_ioctl_lock before starting the transaction. This is to
+ * avoid lock acquisition inversion problems (reported by lockdep) between
+ * qgroup_ioctl_lock and the vfs freeze semaphores, acquired when we
+ * start a transaction.
+ * After we started the transaction lock qgroup_ioctl_lock again and
+ * check if someone else created the quota root in the meanwhile. If so,
+ * just return success and release the transaction handle.
+ *
+ * Also we don't need to worry about someone else calling
+ * btrfs_sysfs_add_qgroups() after we unlock and getting an error because
+ * that function returns 0 (success) when the sysfs entries already exist.
+ */
+ mutex_unlock(&fs_info->qgroup_ioctl_lock);
+
/*
* 1 for quota root item
* 1 for BTRFS_QGROUP_STATUS item
* would be a lot of overkill.
*/
trans = btrfs_start_transaction(tree_root, 2);
+
+ mutex_lock(&fs_info->qgroup_ioctl_lock);
if (IS_ERR(trans)) {
ret = PTR_ERR(trans);
trans = NULL;
goto out;
}
+ if (fs_info->quota_root)
+ goto out;
+
+ fs_info->qgroup_ulist = ulist;
+ ulist = NULL;
+
/*
* initially create the quota tree
*/
if (ret) {
ulist_free(fs_info->qgroup_ulist);
fs_info->qgroup_ulist = NULL;
- if (trans)
- btrfs_end_transaction(trans);
btrfs_sysfs_del_qgroups(fs_info);
}
mutex_unlock(&fs_info->qgroup_ioctl_lock);
+ if (ret && trans)
+ btrfs_end_transaction(trans);
+ else if (trans)
+ ret = btrfs_end_transaction(trans);
+ ulist_free(ulist);
return ret;
}
mutex_lock(&fs_info->qgroup_ioctl_lock);
if (!fs_info->quota_root)
goto out;
+ mutex_unlock(&fs_info->qgroup_ioctl_lock);
/*
* 1 For the root item
*
* We should also reserve enough items for the quota tree deletion in
* btrfs_clean_quota_tree but this is not done.
+ *
+ * Also, we must always start a transaction without holding the mutex
+ * qgroup_ioctl_lock, see btrfs_quota_enable().
*/
trans = btrfs_start_transaction(fs_info->tree_root, 1);
+
+ mutex_lock(&fs_info->qgroup_ioctl_lock);
if (IS_ERR(trans)) {
ret = PTR_ERR(trans);
+ trans = NULL;
goto out;
}
+ if (!fs_info->quota_root)
+ goto out;
+
clear_bit(BTRFS_FS_QUOTA_ENABLED, &fs_info->flags);
btrfs_qgroup_wait_for_completion(fs_info, false);
spin_lock(&fs_info->qgroup_lock);
ret = btrfs_clean_quota_tree(trans, quota_root);
if (ret) {
btrfs_abort_transaction(trans, ret);
- goto end_trans;
+ goto out;
}
ret = btrfs_del_root(trans, "a_root->root_key);
if (ret) {
btrfs_abort_transaction(trans, ret);
- goto end_trans;
+ goto out;
}
list_del("a_root->dirty_list);
btrfs_put_root(quota_root);
-end_trans:
- ret = btrfs_end_transaction(trans);
out:
mutex_unlock(&fs_info->qgroup_ioctl_lock);
+ if (ret && trans)
+ btrfs_end_transaction(trans);
+ else if (trans)
+ ret = btrfs_end_transaction(trans);
+
return ret;
}
struct btrfs_qgroup *member;
struct btrfs_qgroup_list *list;
struct ulist *tmp;
+ unsigned int nofs_flag;
int ret = 0;
/* Check the level of src and dst first */
if (btrfs_qgroup_level(src) >= btrfs_qgroup_level(dst))
return -EINVAL;
+ /* We hold a transaction handle open, must do a NOFS allocation. */
+ nofs_flag = memalloc_nofs_save();
tmp = ulist_alloc(GFP_KERNEL);
+ memalloc_nofs_restore(nofs_flag);
if (!tmp)
return -ENOMEM;
struct btrfs_qgroup_list *list;
struct ulist *tmp;
bool found = false;
+ unsigned int nofs_flag;
int ret = 0;
int ret2;
+ /* We hold a transaction handle open, must do a NOFS allocation. */
+ nofs_flag = memalloc_nofs_save();
tmp = ulist_alloc(GFP_KERNEL);
+ memalloc_nofs_restore(nofs_flag);
if (!tmp)
return -ENOMEM;
{
struct btrfs_trans_handle *trans;
int ret;
+ bool can_commit = true;
/*
* We don't want to run flush again and again, so if there is a running
return 0;
}
+ /*
+ * If current process holds a transaction, we shouldn't flush, as we
+ * assume all space reservation happens before a transaction handle is
+ * held.
+ *
+ * But there are cases like btrfs_delayed_item_reserve_metadata() where
+ * we try to reserve space with one transction handle already held.
+ * In that case we can't commit transaction, but at least try to end it
+ * and hope the started data writes can free some space.
+ */
+ if (current->journal_info &&
+ current->journal_info != BTRFS_SEND_TRANS_STUB)
+ can_commit = false;
+
ret = btrfs_start_delalloc_snapshot(root);
if (ret < 0)
goto out;
goto out;
}
- ret = btrfs_commit_transaction(trans);
+ if (can_commit)
+ ret = btrfs_commit_transaction(trans);
+ else
+ ret = btrfs_end_transaction(trans);
out:
clear_bit(BTRFS_ROOT_QGROUP_FLUSHING, &root->state);
wake_up(&root->qgroup_flush_wait);
ret = clear_extent_bit(&BTRFS_I(inode)->io_tree,
BTRFS_MAX_EXTENT_SIZE >> 1,
(BTRFS_MAX_EXTENT_SIZE >> 1) + sectorsize - 1,
- EXTENT_DELALLOC | EXTENT_UPTODATE, 0, 0, NULL);
+ EXTENT_DELALLOC | EXTENT_DELALLOC_NEW |
+ EXTENT_UPTODATE, 0, 0, NULL);
if (ret) {
test_err("clear_extent_bit returned %d", ret);
goto out;
ret = clear_extent_bit(&BTRFS_I(inode)->io_tree,
BTRFS_MAX_EXTENT_SIZE + sectorsize,
BTRFS_MAX_EXTENT_SIZE + 2 * sectorsize - 1,
- EXTENT_DELALLOC | EXTENT_UPTODATE, 0, 0, NULL);
+ EXTENT_DELALLOC | EXTENT_DELALLOC_NEW |
+ EXTENT_UPTODATE, 0, 0, NULL);
if (ret) {
test_err("clear_extent_bit returned %d", ret);
goto out;
/* Empty */
ret = clear_extent_bit(&BTRFS_I(inode)->io_tree, 0, (u64)-1,
- EXTENT_DELALLOC | EXTENT_UPTODATE, 0, 0, NULL);
+ EXTENT_DELALLOC | EXTENT_DELALLOC_NEW |
+ EXTENT_UPTODATE, 0, 0, NULL);
if (ret) {
test_err("clear_extent_bit returned %d", ret);
goto out;
out:
if (ret)
clear_extent_bit(&BTRFS_I(inode)->io_tree, 0, (u64)-1,
- EXTENT_DELALLOC | EXTENT_UPTODATE, 0, 0, NULL);
+ EXTENT_DELALLOC | EXTENT_DELALLOC_NEW |
+ EXTENT_UPTODATE, 0, 0, NULL);
iput(inode);
btrfs_free_dummy_root(root);
btrfs_free_dummy_fs_info(fs_info);
"invalid root item size, have %u expect %zu or %u",
btrfs_item_size_nr(leaf, slot), sizeof(ri),
btrfs_legacy_root_item_size());
+ return -EUCLEAN;
}
/*
"invalid item size, have %u expect aligned to %zu for key type %u",
btrfs_item_size_nr(leaf, slot),
sizeof(*dref), key->type);
+ return -EUCLEAN;
}
if (!IS_ALIGNED(key->objectid, leaf->fs_info->sectorsize)) {
generic_err(leaf, slot,
extent_err(leaf, slot,
"invalid extent data backref offset, have %llu expect aligned to %u",
offset, leaf->fs_info->sectorsize);
+ return -EUCLEAN;
}
}
return 0;
if (device->bdev != path_bdev) {
bdput(path_bdev);
mutex_unlock(&fs_devices->device_list_mutex);
- btrfs_warn_in_rcu(device->fs_info,
+ /*
+ * device->fs_info may not be reliable here, so
+ * pass in a NULL instead. This avoids a
+ * possible use-after-free when the fs_info and
+ * fs_info->sb are already torn down.
+ */
+ btrfs_warn_in_rcu(NULL,
"duplicate device %s devid %llu generation %llu scanned by %s (%d)",
path, devid, found_transid,
current->comm,
cifs_dbg(VFS, "%s: error %d getting sec desc\n", __func__, rc);
} else if (mode_from_special_sid) {
rc = parse_sec_desc(cifs_sb, pntsd, acllen, fattr, true);
+ kfree(pntsd);
} else {
/* get approximated mode from ACL */
rc = parse_sec_desc(cifs_sb, pntsd, acllen, fattr, false);
list_del_init(&server->tcp_ses_list);
spin_unlock(&cifs_tcp_ses_lock);
+ cancel_delayed_work_sync(&server->echo);
+
spin_lock(&GlobalMid_Lock);
server->tcpStatus = CifsExiting;
spin_unlock(&GlobalMid_Lock);
}
static struct mid_q_entry *
-smb2_find_mid(struct TCP_Server_Info *server, char *buf)
+__smb2_find_mid(struct TCP_Server_Info *server, char *buf, bool dequeue)
{
struct mid_q_entry *mid;
struct smb2_sync_hdr *shdr = (struct smb2_sync_hdr *)buf;
(mid->mid_state == MID_REQUEST_SUBMITTED) &&
(mid->command == shdr->Command)) {
kref_get(&mid->refcount);
+ if (dequeue) {
+ list_del_init(&mid->qhead);
+ mid->mid_flags |= MID_DELETED;
+ }
spin_unlock(&GlobalMid_Lock);
return mid;
}
return NULL;
}
+static struct mid_q_entry *
+smb2_find_mid(struct TCP_Server_Info *server, char *buf)
+{
+ return __smb2_find_mid(server, buf, false);
+}
+
+static struct mid_q_entry *
+smb2_find_dequeue_mid(struct TCP_Server_Info *server, char *buf)
+{
+ return __smb2_find_mid(server, buf, true);
+}
+
static void
smb2_dump_detail(void *buf, struct TCP_Server_Info *server)
{
static int
handle_read_data(struct TCP_Server_Info *server, struct mid_q_entry *mid,
char *buf, unsigned int buf_len, struct page **pages,
- unsigned int npages, unsigned int page_data_size)
+ unsigned int npages, unsigned int page_data_size,
+ bool is_offloaded)
{
unsigned int data_offset;
unsigned int data_len;
if (server->ops->is_session_expired &&
server->ops->is_session_expired(buf)) {
- cifs_reconnect(server);
+ if (!is_offloaded)
+ cifs_reconnect(server);
return -1;
}
cifs_dbg(FYI, "%s: server returned error %d\n",
__func__, rdata->result);
/* normal error on read response */
- dequeue_mid(mid, false);
+ if (is_offloaded)
+ mid->mid_state = MID_RESPONSE_RECEIVED;
+ else
+ dequeue_mid(mid, false);
return 0;
}
cifs_dbg(FYI, "%s: data offset (%u) beyond end of smallbuf\n",
__func__, data_offset);
rdata->result = -EIO;
- dequeue_mid(mid, rdata->result);
+ if (is_offloaded)
+ mid->mid_state = MID_RESPONSE_MALFORMED;
+ else
+ dequeue_mid(mid, rdata->result);
return 0;
}
cifs_dbg(FYI, "%s: data offset (%u) beyond 1st page of response\n",
__func__, data_offset);
rdata->result = -EIO;
- dequeue_mid(mid, rdata->result);
+ if (is_offloaded)
+ mid->mid_state = MID_RESPONSE_MALFORMED;
+ else
+ dequeue_mid(mid, rdata->result);
return 0;
}
if (data_len > page_data_size - pad_len) {
/* data_len is corrupt -- discard frame */
rdata->result = -EIO;
- dequeue_mid(mid, rdata->result);
+ if (is_offloaded)
+ mid->mid_state = MID_RESPONSE_MALFORMED;
+ else
+ dequeue_mid(mid, rdata->result);
return 0;
}
rdata->result = init_read_bvec(pages, npages, page_data_size,
cur_off, &bvec);
if (rdata->result != 0) {
- dequeue_mid(mid, rdata->result);
+ if (is_offloaded)
+ mid->mid_state = MID_RESPONSE_MALFORMED;
+ else
+ dequeue_mid(mid, rdata->result);
return 0;
}
/* read response payload cannot be in both buf and pages */
WARN_ONCE(1, "buf can not contain only a part of read data");
rdata->result = -EIO;
- dequeue_mid(mid, rdata->result);
+ if (is_offloaded)
+ mid->mid_state = MID_RESPONSE_MALFORMED;
+ else
+ dequeue_mid(mid, rdata->result);
return 0;
}
if (length < 0)
return length;
- dequeue_mid(mid, false);
+ if (is_offloaded)
+ mid->mid_state = MID_RESPONSE_RECEIVED;
+ else
+ dequeue_mid(mid, false);
return length;
}
}
dw->server->lstrp = jiffies;
- mid = smb2_find_mid(dw->server, dw->buf);
+ mid = smb2_find_dequeue_mid(dw->server, dw->buf);
if (mid == NULL)
cifs_dbg(FYI, "mid not found\n");
else {
mid->decrypted = true;
rc = handle_read_data(dw->server, mid, dw->buf,
dw->server->vals->read_rsp_size,
- dw->ppages, dw->npages, dw->len);
- mid->callback(mid);
+ dw->ppages, dw->npages, dw->len,
+ true);
+ if (rc >= 0) {
+#ifdef CONFIG_CIFS_STATS2
+ mid->when_received = jiffies;
+#endif
+ mid->callback(mid);
+ } else {
+ spin_lock(&GlobalMid_Lock);
+ if (dw->server->tcpStatus == CifsNeedReconnect) {
+ mid->mid_state = MID_RETRY_NEEDED;
+ spin_unlock(&GlobalMid_Lock);
+ mid->callback(mid);
+ } else {
+ mid->mid_state = MID_REQUEST_SUBMITTED;
+ mid->mid_flags &= ~(MID_DELETED);
+ list_add_tail(&mid->qhead,
+ &dw->server->pending_mid_q);
+ spin_unlock(&GlobalMid_Lock);
+ }
+ }
cifs_mid_q_entry_release(mid);
}
(*mid)->decrypted = true;
rc = handle_read_data(server, *mid, buf,
server->vals->read_rsp_size,
- pages, npages, len);
+ pages, npages, len, false);
}
free_pages:
char *buf = server->large_buf ? server->bigbuf : server->smallbuf;
return handle_read_data(server, mid, buf, server->pdu_size,
- NULL, 0, 0);
+ NULL, 0, 0, false);
}
static int
return -EAGAIN;
if (signal_pending(current)) {
- cifs_dbg(FYI, "signal is pending before sending any data\n");
- return -EINTR;
+ cifs_dbg(FYI, "signal pending before send request\n");
+ return -ERESTARTSYS;
}
/* cork the socket */
int i;
/* The file must need contents encryption, not filenames encryption */
- if (!fscrypt_needs_contents_encryption(inode))
+ if (!S_ISREG(inode->i_mode))
return 0;
/* The crypto mode must have a blk-crypto counterpart */
#include <linux/efi.h>
#include <linux/fs.h>
#include <linux/ctype.h>
+#include <linux/kmemleak.h>
#include <linux/slab.h>
#include <linux/uuid.h>
var->var.VariableName[i] = '\0';
inode->i_private = var;
+ kmemleak_ignore(var);
err = efivar_entry_add(var, &efivarfs_list);
if (err)
blocks */
#define EXT4_MOUNT2_HURD_COMPAT 0x00000004 /* Support HURD-castrated
file systems */
-#define EXT4_MOUNT2_DAX_NEVER 0x00000008 /* Do not allow Direct Access */
-#define EXT4_MOUNT2_DAX_INODE 0x00000010 /* For printing options only */
-
#define EXT4_MOUNT2_EXPLICIT_JOURNAL_CHECKSUM 0x00000008 /* User explicitly
specified journal checksum */
#define EXT4_MOUNT2_JOURNAL_FAST_COMMIT 0x00000010 /* Journal fast commit */
+#define EXT4_MOUNT2_DAX_NEVER 0x00000020 /* Do not allow Direct Access */
+#define EXT4_MOUNT2_DAX_INODE 0x00000040 /* For printing options only */
+
#define clear_opt(sb, opt) EXT4_SB(sb)->s_mount_opt &= \
~EXT4_MOUNT_##opt
struct ext4_filename *fname);
static inline void ext4_update_dx_flag(struct inode *inode)
{
- if (!ext4_has_feature_dir_index(inode->i_sb)) {
+ if (!ext4_has_feature_dir_index(inode->i_sb) &&
+ ext4_test_inode_flag(inode, EXT4_INODE_INDEX)) {
/* ext4_iget() should have caught this... */
WARN_ON_ONCE(ext4_has_feature_metadata_csum(inode->i_sb));
ext4_clear_inode_flag(inode, EXT4_INODE_INDEX);
if (!ext4_has_metadata_csum(sb))
return;
- /*
- * Locking the superblock prevents the scenario
- * where:
- * 1) a first thread pauses during checksum calculation.
- * 2) a second thread updates the superblock, recalculates
- * the checksum, and updates s_checksum
- * 3) the first thread resumes and finishes its checksum calculation
- * and updates s_checksum with a potentially stale or torn value.
- */
- lock_buffer(EXT4_SB(sb)->s_sbh);
es->s_checksum = ext4_superblock_csum(sb, es);
- unlock_buffer(EXT4_SB(sb)->s_sbh);
}
ext4_fsblk_t ext4_block_bitmap(struct super_block *sb,
} else if (test_opt2(sb, DAX_INODE)) {
SEQ_OPTS_PUTS("dax=inode");
}
-
- if (test_opt2(sb, JOURNAL_FAST_COMMIT))
- SEQ_OPTS_PUTS("fast_commit");
-
ext4_show_quota_options(seq, sb);
return 0;
}
if (error)
return error;
if (!buffer_mapped(bh_result))
- return -EIO;
+ return -ENODATA;
return 0;
}
trace_gfs2_bmap(ip, bh_map, lblock, create, 1);
ret = gfs2_iomap_get(inode, pos, length, flags, &iomap, &mp);
- if (!ret && iomap.type == IOMAP_HOLE) {
- if (create)
- ret = gfs2_iomap_alloc(inode, &iomap, &mp);
- else
- ret = -ENODATA;
- }
+ if (create && !ret && iomap.type == IOMAP_HOLE)
+ ret = gfs2_iomap_alloc(inode, &iomap, &mp);
release_metapath(&mp);
if (ret)
goto out;
gl->gl_node.next = NULL;
gl->gl_flags = 0;
gl->gl_name = name;
+ lockdep_set_subclass(&gl->gl_lockref.lock, glops->go_subclass);
gl->gl_lockref.count = 1;
gl->gl_state = LM_ST_UNLOCKED;
gl->gl_target = LM_ST_UNLOCKED;
static void gfs2_rgrp_go_dump(struct seq_file *seq, struct gfs2_glock *gl,
const char *fs_id_buf)
{
- struct gfs2_rgrpd *rgd = gfs2_glock2rgrp(gl);
+ struct gfs2_rgrpd *rgd = gl->gl_object;
if (rgd)
gfs2_rgrp_dump(seq, rgd, fs_id_buf);
int error = 0;
struct gfs2_sbd *sdp = gl->gl_name.ln_sbd;
- if (gl->gl_req == LM_ST_EXCLUSIVE && !gfs2_withdrawn(sdp)) {
+ /*
+ * We need to check gl_state == LM_ST_SHARED here and not gl_req ==
+ * LM_ST_EXCLUSIVE. That's because when any node does a freeze,
+ * all the nodes should have the freeze glock in SH mode and they all
+ * call do_xmote: One for EX and the others for UN. They ALL must
+ * freeze locally, and they ALL must queue freeze work. The freeze_work
+ * calls freeze_func, which tries to reacquire the freeze glock in SH,
+ * effectively waiting for the thaw on the node who holds it in EX.
+ * Once thawed, the work func acquires the freeze glock in
+ * SH and everybody goes back to thawed.
+ */
+ if (gl->gl_state == LM_ST_SHARED && !gfs2_withdrawn(sdp) &&
+ !test_bit(SDF_NORECOVERY, &sdp->sd_flags)) {
atomic_set(&sdp->sd_freeze_state, SFS_STARTING_FREEZE);
error = freeze_super(sdp->sd_vfs);
if (error) {
.go_callback = iopen_go_callback,
.go_demote_ok = iopen_go_demote_ok,
.go_flags = GLOF_LRU | GLOF_NONDISK,
+ .go_subclass = 1,
};
const struct gfs2_glock_operations gfs2_flock_glops = {
const char *fs_id_buf);
void (*go_callback)(struct gfs2_glock *gl, bool remote);
void (*go_free)(struct gfs2_glock *gl);
+ const int go_subclass;
const int go_type;
const unsigned long go_flags;
#define GLOF_ASPACE 1 /* address space attached */
error = gfs2_glock_get(sdp, no_addr, &gfs2_iopen_glops, CREATE, &io_gl);
if (unlikely(error))
goto fail;
+ if (blktype != GFS2_BLKST_UNLINKED)
+ gfs2_cancel_delete_work(io_gl);
if (type == DT_UNKNOWN || blktype != GFS2_BLKST_FREE) {
/*
error = gfs2_glock_nq_init(io_gl, LM_ST_SHARED, GL_EXACT, &ip->i_iopen_gh);
if (unlikely(error))
goto fail;
- if (blktype != GFS2_BLKST_UNLINKED)
- gfs2_cancel_delete_work(ip->i_iopen_gh.gh_gl);
glock_set_object(ip->i_iopen_gh.gh_gl, ip);
gfs2_glock_put(io_gl);
io_gl = NULL;
flush_delayed_work(&ip->i_gl->gl_work);
glock_set_object(ip->i_gl, ip);
- error = gfs2_glock_nq_init(ip->i_gl, LM_ST_EXCLUSIVE, GL_SKIP, ghs + 1);
+ error = gfs2_glock_get(sdp, ip->i_no_addr, &gfs2_iopen_glops, CREATE, &io_gl);
if (error)
goto fail_free_inode;
+ gfs2_cancel_delete_work(io_gl);
+ glock_set_object(io_gl, ip);
+
+ error = gfs2_glock_nq_init(ip->i_gl, LM_ST_EXCLUSIVE, GL_SKIP, ghs + 1);
+ if (error)
+ goto fail_gunlock2;
error = gfs2_trans_begin(sdp, blocks, 0);
if (error)
- goto fail_free_inode;
+ goto fail_gunlock2;
if (blocks > 1) {
ip->i_eattr = ip->i_no_addr + 1;
init_dinode(dip, ip, symname);
gfs2_trans_end(sdp);
- error = gfs2_glock_get(sdp, ip->i_no_addr, &gfs2_iopen_glops, CREATE, &io_gl);
- if (error)
- goto fail_free_inode;
-
BUG_ON(test_and_set_bit(GLF_INODE_CREATING, &io_gl->gl_flags));
error = gfs2_glock_nq_init(io_gl, LM_ST_SHARED, GL_EXACT, &ip->i_iopen_gh);
if (error)
goto fail_gunlock2;
- gfs2_cancel_delete_work(ip->i_iopen_gh.gh_gl);
- glock_set_object(ip->i_iopen_gh.gh_gl, ip);
gfs2_set_iop(inode);
insert_inode_hash(inode);
gfs2_glock_dq_uninit(&ip->i_iopen_gh);
fail_gunlock2:
clear_bit(GLF_INODE_CREATING, &io_gl->gl_flags);
+ glock_clear_object(io_gl, ip);
gfs2_glock_put(io_gl);
fail_free_inode:
if (ip->i_gl) {
return vfs_setpos(file, ret, inode->i_sb->s_maxbytes);
}
+static int gfs2_update_time(struct inode *inode, struct timespec64 *time,
+ int flags)
+{
+ struct gfs2_inode *ip = GFS2_I(inode);
+ struct gfs2_glock *gl = ip->i_gl;
+ struct gfs2_holder *gh;
+ int error;
+
+ gh = gfs2_glock_is_locked_by_me(gl);
+ if (gh && !gfs2_glock_is_held_excl(gl)) {
+ gfs2_glock_dq(gh);
+ gfs2_holder_reinit(LM_ST_EXCLUSIVE, 0, gh);
+ error = gfs2_glock_nq(gh);
+ if (error)
+ return error;
+ }
+ return generic_update_time(inode, time, flags);
+}
+
const struct inode_operations gfs2_file_iops = {
.permission = gfs2_permission,
.setattr = gfs2_setattr,
.fiemap = gfs2_fiemap,
.get_acl = gfs2_get_acl,
.set_acl = gfs2_set_acl,
+ .update_time = gfs2_update_time,
};
const struct inode_operations gfs2_dir_iops = {
.fiemap = gfs2_fiemap,
.get_acl = gfs2_get_acl,
.set_acl = gfs2_set_acl,
+ .update_time = gfs2_update_time,
.atomic_open = gfs2_atomic_open,
};
spin_unlock(&sdp->sd_ail_lock);
ret = generic_writepages(mapping, wbc);
spin_lock(&sdp->sd_ail_lock);
+ if (ret == -ENODATA) /* if a jdata write into a new hole */
+ ret = 0; /* ignore it */
if (ret || wbc->nr_to_write <= 0)
break;
return -EBUSY;
if (error < 0)
return error;
+ if (RB_EMPTY_ROOT(&sdp->sd_rindex_tree)) {
+ fs_err(sdp, "no resource groups found in the file system.\n");
+ return -ENOENT;
+ }
set_rgrp_preferences(sdp);
sdp->sd_rindex_uptodate = 1;
rbm.rgd = rgd;
error = gfs2_rbm_from_block(&rbm, no_addr);
- if (WARN_ON_ONCE(error))
- goto fail;
-
- if (gfs2_testbit(&rbm, false) != type)
- error = -ESTALE;
+ if (!WARN_ON_ONCE(error)) {
+ if (gfs2_testbit(&rbm, false) != type)
+ error = -ESTALE;
+ }
gfs2_glock_dq_uninit(&rgd_gh);
+
fail:
return error;
}
struct list_head file_list;
struct fixed_file_data *file_data;
struct llist_node llist;
+ bool done;
};
struct fixed_file_data {
struct io_open {
struct file *file;
int dfd;
+ bool ignore_nonblock;
struct filename *filename;
struct open_how how;
unsigned long nofile;
return false;
req->work.flags |= IO_WQ_WORK_FSIZE;
}
-
- if (!(req->work.flags & IO_WQ_WORK_FILES) &&
- (def->work_flags & IO_WQ_WORK_FILES) &&
- !(req->flags & REQ_F_NO_FILE_TABLE)) {
- if (id->files != current->files ||
- id->nsproxy != current->nsproxy)
- return false;
- atomic_inc(&id->files->count);
- get_nsproxy(id->nsproxy);
- req->flags |= REQ_F_INFLIGHT;
-
- spin_lock_irq(&ctx->inflight_lock);
- list_add(&req->inflight_entry, &ctx->inflight_list);
- spin_unlock_irq(&ctx->inflight_lock);
- req->work.flags |= IO_WQ_WORK_FILES;
- }
#ifdef CONFIG_BLK_CGROUP
if (!(req->work.flags & IO_WQ_WORK_BLKCG) &&
(def->work_flags & IO_WQ_WORK_BLKCG)) {
}
spin_unlock(¤t->fs->lock);
}
+ if (!(req->work.flags & IO_WQ_WORK_FILES) &&
+ (def->work_flags & IO_WQ_WORK_FILES) &&
+ !(req->flags & REQ_F_NO_FILE_TABLE)) {
+ if (id->files != current->files ||
+ id->nsproxy != current->nsproxy)
+ return false;
+ atomic_inc(&id->files->count);
+ get_nsproxy(id->nsproxy);
+ req->flags |= REQ_F_INFLIGHT;
+
+ spin_lock_irq(&ctx->inflight_lock);
+ list_add(&req->inflight_entry, &ctx->inflight_list);
+ spin_unlock_irq(&ctx->inflight_lock);
+ req->work.flags |= IO_WQ_WORK_FILES;
+ }
return true;
}
}
end_req:
req_set_fail_links(req);
- io_req_complete(req, ret);
return false;
}
#endif
rw->free_iovec = iovec;
rw->bytes_done = 0;
/* can only be fixed buffers, no need to do anything */
- if (iter->type == ITER_BVEC)
+ if (iov_iter_is_bvec(iter))
return;
if (!iovec) {
unsigned iov_off = 0;
* we return to userspace.
*/
if (req->flags & REQ_F_ISREG) {
- __sb_start_write(file_inode(req->file)->i_sb,
- SB_FREEZE_WRITE, true);
+ sb_start_write(file_inode(req->file)->i_sb);
__sb_writers_release(file_inode(req->file)->i_sb,
SB_FREEZE_WRITE);
}
return ret;
}
req->open.nofile = rlimit(RLIMIT_NOFILE);
+ req->open.ignore_nonblock = false;
req->flags |= REQ_F_NEED_CLEANUP;
return 0;
}
struct file *file;
int ret;
- if (force_nonblock)
+ if (force_nonblock && !req->open.ignore_nonblock)
return -EAGAIN;
ret = build_open_flags(&req->open.how, &op);
if (IS_ERR(file)) {
put_unused_fd(ret);
ret = PTR_ERR(file);
+ /*
+ * A work-around to ensure that /proc/self works that way
+ * that it should - if we get -EOPNOTSUPP back, then assume
+ * that proc_self_get_link() failed us because we're in async
+ * context. We should be safe to retry this from the task
+ * itself with force_nonblock == false set, as it should not
+ * block on lookup. Would be nice to know this upfront and
+ * avoid the async dance, but doesn't seem feasible.
+ */
+ if (ret == -EOPNOTSUPP && io_wq_current_is_worker()) {
+ req->open.ignore_nonblock = true;
+ refcount_inc(&req->refs);
+ io_req_task_queue(req);
+ return 0;
+ }
} else {
fsnotify_open(file);
fd_install(ret, file);
return -ENXIO;
spin_lock(&data->lock);
- if (!list_empty(&data->ref_list))
- ref_node = list_first_entry(&data->ref_list,
- struct fixed_file_ref_node, node);
+ ref_node = data->node;
spin_unlock(&data->lock);
if (ref_node)
percpu_ref_kill(&ref_node->refs);
kfree(pfile);
}
- spin_lock(&file_data->lock);
- list_del(&ref_node->node);
- spin_unlock(&file_data->lock);
-
percpu_ref_exit(&ref_node->refs);
kfree(ref_node);
percpu_ref_put(&file_data->refs);
static void io_file_data_ref_zero(struct percpu_ref *ref)
{
struct fixed_file_ref_node *ref_node;
+ struct fixed_file_data *data;
struct io_ring_ctx *ctx;
- bool first_add;
+ bool first_add = false;
int delay = HZ;
ref_node = container_of(ref, struct fixed_file_ref_node, refs);
- ctx = ref_node->file_data->ctx;
+ data = ref_node->file_data;
+ ctx = data->ctx;
- if (percpu_ref_is_dying(&ctx->file_data->refs))
+ spin_lock(&data->lock);
+ ref_node->done = true;
+
+ while (!list_empty(&data->ref_list)) {
+ ref_node = list_first_entry(&data->ref_list,
+ struct fixed_file_ref_node, node);
+ /* recycle ref nodes in order */
+ if (!ref_node->done)
+ break;
+ list_del(&ref_node->node);
+ first_add |= llist_add(&ref_node->llist, &ctx->file_put_llist);
+ }
+ spin_unlock(&data->lock);
+
+ if (percpu_ref_is_dying(&data->refs))
delay = 0;
- first_add = llist_add(&ref_node->llist, &ctx->file_put_llist);
if (!delay)
mod_delayed_work(system_wq, &ctx->file_put_work, 0);
else if (first_add)
INIT_LIST_HEAD(&ref_node->node);
INIT_LIST_HEAD(&ref_node->file_list);
ref_node->file_data = ctx->file_data;
+ ref_node->done = false;
return ref_node;
}
file_data->node = ref_node;
spin_lock(&file_data->lock);
- list_add(&ref_node->node, &file_data->ref_list);
+ list_add_tail(&ref_node->node, &file_data->ref_list);
spin_unlock(&file_data->lock);
percpu_ref_get(&file_data->refs);
return ret;
if (needs_switch) {
percpu_ref_kill(&data->node->refs);
spin_lock(&data->lock);
- list_add(&ref_node->node, &data->ref_list);
+ list_add_tail(&ref_node->node, &data->ref_list);
data->node = ref_node;
spin_unlock(&data->lock);
percpu_ref_get(&ctx->file_data->refs);
* to a power-of-two, if it isn't already. We do NOT impose
* any cq vs sq ring sizing.
*/
- if (p->cq_entries < p->sq_entries)
+ if (!p->cq_entries)
return -EINVAL;
if (p->cq_entries > IORING_MAX_CQ_ENTRIES) {
if (!(p->flags & IORING_SETUP_CLAMP))
p->cq_entries = IORING_MAX_CQ_ENTRIES;
}
p->cq_entries = roundup_pow_of_two(p->cq_entries);
+ if (p->cq_entries < p->sq_entries)
+ return -EINVAL;
} else {
p->cq_entries = 2 * p->sq_entries;
}
}
/**
- * Force and wait upon a commit if the calling process is not within
- * transaction. This is used for forcing out undo-protected data which contains
- * bitmaps, when the fs is running out of space.
+ * jbd2_journal_force_commit_nested - Force and wait upon a commit if the
+ * calling process is not within transaction.
*
* @journal: journal to force
* Returns true if progress was made.
+ *
+ * This is used for forcing out undo-protected data which contains
+ * bitmaps, when the fs is running out of space.
*/
int jbd2_journal_force_commit_nested(journal_t *journal)
{
}
/**
- * int journal_force_commit() - force any uncommitted transactions
+ * jbd2_journal_force_commit() - force any uncommitted transactions
* @journal: journal to force
*
* Caller want unconditional commit. We can only force the running transaction
/**
- * int jbd2_journal_load() - Read journal from disk.
+ * jbd2_journal_load() - Read journal from disk.
* @journal: Journal to act on.
*
* Given a journal_t structure which tells us which disk blocks contain
}
/**
- * void jbd2_journal_destroy() - Release a journal_t structure.
+ * jbd2_journal_destroy() - Release a journal_t structure.
* @journal: Journal to act on.
*
* Release a journal_t structure once it is no longer in use by the
/**
- *int jbd2_journal_check_used_features() - Check if features specified are used.
+ * jbd2_journal_check_used_features() - Check if features specified are used.
* @journal: Journal to check.
* @compat: bitmask of compatible features
* @ro: bitmask of features that force read-only mount
}
/**
- * int jbd2_journal_check_available_features() - Check feature set in journalling layer
+ * jbd2_journal_check_available_features() - Check feature set in journalling layer
* @journal: Journal to check.
* @compat: bitmask of compatible features
* @ro: bitmask of features that force read-only mount
}
/**
- * int jbd2_journal_set_features() - Mark a given journal feature in the superblock
+ * jbd2_journal_set_features() - Mark a given journal feature in the superblock
* @journal: Journal to act on.
* @compat: bitmask of compatible features
* @ro: bitmask of features that force read-only mount
}
/*
- * jbd2_journal_clear_features () - Clear a given journal feature in the
+ * jbd2_journal_clear_features() - Clear a given journal feature in the
* superblock
* @journal: Journal to act on.
* @compat: bitmask of compatible features
EXPORT_SYMBOL(jbd2_journal_clear_features);
/**
- * int jbd2_journal_flush () - Flush journal
+ * jbd2_journal_flush() - Flush journal
* @journal: Journal to act on.
*
* Flush all data for a given journal to disk and empty the journal.
}
/**
- * int jbd2_journal_wipe() - Wipe journal contents
+ * jbd2_journal_wipe() - Wipe journal contents
* @journal: Journal to act on.
* @write: flag (see below)
*
}
/**
- * void jbd2_journal_abort () - Shutdown the journal immediately.
+ * jbd2_journal_abort () - Shutdown the journal immediately.
* @journal: the journal to shutdown.
* @errno: an error number to record in the journal indicating
* the reason for the shutdown.
}
/**
- * int jbd2_journal_errno () - returns the journal's error state.
+ * jbd2_journal_errno() - returns the journal's error state.
* @journal: journal to examine.
*
* This is the errno number set with jbd2_journal_abort(), the last
}
/**
- * int jbd2_journal_clear_err () - clears the journal's error state
+ * jbd2_journal_clear_err() - clears the journal's error state
* @journal: journal to act on.
*
* An error must be cleared or acked to take a FS out of readonly
}
/**
- * void jbd2_journal_ack_err() - Ack journal err.
+ * jbd2_journal_ack_err() - Ack journal err.
* @journal: journal to act on.
*
* An error must be cleared or acked to take a FS out of readonly
/**
- * handle_t *jbd2_journal_start() - Obtain a new handle.
+ * jbd2_journal_start() - Obtain a new handle.
* @journal: Journal to start transaction on.
* @nblocks: number of block buffer we might modify
*
EXPORT_SYMBOL(jbd2_journal_free_reserved);
/**
- * int jbd2_journal_start_reserved() - start reserved handle
+ * jbd2_journal_start_reserved() - start reserved handle
* @handle: handle to start
* @type: for handle statistics
* @line_no: for handle statistics
EXPORT_SYMBOL(jbd2_journal_start_reserved);
/**
- * int jbd2_journal_extend() - extend buffer credits.
+ * jbd2_journal_extend() - extend buffer credits.
* @handle: handle to 'extend'
* @nblocks: nr blocks to try to extend by.
* @revoke_records: number of revoke records to try to extend by.
}
/**
- * int jbd2_journal_restart() - restart a handle .
+ * jbd2__journal_restart() - restart a handle .
* @handle: handle to restart
* @nblocks: nr credits requested
* @revoke_records: number of revoke record credits requested
EXPORT_SYMBOL(jbd2_journal_restart);
/**
- * void jbd2_journal_lock_updates () - establish a transaction barrier.
+ * jbd2_journal_lock_updates () - establish a transaction barrier.
* @journal: Journal to establish a barrier on.
*
* This locks out any further updates from being started, and blocks
}
/**
- * void jbd2_journal_unlock_updates (journal_t* journal) - release barrier
+ * jbd2_journal_unlock_updates () - release barrier
* @journal: Journal to release the barrier on.
*
* Release a transaction barrier obtained with jbd2_journal_lock_updates().
}
/**
- * int jbd2_journal_get_write_access() - notify intent to modify a buffer for metadata (not data) update.
+ * jbd2_journal_get_write_access() - notify intent to modify a buffer
+ * for metadata (not data) update.
* @handle: transaction to add buffer modifications to
* @bh: bh to be used for metadata writes
*
* unlocked buffer beforehand. */
/**
- * int jbd2_journal_get_create_access () - notify intent to use newly created bh
+ * jbd2_journal_get_create_access () - notify intent to use newly created bh
* @handle: transaction to new buffer to
* @bh: new buffer.
*
}
/**
- * int jbd2_journal_get_undo_access() - Notify intent to modify metadata with
+ * jbd2_journal_get_undo_access() - Notify intent to modify metadata with
* non-rewindable consequences
* @handle: transaction
* @bh: buffer to undo
}
/**
- * void jbd2_journal_set_triggers() - Add triggers for commit writeout
+ * jbd2_journal_set_triggers() - Add triggers for commit writeout
* @bh: buffer to trigger on
* @type: struct jbd2_buffer_trigger_type containing the trigger(s).
*
}
/**
- * int jbd2_journal_dirty_metadata() - mark a buffer as containing dirty metadata
+ * jbd2_journal_dirty_metadata() - mark a buffer as containing dirty metadata
* @handle: transaction to add buffer to.
* @bh: buffer to mark
*
}
/**
- * void jbd2_journal_forget() - bforget() for potentially-journaled buffers.
+ * jbd2_journal_forget() - bforget() for potentially-journaled buffers.
* @handle: transaction handle
* @bh: bh to 'forget'
*
}
/**
- * int jbd2_journal_stop() - complete a transaction
+ * jbd2_journal_stop() - complete a transaction
* @handle: transaction to complete.
*
* All done for a particular handle.
}
/**
- * int jbd2_journal_try_to_free_buffers() - try to free page buffers.
+ * jbd2_journal_try_to_free_buffers() - try to free page buffers.
* @journal: journal for operation
* @page: to try and free
*
}
/**
- * void jbd2_journal_invalidatepage()
+ * jbd2_journal_invalidatepage()
* @journal: journal to use for flush...
* @page: page to flush
* @offset: start of the range to invalidate
size_t len, loff_t *ppos)
{
struct simple_attr *attr;
- u64 val;
+ unsigned long long val;
size_t size;
ssize_t ret;
goto out;
attr->set_buf[size] = '\0';
- val = simple_strtoll(attr->set_buf, NULL, 0);
+ ret = kstrtoull(attr->set_buf, 0, &val);
+ if (ret)
+ goto out;
ret = attr->set(attr->data, val);
if (ret == 0)
ret = len; /* on success, claim we got the whole input */
struct inode *inode = d_inode(dentry);
struct dentry *parent;
bool parent_watched = dentry->d_flags & DCACHE_FSNOTIFY_PARENT_WATCHED;
+ bool parent_needed, parent_interested;
__u32 p_mask;
struct inode *p_inode = NULL;
struct name_snapshot name;
return 0;
parent = NULL;
- if (!parent_watched && !fsnotify_event_needs_parent(inode, mnt, mask))
+ parent_needed = fsnotify_event_needs_parent(inode, mnt, mask);
+ if (!parent_watched && !parent_needed)
goto notify;
/* Does parent inode care about events on children? */
/*
* Include parent/name in notification either if some notification
- * groups require parent info (!parent_watched case) or the parent is
- * interested in this event.
+ * groups require parent info or the parent is interested in this event.
*/
- if (!parent_watched || (mask & p_mask & ALL_FSNOTIFY_EVENTS)) {
+ parent_interested = mask & p_mask & ALL_FSNOTIFY_EVENTS;
+ if (parent_needed || parent_interested) {
/* When notifying parent, child should be passed as data */
WARN_ON_ONCE(inode != fsnotify_data_inode(data, data_type));
/* Notify both parent and child with child name info */
take_dentry_name_snapshot(&name, dentry);
file_name = &name.name;
- if (parent_watched)
+ if (parent_interested)
mask |= FS_EVENT_ON_CHILD;
}
oi->ip_blkno = 0ULL;
oi->ip_clusters = 0;
+ oi->ip_next_orphan = NULL;
ocfs2_resv_init_once(&oi->ip_la_data_resv);
pid_t tgid = task_tgid_nr_ns(current, ns);
char *name;
+ /*
+ * Not currently supported. Once we can inherit all of struct pid,
+ * we can allow this.
+ */
+ if (current->flags & PF_KTHREAD)
+ return ERR_PTR(-EOPNOTSUPP);
+
if (!tgid)
return ERR_PTR(-ENOENT);
/* max length of unsigned int in decimal + NULL term */
}
EXPORT_SYMBOL(super_setup_bdi);
-/*
- * This is an internal function, please use sb_end_{write,pagefault,intwrite}
- * instead.
- */
-void __sb_end_write(struct super_block *sb, int level)
-{
- percpu_up_read(sb->s_writers.rw_sem + level-1);
-}
-EXPORT_SYMBOL(__sb_end_write);
-
-/*
- * This is an internal function, please use sb_start_{write,pagefault,intwrite}
- * instead.
- */
-int __sb_start_write(struct super_block *sb, int level, bool wait)
-{
- bool force_trylock = false;
- int ret = 1;
-
-#ifdef CONFIG_LOCKDEP
- /*
- * We want lockdep to tell us about possible deadlocks with freezing
- * but it's it bit tricky to properly instrument it. Getting a freeze
- * protection works as getting a read lock but there are subtle
- * problems. XFS for example gets freeze protection on internal level
- * twice in some cases, which is OK only because we already hold a
- * freeze protection also on higher level. Due to these cases we have
- * to use wait == F (trylock mode) which must not fail.
- */
- if (wait) {
- int i;
-
- for (i = 0; i < level - 1; i++)
- if (percpu_rwsem_is_held(sb->s_writers.rw_sem + i)) {
- force_trylock = true;
- break;
- }
- }
-#endif
- if (wait && !force_trylock)
- percpu_down_read(sb->s_writers.rw_sem + level-1);
- else
- ret = percpu_down_read_trylock(sb->s_writers.rw_sem + level-1);
-
- WARN_ON(force_trylock && !ret);
- return ret;
-}
-EXPORT_SYMBOL(__sb_start_write);
-
/**
* sb_wait_write - wait until all writers to given file system finish
* @sb: the super for which we wait
*========================================================================*/
/*
- * Query whether the requested number of additional bytes of extended
+ * Query whether the total requested number of attr fork bytes of extended
* attribute space will be able to fit inline.
*
* Returns zero if not, else the di_forkoff fork offset to be used in the
int maxforkoff;
int offset;
+ /*
+ * Check if the new size could fit at all first:
+ */
+ if (bytes > XFS_LITINO(mp))
+ return 0;
+
/* rounded down */
offset = (XFS_LITINO(mp) - bytes) >> 3;
* record for our insertion point. This will also give us the record for
* start block contiguity tests.
*/
- error = xfs_rmap_lookup_le_range(cur, bno, owner, offset, flags,
+ error = xfs_rmap_lookup_le_range(cur, bno, owner, offset, oldext,
&PREV, &i);
if (error)
goto done;
if (info->whichfork == XFS_ATTR_FORK)
rflags |= XFS_RMAP_ATTR_FORK;
+ if (irec->br_state == XFS_EXT_UNWRITTEN)
+ rflags |= XFS_RMAP_UNWRITTEN;
/*
* CoW staging extents are owned (on disk) by the refcountbt, so
* which doesn't track unwritten state.
*/
if (owner != XFS_RMAP_OWN_COW &&
- irec->br_state == XFS_EXT_UNWRITTEN &&
- !(rmap.rm_flags & XFS_RMAP_UNWRITTEN))
+ !!(irec->br_state == XFS_EXT_UNWRITTEN) !=
+ !!(rmap.rm_flags & XFS_RMAP_UNWRITTEN))
xchk_fblock_xref_set_corrupt(info->sc, info->whichfork,
irec->br_startoff);
- if (info->whichfork == XFS_ATTR_FORK &&
- !(rmap.rm_flags & XFS_RMAP_ATTR_FORK))
+ if (!!(info->whichfork == XFS_ATTR_FORK) !=
+ !!(rmap.rm_flags & XFS_RMAP_ATTR_FORK))
xchk_fblock_xref_set_corrupt(info->sc, info->whichfork,
irec->br_startoff);
if (rmap.rm_flags & XFS_RMAP_BMBT_BLOCK)
int level,
struct xfs_btree_block *block)
{
- unsigned int numrecs;
- int ok_level;
-
- numrecs = be16_to_cpu(block->bb_numrecs);
+ struct xfs_btree_cur *cur = bs->cur;
+ unsigned int root_level = cur->bc_nlevels - 1;
+ unsigned int numrecs = be16_to_cpu(block->bb_numrecs);
/* More records than minrecs means the block is ok. */
- if (numrecs >= bs->cur->bc_ops->get_minrecs(bs->cur, level))
+ if (numrecs >= cur->bc_ops->get_minrecs(cur, level))
return;
/*
- * Certain btree blocks /can/ have fewer than minrecs records. Any
- * level greater than or equal to the level of the highest dedicated
- * btree block are allowed to violate this constraint.
- *
- * For a btree rooted in a block, the btree root can have fewer than
- * minrecs records. If the btree is rooted in an inode and does not
- * store records in the root, the direct children of the root and the
- * root itself can have fewer than minrecs records.
+ * For btrees rooted in the inode, it's possible that the root block
+ * contents spilled into a regular ondisk block because there wasn't
+ * enough space in the inode root. The number of records in that
+ * child block might be less than the standard minrecs, but that's ok
+ * provided that there's only one direct child of the root.
*/
- ok_level = bs->cur->bc_nlevels - 1;
- if (bs->cur->bc_flags & XFS_BTREE_ROOT_IN_INODE)
- ok_level--;
- if (level >= ok_level)
+ if ((cur->bc_flags & XFS_BTREE_ROOT_IN_INODE) &&
+ level == cur->bc_nlevels - 2) {
+ struct xfs_btree_block *root_block;
+ struct xfs_buf *root_bp;
+ int root_maxrecs;
+
+ root_block = xfs_btree_get_block(cur, root_level, &root_bp);
+ root_maxrecs = cur->bc_ops->get_dmaxrecs(cur, root_level);
+ if (be16_to_cpu(root_block->bb_numrecs) != 1 ||
+ numrecs <= root_maxrecs)
+ xchk_btree_set_corrupt(bs->sc, cur, level);
return;
+ }
- xchk_btree_set_corrupt(bs->sc, bs->cur, level);
+ /*
+ * Otherwise, only the root level is allowed to have fewer than minrecs
+ * records or keyptrs.
+ */
+ if (level < root_level)
+ xchk_btree_set_corrupt(bs->sc, cur, level);
}
/*
/* Check all the bestfree entries. */
for (i = 0; i < bestcount; i++, bestp++) {
best = be16_to_cpu(*bestp);
- if (best == NULLDATAOFF)
- continue;
error = xfs_dir3_data_read(sc->tp, sc->ip,
- i * args->geo->fsbcount, 0, &dbp);
+ xfs_dir2_db_to_da(args->geo, i),
+ XFS_DABUF_MAP_HOLE_OK,
+ &dbp);
if (!xchk_fblock_process_error(sc, XFS_DATA_FORK, lblk,
&error))
break;
- xchk_directory_check_freesp(sc, lblk, dbp, best);
+
+ if (!dbp) {
+ if (best != NULLDATAOFF) {
+ xchk_fblock_set_corrupt(sc, XFS_DATA_FORK,
+ lblk);
+ break;
+ }
+ continue;
+ }
+
+ if (best == NULLDATAOFF)
+ xchk_fblock_set_corrupt(sc, XFS_DATA_FORK, lblk);
+ else
+ xchk_directory_check_freesp(sc, lblk, dbp, best);
xfs_trans_brelse(sc->tp, dbp);
if (sc->sm->sm_flags & XFS_SCRUB_OFLAG_CORRUPT)
break;
*/
INIT_LIST_HEAD(&worklist);
rbno = NULLAGBLOCK;
- nr = 1;
/* Make sure the fragments actually /are/ in agbno order. */
bno = 0;
* Find all the rmaps that start at or before the refc extent,
* and put them on the worklist.
*/
+ nr = 0;
list_for_each_entry_safe(frag, n, &refchk->fragments, list) {
- if (frag->rm.rm_startblock > refchk->bno)
- goto done;
+ if (frag->rm.rm_startblock > refchk->bno || nr > target_nr)
+ break;
bno = frag->rm.rm_startblock + frag->rm.rm_blockcount;
if (bno < rbno)
rbno = bno;
list_move_tail(&frag->list, &worklist);
- if (nr == target_nr)
- break;
nr++;
}
return 0;
}
+/*
+ * Check that the imap we are going to return to the caller spans the entire
+ * range that the caller requested for the IO.
+ */
+static bool
+imap_spans_range(
+ struct xfs_bmbt_irec *imap,
+ xfs_fileoff_t offset_fsb,
+ xfs_fileoff_t end_fsb)
+{
+ if (imap->br_startoff > offset_fsb)
+ return false;
+ if (imap->br_startoff + imap->br_blockcount < end_fsb)
+ return false;
+ return true;
+}
+
static int
xfs_direct_write_iomap_begin(
struct inode *inode,
if (imap_needs_alloc(inode, flags, &imap, nimaps))
goto allocate_blocks;
+ /*
+ * NOWAIT IO needs to span the entire requested IO with a single map so
+ * that we avoid partial IO failures due to the rest of the IO range not
+ * covered by this map triggering an EAGAIN condition when it is
+ * subsequently mapped and aborting the IO.
+ */
+ if ((flags & IOMAP_NOWAIT) &&
+ !imap_spans_range(&imap, offset_fsb, end_fsb)) {
+ error = -EAGAIN;
+ goto out_unlock;
+ }
+
xfs_iunlock(ip, lockmode);
trace_xfs_iomap_found(ip, offset, length, XFS_DATA_FORK, &imap);
return xfs_bmbt_to_iomap(ip, iomap, &imap, iomap_flags);
/* Where do we start the traversal? */
xfs_ino_t startino;
+ /* What was the last inode number we saw when iterating the inobt? */
+ xfs_ino_t lastino;
+
/* Array of inobt records we cache. */
struct xfs_inobt_rec_incore *recs;
if (XFS_IS_CORRUPT(mp, *has_more != 1))
return -EFSCORRUPTED;
+ iwag->lastino = XFS_AGINO_TO_INO(mp, agno,
+ irec->ir_startino + XFS_INODES_PER_CHUNK - 1);
+
/*
* If the LE lookup yielded an inobt record before the cursor position,
* skip it and see if there's another one after it.
struct xfs_mount *mp = iwag->mp;
struct xfs_trans *tp = iwag->tp;
struct xfs_inobt_rec_incore *irec;
- xfs_agino_t restart;
+ xfs_agino_t next_agino;
int error;
+ next_agino = XFS_INO_TO_AGINO(mp, iwag->lastino) + 1;
+
ASSERT(iwag->nr_recs > 0);
/* Delete cursor but remember the last record we cached... */
xfs_iwalk_del_inobt(tp, curpp, agi_bpp, 0);
irec = &iwag->recs[iwag->nr_recs - 1];
- restart = irec->ir_startino + XFS_INODES_PER_CHUNK - 1;
+ ASSERT(next_agino == irec->ir_startino + XFS_INODES_PER_CHUNK);
error = xfs_iwalk_ag_recs(iwag);
if (error)
if (error)
return error;
- return xfs_inobt_lookup(*curpp, restart, XFS_LOOKUP_GE, has_more);
+ return xfs_inobt_lookup(*curpp, next_agino, XFS_LOOKUP_GE, has_more);
}
/* Walk all inodes in a single AG, from @iwag->startino to the end of the AG. */
while (!error && has_more) {
struct xfs_inobt_rec_incore *irec;
+ xfs_ino_t rec_fsino;
cond_resched();
if (xfs_pwork_want_abort(&iwag->pwork))
if (error || !has_more)
break;
+ /* Make sure that we always move forward. */
+ rec_fsino = XFS_AGINO_TO_INO(mp, agno, irec->ir_startino);
+ if (iwag->lastino != NULLFSINO &&
+ XFS_IS_CORRUPT(mp, iwag->lastino >= rec_fsino)) {
+ error = -EFSCORRUPTED;
+ goto out;
+ }
+ iwag->lastino = rec_fsino + XFS_INODES_PER_CHUNK - 1;
+
/* No allocated inodes in this chunk; skip it. */
if (iwag->skip_empty && irec->ir_freecount == irec->ir_count) {
error = xfs_btree_increment(cur, 0, &has_more);
.trim_start = 1,
.skip_empty = 1,
.pwork = XFS_PWORK_SINGLE_THREADED,
+ .lastino = NULLFSINO,
};
xfs_agnumber_t agno = XFS_INO_TO_AGNO(mp, startino);
int error;
iwag->data = data;
iwag->startino = startino;
iwag->sz_recs = xfs_iwalk_prefetch(inode_records);
+ iwag->lastino = NULLFSINO;
xfs_pwork_queue(&pctl, &iwag->pwork);
startino = XFS_AGINO_TO_INO(mp, agno + 1, 0);
if (flags & XFS_INOBT_WALK_SAME_AG)
.startino = startino,
.sz_recs = xfs_inobt_walk_prefetch(inobt_records),
.pwork = XFS_PWORK_SINGLE_THREADED,
+ .lastino = NULLFSINO,
};
xfs_agnumber_t agno = XFS_INO_TO_AGNO(mp, startino);
int error;
}
pag = kmem_zalloc(sizeof(*pag), KM_MAYFAIL);
- if (!pag)
+ if (!pag) {
+ error = -ENOMEM;
goto out_unwind_new_pags;
+ }
pag->pag_agno = index;
pag->pag_mount = mp;
spin_lock_init(&pag->pag_ici_lock);
INIT_RADIX_TREE(&pag->pag_ici_root, GFP_ATOMIC);
- if (xfs_buf_hash_init(pag))
+
+ error = xfs_buf_hash_init(pag);
+ if (error)
goto out_free_pag;
init_waitqueue_head(&pag->pagb_wait);
spin_lock_init(&pag->pagb_lock);
pag->pagb_count = 0;
pag->pagb_tree = RB_ROOT;
- if (radix_tree_preload(GFP_NOFS))
+ error = radix_tree_preload(GFP_NOFS);
+ if (error)
goto out_hash_destroy;
spin_lock(&mp->m_perag_lock);
goto out_unlock;
error = invalidate_inode_pages2(inode->i_mapping);
if (WARN_ON_ONCE(error))
- return error;
+ goto out_unlock;
end_fsb = XFS_B_TO_FSB(mp, (xfs_ufsize_t)offset + length);
offset_fsb = XFS_B_TO_FSBT(mp, offset);
#ifndef __ASSEMBLY__
+#include <linux/compiler.h>
#include <asm/rwonce.h>
#ifndef nop
#define __this_cpu_generic_read_nopreempt(pcp) \
({ \
- typeof(pcp) __ret; \
+ typeof(pcp) ___ret; \
preempt_disable_notrace(); \
- __ret = READ_ONCE(*raw_cpu_ptr(&(pcp))); \
+ ___ret = READ_ONCE(*raw_cpu_ptr(&(pcp))); \
preempt_enable_notrace(); \
- __ret; \
+ ___ret; \
})
#define __this_cpu_generic_read_noirq(pcp) \
({ \
- typeof(pcp) __ret; \
- unsigned long __flags; \
- raw_local_irq_save(__flags); \
- __ret = raw_cpu_generic_read(pcp); \
- raw_local_irq_restore(__flags); \
- __ret; \
+ typeof(pcp) ___ret; \
+ unsigned long ___flags; \
+ raw_local_irq_save(___flags); \
+ ___ret = raw_cpu_generic_read(pcp); \
+ raw_local_irq_restore(___flags); \
+ ___ret; \
})
#define this_cpu_generic_read(pcp) \
#define IMX_SC_R_CAN_0 105
#define IMX_SC_R_CAN_1 106
#define IMX_SC_R_CAN_2 107
+#define IMX_SC_R_CAN(x) (IMX_SC_R_CAN_0 + (x))
#define IMX_SC_R_DMA_1_CH0 108
#define IMX_SC_R_DMA_1_CH1 109
#define IMX_SC_R_DMA_1_CH2 110
u8 ticket[]; /* the encrypted ticket */
};
-/*
- * Kerberos 5 principal
- * name/name/name@realm
- */
-struct krb5_principal {
- u8 n_name_parts; /* N of parts of the name part of the principal */
- char **name_parts; /* parts of the name part of the principal */
- char *realm; /* parts of the realm part of the principal */
-};
-
-/*
- * Kerberos 5 tagged data
- */
-struct krb5_tagged_data {
- /* for tag value, see /usr/include/krb5/krb5.h
- * - KRB5_AUTHDATA_* for auth data
- * -
- */
- s32 tag;
- u32 data_len;
- u8 *data;
-};
-
-/*
- * RxRPC key for Kerberos V (type-5 security)
- */
-struct rxk5_key {
- u64 authtime; /* time at which auth token generated */
- u64 starttime; /* time at which auth token starts */
- u64 endtime; /* time at which auth token expired */
- u64 renew_till; /* time to which auth token can be renewed */
- s32 is_skey; /* T if ticket is encrypted in another ticket's
- * skey */
- s32 flags; /* mask of TKT_FLG_* bits (krb5/krb5.h) */
- struct krb5_principal client; /* client principal name */
- struct krb5_principal server; /* server principal name */
- u16 ticket_len; /* length of ticket */
- u16 ticket2_len; /* length of second ticket */
- u8 n_authdata; /* number of authorisation data elements */
- u8 n_addresses; /* number of addresses */
- struct krb5_tagged_data session; /* session data; tag is enctype */
- struct krb5_tagged_data *addresses; /* addresses */
- u8 *ticket; /* krb5 ticket */
- u8 *ticket2; /* second krb5 ticket, if related to ticket (via
- * DUPLICATE-SKEY or ENC-TKT-IN-SKEY) */
- struct krb5_tagged_data *authdata; /* authorisation data */
-};
-
/*
* list of tokens attached to an rxrpc key
*/
struct rxrpc_key_token {
u16 security_index; /* RxRPC header security index */
+ bool no_leak_key; /* Don't copy the key to userspace */
struct rxrpc_key_token *next; /* the next token in the list */
union {
struct rxkad_key *kad;
- struct rxk5_key *k5;
};
};
#define AFSTOKEN_RK_TIX_MAX 12000 /* max RxKAD ticket size */
#define AFSTOKEN_GK_KEY_MAX 64 /* max GSSAPI key size */
#define AFSTOKEN_GK_TOKEN_MAX 16384 /* max GSSAPI token size */
-#define AFSTOKEN_K5_COMPONENTS_MAX 16 /* max K5 components */
-#define AFSTOKEN_K5_NAME_MAX 128 /* max K5 name length */
-#define AFSTOKEN_K5_REALM_MAX 64 /* max K5 realm name length */
-#define AFSTOKEN_K5_TIX_MAX 16384 /* max K5 ticket size */
-#define AFSTOKEN_K5_ADDRESSES_MAX 16 /* max K5 addresses */
-#define AFSTOKEN_K5_AUTHDATA_MAX 16 /* max K5 pieces of auth data */
/*
* Truncate a time64_t to the range from 1970 to 2106 as in the network
KUNIT_ASSERTION(test, \
strcmp(__left, __right) op 0, \
kunit_binary_str_assert, \
- KUNIT_INIT_BINARY_ASSERT_STRUCT(test, \
+ KUNIT_INIT_BINARY_STR_ASSERT_STRUCT(test, \
assert_type, \
#op, \
#left, \
void __user *arg);
#endif
int (*send)(struct atm_vcc *vcc,struct sk_buff *skb);
+ int (*send_bh)(struct atm_vcc *vcc, struct sk_buff *skb);
int (*send_oam)(struct atm_vcc *vcc,void *cell,int flags);
void (*phy_put)(struct atm_dev *dev,unsigned char value,
unsigned long addr);
#define BOOTCONFIG_MAGIC "#BOOTCONFIG\n"
#define BOOTCONFIG_MAGIC_LEN 12
+#define BOOTCONFIG_ALIGN_SHIFT 2
+#define BOOTCONFIG_ALIGN (1 << BOOTCONFIG_ALIGN_SHIFT)
+#define BOOTCONFIG_ALIGN_MASK (BOOTCONFIG_ALIGN - 1)
/* XBC tree node */
struct xbc_node {
}
/*
- * get_can_dlc(value) - helper macro to cast a given data length code (dlc)
- * to u8 and ensure the dlc value to be max. 8 bytes.
+ * can_cc_dlc2len(value) - convert a given data length code (dlc) of a
+ * Classical CAN frame into a valid data length of max. 8 bytes.
*
* To be used in the CAN netdriver receive path to ensure conformance with
* ISO 11898-1 Chapter 8.4.2.3 (DLC field)
*/
-#define get_can_dlc(i) (min_t(u8, (i), CAN_MAX_DLC))
-#define get_canfd_dlc(i) (min_t(u8, (i), CANFD_MAX_DLC))
+#define can_cc_dlc2len(dlc) (min_t(u8, (dlc), CAN_MAX_DLEN))
/* Check for outgoing skbs that have not been created by the CAN subsystem */
static inline bool can_skb_headroom_valid(struct net_device *dev,
return skb->len == CANFD_MTU;
}
+/* helper to get the data length code (DLC) for Classical CAN raw DLC access */
+static inline u8 can_get_cc_dlc(const struct can_frame *cf, const u32 ctrlmode)
+{
+ /* return len8_dlc as dlc value only if all conditions apply */
+ if ((ctrlmode & CAN_CTRLMODE_CC_LEN8_DLC) &&
+ (cf->len == CAN_MAX_DLEN) &&
+ (cf->len8_dlc > CAN_MAX_DLEN && cf->len8_dlc <= CAN_MAX_RAW_DLC))
+ return cf->len8_dlc;
+
+ /* return the payload length as dlc value */
+ return cf->len;
+}
+
+/* helper to set len and len8_dlc value for Classical CAN raw DLC access */
+static inline void can_frame_set_cc_len(struct can_frame *cf, const u8 dlc,
+ const u32 ctrlmode)
+{
+ /* the caller already ensured that dlc is a value from 0 .. 15 */
+ if (ctrlmode & CAN_CTRLMODE_CC_LEN8_DLC && dlc > CAN_MAX_DLEN)
+ cf->len8_dlc = dlc;
+
+ /* limit the payload length 'len' to CAN_MAX_DLEN */
+ cf->len = can_cc_dlc2len(dlc);
+}
+
/* helper to define static CAN controller features at device creation time */
static inline void can_set_static_ctrlmode(struct net_device *dev,
u32 static_mode)
dev->mtu = CANFD_MTU;
}
-/* get data length from can_dlc with sanitized can_dlc */
-u8 can_dlc2len(u8 can_dlc);
+/* get data length from raw data length code (DLC) */
+u8 can_fd_dlc2len(u8 dlc);
/* map the sanitized data length to an appropriate data length code */
-u8 can_len2dlc(u8 len);
+u8 can_fd_len2dlc(u8 len);
struct net_device *alloc_candev_mqs(int sizeof_priv, unsigned int echo_skb_max,
unsigned int txqs, unsigned int rxqs);
}
/* return the dlc value from any received message channel_dlc field */
-static inline int pucan_msg_get_dlc(const struct pucan_rx_msg *msg)
+static inline u8 pucan_msg_get_dlc(const struct pucan_rx_msg *msg)
{
return msg->channel_dlc >> 4;
}
+ __clang_patchlevel__)
#if CLANG_VERSION < 100001
+#ifndef __BPF_TRACING__
# error Sorry, your version of Clang is too old - please use 10.0.1 or newer.
#endif
+#endif
/* Compiler specific definitions for Clang compiler */
#define COMPILER_HAS_GENERIC_BUILTIN_OVERFLOW 1
#endif
-/* The following are for compatibility with GCC, from compiler-gcc.h,
- * and may be redefined here because they should not be shared with other
- * compilers, like ICC.
- */
-#define barrier() __asm__ __volatile__("" : : : "memory")
-
#if __has_feature(shadow_call_stack)
# define __noscs __attribute__((__no_sanitize__("shadow-call-stack")))
#endif
# error Sorry, your version of GCC is too old - please use 4.9 or newer.
#endif
-/* Optimization barrier */
-
-/* The "volatile" is due to gcc bugs */
-#define barrier() __asm__ __volatile__("": : :"memory")
-/*
- * This version is i.e. to prevent dead stores elimination on @ptr
- * where gcc and llvm may behave differently when otherwise using
- * normal barrier(): while gcc behavior gets along with a normal
- * barrier(), llvm needs an explicit input variable to be assumed
- * clobbered. The issue is as follows: while the inline asm might
- * access any memory it wants, the compiler could have fit all of
- * @ptr into memory registers instead, and since @ptr never escaped
- * from that, it proved that the inline asm wasn't touching any of
- * it. This version works well with both compilers, i.e. we're telling
- * the compiler that the inline asm absolutely may see the contents
- * of @ptr. See also: https://llvm.org/bugs/show_bug.cgi?id=15495
- */
-#define barrier_data(ptr) __asm__ __volatile__("": :"r"(ptr) :"memory")
-
/*
* This macro obfuscates arithmetic on a variable address so that gcc
* shouldn't recognize the original var, and make assumptions about it.
/* Optimization barrier */
#ifndef barrier
-# define barrier() __memory_barrier()
+/* The "volatile" is due to gcc bugs */
+# define barrier() __asm__ __volatile__("": : :"memory")
#endif
#ifndef barrier_data
-# define barrier_data(ptr) barrier()
+/*
+ * This version is i.e. to prevent dead stores elimination on @ptr
+ * where gcc and llvm may behave differently when otherwise using
+ * normal barrier(): while gcc behavior gets along with a normal
+ * barrier(), llvm needs an explicit input variable to be assumed
+ * clobbered. The issue is as follows: while the inline asm might
+ * access any memory it wants, the compiler could have fit all of
+ * @ptr into memory registers instead, and since @ptr never escaped
+ * from that, it proved that the inline asm wasn't touching any of
+ * it. This version works well with both compilers, i.e. we're telling
+ * the compiler that the inline asm absolutely may see the contents
+ * of @ptr. See also: https://llvm.org/bugs/show_bug.cgi?id=15495
+ */
+# define barrier_data(ptr) __asm__ __volatile__("": :"r"(ptr) :"memory")
#endif
/* workaround for GCC PR82365 if needed */
#define ETHTOOL_COALESCE_TX_USECS_HIGH BIT(19)
#define ETHTOOL_COALESCE_TX_MAX_FRAMES_HIGH BIT(20)
#define ETHTOOL_COALESCE_RATE_SAMPLE_INTERVAL BIT(21)
+#define ETHTOOL_COALESCE_ALL_PARAMS GENMASK(21, 0)
#define ETHTOOL_COALESCE_USECS \
(ETHTOOL_COALESCE_RX_USECS | ETHTOOL_COALESCE_TX_USECS)
#define ZYNQMP_PM_CAPABILITY_WAKEUP 0x4U
#define ZYNQMP_PM_CAPABILITY_UNUSABLE 0x8U
-/* Feature check status */
-#define PM_FEATURE_INVALID -1
-#define PM_FEATURE_UNCHECKED 0
-
/*
* Firmware FPGA Manager flags
* XILINX_ZYNQMP_PM_FPGA_FULL: FPGA full reconfiguration
* Snapshotting support.
*/
-void __sb_end_write(struct super_block *sb, int level);
-int __sb_start_write(struct super_block *sb, int level, bool wait);
+/*
+ * These are internal functions, please use sb_start_{write,pagefault,intwrite}
+ * instead.
+ */
+static inline void __sb_end_write(struct super_block *sb, int level)
+{
+ percpu_up_read(sb->s_writers.rw_sem + level-1);
+}
+
+static inline void __sb_start_write(struct super_block *sb, int level)
+{
+ percpu_down_read(sb->s_writers.rw_sem + level - 1);
+}
+
+static inline bool __sb_start_write_trylock(struct super_block *sb, int level)
+{
+ return percpu_down_read_trylock(sb->s_writers.rw_sem + level - 1);
+}
#define __sb_writers_acquired(sb, lev) \
percpu_rwsem_acquire(&(sb)->s_writers.rw_sem[(lev)-1], 1, _THIS_IP_)
*/
static inline void sb_start_write(struct super_block *sb)
{
- __sb_start_write(sb, SB_FREEZE_WRITE, true);
+ __sb_start_write(sb, SB_FREEZE_WRITE);
}
-static inline int sb_start_write_trylock(struct super_block *sb)
+static inline bool sb_start_write_trylock(struct super_block *sb)
{
- return __sb_start_write(sb, SB_FREEZE_WRITE, false);
+ return __sb_start_write_trylock(sb, SB_FREEZE_WRITE);
}
/**
*/
static inline void sb_start_pagefault(struct super_block *sb)
{
- __sb_start_write(sb, SB_FREEZE_PAGEFAULT, true);
+ __sb_start_write(sb, SB_FREEZE_PAGEFAULT);
}
/*
*/
static inline void sb_start_intwrite(struct super_block *sb)
{
- __sb_start_write(sb, SB_FREEZE_FS, true);
+ __sb_start_write(sb, SB_FREEZE_FS);
}
-static inline int sb_start_intwrite_trylock(struct super_block *sb)
+static inline bool sb_start_intwrite_trylock(struct super_block *sb)
{
- return __sb_start_write(sb, SB_FREEZE_FS, false);
+ return __sb_start_write_trylock(sb, SB_FREEZE_FS);
}
{
if (!S_ISREG(file_inode(file)->i_mode))
return;
- __sb_start_write(file_inode(file)->i_sb, SB_FREEZE_WRITE, true);
+ sb_start_write(file_inode(file)->i_sb);
}
static inline bool file_start_write_trylock(struct file *file)
{
if (!S_ISREG(file_inode(file)->i_mode))
return true;
- return __sb_start_write(file_inode(file)->i_sb, SB_FREEZE_WRITE, false);
+ return sb_start_write_trylock(file_inode(file)->i_sb);
}
static inline void file_end_write(struct file *file)
extern void disk_block_events(struct gendisk *disk);
extern void disk_unblock_events(struct gendisk *disk);
extern void disk_flush_events(struct gendisk *disk, unsigned int mask);
-void set_capacity_revalidate_and_notify(struct gendisk *disk, sector_t size,
+bool set_capacity_revalidate_and_notify(struct gendisk *disk, sector_t size,
bool update_bdev);
/* drivers/char/random.c */
nla_get_u64, nla_put_u64_0pad, false)
#define __str_field(attr_nr, attr_flag, name, maxlen) \
__array(attr_nr, attr_flag, name, NLA_NUL_STRING, char, maxlen, \
- nla_strlcpy, nla_put, false)
+ nla_strscpy, nla_put, false)
#define __bin_field(attr_nr, attr_flag, name, maxlen) \
__array(attr_nr, attr_flag, name, NLA_BINARY, char, maxlen, \
nla_memcpy, nla_put, false)
+++ /dev/null
-/* SPDX-License-Identifier: GPL-2.0-or-later */
-/*
- * DLCI/FRAD Definitions for Frame Relay Access Devices. DLCI devices are
- * created for each DLCI associated with a FRAD. The FRAD driver
- * is not truly a network device, but the lower level device
- * handler. This allows other FRAD manufacturers to use the DLCI
- * code, including its RFC1490 encapsulation alongside the current
- * implementation for the Sangoma cards.
- *
- * Version: @(#)if_ifrad.h 0.15 31 Mar 96
- *
- * Author: Mike McLagan <mike.mclagan@linux.org>
- *
- * Changes:
- * 0.15 Mike McLagan changed structure defs (packed)
- * re-arranged flags
- * added DLCI_RET vars
- */
-#ifndef _FRAD_H_
-#define _FRAD_H_
-
-#include <uapi/linux/if_frad.h>
-
-
-#if defined(CONFIG_DLCI) || defined(CONFIG_DLCI_MODULE)
-
-/* these are the fields of an RFC 1490 header */
-struct frhdr
-{
- unsigned char control;
-
- /* for IP packets, this can be the NLPID */
- unsigned char pad;
-
- unsigned char NLPID;
- unsigned char OUI[3];
- __be16 PID;
-
-#define IP_NLPID pad
-} __packed;
-
-/* see RFC 1490 for the definition of the following */
-#define FRAD_I_UI 0x03
-
-#define FRAD_P_PADDING 0x00
-#define FRAD_P_Q933 0x08
-#define FRAD_P_SNAP 0x80
-#define FRAD_P_CLNP 0x81
-#define FRAD_P_IP 0xCC
-
-struct dlci_local
-{
- struct net_device *master;
- struct net_device *slave;
- struct dlci_conf config;
- int configured;
- struct list_head list;
-
- /* callback function */
- void (*receive)(struct sk_buff *skb, struct net_device *);
-};
-
-struct frad_local
-{
- /* devices which this FRAD is slaved to */
- struct net_device *master[CONFIG_DLCI_MAX];
- short dlci[CONFIG_DLCI_MAX];
-
- struct frad_conf config;
- int configured; /* has this device been configured */
- int initialized; /* mem_start, port, irq set ? */
-
- /* callback functions */
- int (*activate)(struct net_device *, struct net_device *);
- int (*deactivate)(struct net_device *, struct net_device *);
- int (*assoc)(struct net_device *, struct net_device *);
- int (*deassoc)(struct net_device *, struct net_device *);
- int (*dlci_conf)(struct net_device *, struct net_device *, int get);
-
- /* fields that are used by the Sangoma SDLA cards */
- struct timer_list timer;
- struct net_device *dev;
- int type; /* adapter type */
- int state; /* state of the S502/8 control latch */
- int buffer; /* current buffer for S508 firmware */
-};
-
-#endif /* CONFIG_DLCI || CONFIG_DLCI_MODULE */
-
-extern void dlci_ioctl_set(int (*hook)(unsigned int, void __user *));
-
-#endif
enum macvlan_mode mode;
u16 flags;
unsigned int macaddr_count;
+ u32 bc_queue_len_req;
#ifdef CONFIG_NET_POLL_CONTROLLER
struct netpoll *netpoll;
#endif
extern int iommu_calculate_max_sagaw(struct intel_iommu *iommu);
extern int dmar_disabled;
extern int intel_iommu_enabled;
-extern int intel_iommu_tboot_noforce;
extern int intel_iommu_gfx_mapped;
#else
static inline int iommu_calculate_agaw(struct intel_iommu *iommu)
#define JI_WAIT_DATA (1 << __JI_WAIT_DATA)
/**
- * struct jbd_inode - The jbd_inode type is the structure linking inodes in
+ * struct jbd2_inode - The jbd_inode type is the structure linking inodes in
* ordered mode present in a transaction so that we can sync them during commit.
*/
struct jbd2_inode {
* clear the contents.
*/
struct key_preparsed_payload {
+ const char *orig_description; /* Actual or proposed description (maybe NULL) */
char *description; /* Proposed key description (or NULL) */
union key_payload payload; /* Proposed payload */
const void *data; /* Raw data */
this_cpu_read(hardirqs_enabled))); \
} while (0)
+/*
+ * Acceptable for protecting per-CPU resources accessed from BH.
+ * Much like in_softirq() - semantics are ambiguous, use carefully.
+ */
+#define lockdep_assert_in_softirq() \
+do { \
+ WARN_ON_ONCE(__lockdep_enabled && \
+ (!in_softirq() || in_irq() || in_nmi())); \
+} while (0)
+
#else
# define might_lock(lock) do { } while (0)
# define might_lock_read(lock) do { } while (0)
# define lockdep_assert_preemption_enabled() do { } while (0)
# define lockdep_assert_preemption_disabled() do { } while (0)
+# define lockdep_assert_in_softirq() do { } while (0)
#endif
#ifdef CONFIG_PROVE_RAW_LOCK_NESTING
struct lsm_network_audit {
int netif;
- struct sock *sk;
+ const struct sock *sk;
u16 family;
__be16 dport;
__be16 sport;
struct sock *newsk)
LSM_HOOK(void, LSM_RET_VOID, sk_getsecid, struct sock *sk, u32 *secid)
LSM_HOOK(void, LSM_RET_VOID, sock_graft, struct sock *sk, struct socket *parent)
-LSM_HOOK(int, 0, inet_conn_request, struct sock *sk, struct sk_buff *skb,
+LSM_HOOK(int, 0, inet_conn_request, const struct sock *sk, struct sk_buff *skb,
struct request_sock *req)
LSM_HOOK(void, LSM_RET_VOID, inet_csk_clone, struct sock *newsk,
const struct request_sock *req)
MEMCG_PADDING(_pad1_);
- /*
- * set > 0 if pages under this cgroup are moving to other cgroup.
- */
- atomic_t moving_account;
- struct task_struct *move_lock_task;
-
- /* Legacy local VM stats and events */
- struct memcg_vmstats_percpu __percpu *vmstats_local;
-
- /* Subtree VM stats and events (batched updates) */
- struct memcg_vmstats_percpu __percpu *vmstats_percpu;
-
- MEMCG_PADDING(_pad2_);
-
atomic_long_t vmstats[MEMCG_NR_STAT];
atomic_long_t vmevents[NR_VM_EVENT_ITEMS];
struct list_head objcg_list; /* list of inherited objcgs */
#endif
+ MEMCG_PADDING(_pad2_);
+
+ /*
+ * set > 0 if pages under this cgroup are moving to other cgroup.
+ */
+ atomic_t moving_account;
+ struct task_struct *move_lock_task;
+
+ /* Legacy local VM stats and events */
+ struct memcg_vmstats_percpu __percpu *vmstats_local;
+
+ /* Subtree VM stats and events (batched updates) */
+ struct memcg_vmstats_percpu __percpu *vmstats_percpu;
+
#ifdef CONFIG_CGROUP_WRITEBACK
struct list_head cgwb_list;
struct wb_domain cgwb_domain;
static inline void memcg_memory_event(struct mem_cgroup *memcg,
enum memcg_memory_event event)
{
+ bool swap_event = event == MEMCG_SWAP_HIGH || event == MEMCG_SWAP_MAX ||
+ event == MEMCG_SWAP_FAIL;
+
atomic_long_inc(&memcg->memory_events_local[event]);
- cgroup_file_notify(&memcg->events_local_file);
+ if (!swap_event)
+ cgroup_file_notify(&memcg->events_local_file);
do {
atomic_long_inc(&memcg->memory_events[event]);
- cgroup_file_notify(&memcg->events_file);
+ if (swap_event)
+ cgroup_file_notify(&memcg->swap_events_file);
+ else
+ cgroup_file_notify(&memcg->events_file);
if (!cgroup_subsys_on_dfl(memory_cgrp_subsys))
break;
}
#endif /* ! CONFIG_MEMORY_HOTPLUG */
-#ifdef CONFIG_NUMA
-extern int memory_add_physaddr_to_nid(u64 start);
-extern int phys_to_target_node(u64 start);
-#else
-static inline int memory_add_physaddr_to_nid(u64 start)
-{
- return 0;
-}
-static inline int phys_to_target_node(u64 start)
-{
- return 0;
-}
-#endif
-
#if defined(CONFIG_MEMORY_HOTPLUG) || defined(CONFIG_DEFERRED_STRUCT_PAGE_INIT)
/*
* pgdat resizing functions
MLX5_EVENT_TYPE_NIC_VPORT_CHANGE = 0xd,
MLX5_EVENT_TYPE_ESW_FUNCTIONS_CHANGED = 0xe,
+ MLX5_EVENT_TYPE_VHCA_STATE_CHANGE = 0xf,
MLX5_EVENT_TYPE_DCT_DRAINED = 0x1c,
MLX5_EVENT_TYPE_DCT_KEY_VIOLATION = 0x1d,
u8 sync_rst_state;
};
+struct mlx5_eqe_vhca_state {
+ __be16 ec_function;
+ __be16 function_id;
+} __packed;
+
union ev_data {
__be32 raw[7];
struct mlx5_eqe_cmd cmd;
struct mlx5_eqe_temp_warning temp_warning;
struct mlx5_eqe_xrq_err xrq_err;
struct mlx5_eqe_sync_fw_update sync_fw_update;
+ struct mlx5_eqe_vhca_state vhca_state;
} __packed;
struct mlx5_eqe {
MLX5_MATCH_INNER_HEADERS = 1 << 2,
MLX5_MATCH_MISC_PARAMETERS_2 = 1 << 3,
MLX5_MATCH_MISC_PARAMETERS_3 = 1 << 4,
+ MLX5_MATCH_MISC_PARAMETERS_4 = 1 << 5,
};
enum {
atomic_t reg_pages;
struct list_head free_list;
int vfs_pages;
- int peer_pf_pages;
+ int host_pf_pages;
struct mlx5_core_health health;
CMD_ALLOWED_OPCODE_ALL,
};
-int mlx5_cmd_init(struct mlx5_core_dev *dev);
-void mlx5_cmd_cleanup(struct mlx5_core_dev *dev);
-void mlx5_cmd_set_state(struct mlx5_core_dev *dev,
- enum mlx5_cmdif_state cmdif_state);
void mlx5_cmd_use_events(struct mlx5_core_dev *dev);
void mlx5_cmd_use_polling(struct mlx5_core_dev *dev);
void mlx5_cmd_allowed_opcode(struct mlx5_core_dev *dev, u16 opcode);
return dev->coredev_type == MLX5_COREDEV_VF;
}
-static inline bool mlx5_core_is_ecpf(struct mlx5_core_dev *dev)
+static inline bool mlx5_core_is_ecpf(const struct mlx5_core_dev *dev)
{
return dev->caps.embedded_cpu;
}
MLX5_FLOW_TABLE_TUNNEL_EN_DECAP = BIT(1),
MLX5_FLOW_TABLE_TERMINATION = BIT(2),
MLX5_FLOW_TABLE_UNMANAGED = BIT(3),
+ MLX5_FLOW_TABLE_OTHER_VPORT = BIT(4),
};
#define LEFTOVERS_RULE_NUM 2
struct mlx5_pkt_reformat *pkt_reformat;
u8 flags;
} vport;
+ u32 sampler_id;
};
};
struct mlx5_flow_table *
mlx5_create_vport_flow_table(struct mlx5_flow_namespace *ns,
- int prio,
- int num_flow_table_entries,
- u32 level, u16 vport);
+ struct mlx5_flow_table_attr *ft_attr, u16 vport);
struct mlx5_flow_table *mlx5_create_lag_demux_flow_table(
struct mlx5_flow_namespace *ns,
int prio, u32 level);
MLX5_CMD_OP_CREATE_UMEM = 0xa08,
MLX5_CMD_OP_DESTROY_UMEM = 0xa0a,
MLX5_CMD_OP_SYNC_STEERING = 0xb00,
+ MLX5_CMD_OP_QUERY_VHCA_STATE = 0xb0d,
+ MLX5_CMD_OP_MODIFY_VHCA_STATE = 0xb0e,
MLX5_CMD_OP_MAX
};
u8 reserved_at_140[0xc0];
};
+struct mlx5_ifc_fte_match_set_misc4_bits {
+ u8 prog_sample_field_value_0[0x20];
+
+ u8 prog_sample_field_id_0[0x20];
+
+ u8 prog_sample_field_value_1[0x20];
+
+ u8 prog_sample_field_id_1[0x20];
+
+ u8 prog_sample_field_value_2[0x20];
+
+ u8 prog_sample_field_id_2[0x20];
+
+ u8 prog_sample_field_value_3[0x20];
+
+ u8 prog_sample_field_id_3[0x20];
+
+ u8 reserved_at_100[0x100];
+};
+
struct mlx5_ifc_cmd_pas_bits {
u8 pa_h[0x20];
u8 tunnel_stateless_ipv4_over_vxlan[0x1];
u8 tunnel_stateless_ip_over_ip[0x1];
u8 insert_trailer[0x1];
- u8 reserved_at_2b[0x5];
+ u8 reserved_at_2b[0x1];
+ u8 tunnel_stateless_ip_over_ip_rx[0x1];
+ u8 tunnel_stateless_ip_over_ip_tx[0x1];
+ u8 reserved_at_2e[0x2];
u8 max_vxlan_udp_ports[0x8];
u8 reserved_at_38[0x6];
u8 max_geneve_opt_len[0x1];
#define MLX5_FC_BULK_NUM_FCS(fc_enum) (MLX5_FC_BULK_SIZE_FACTOR * (fc_enum))
+enum {
+ MLX5_STEERING_FORMAT_CONNECTX_5 = 0,
+ MLX5_STEERING_FORMAT_CONNECTX_6DX = 1,
+};
+
struct mlx5_ifc_cmd_hca_cap_bits {
- u8 reserved_at_0[0x30];
+ u8 reserved_at_0[0x1f];
+ u8 vhca_resource_manager[0x1];
+
+ u8 reserved_at_20[0x3];
+ u8 event_on_vhca_state_teardown_request[0x1];
+ u8 event_on_vhca_state_in_use[0x1];
+ u8 event_on_vhca_state_active[0x1];
+ u8 event_on_vhca_state_allocated[0x1];
+ u8 event_on_vhca_state_invalid[0x1];
+ u8 reserved_at_28[0x8];
u8 vhca_id[0x10];
u8 reserved_at_40[0x40];
u8 ece_support[0x1];
u8 reserved_at_a4[0x7];
u8 log_max_srq[0x5];
- u8 reserved_at_b0[0x10];
+ u8 reserved_at_b0[0x2];
+ u8 ts_cqe_to_dest_cqn[0x1];
+ u8 reserved_at_b3[0xd];
u8 max_sgl_for_optimized_performance[0x8];
u8 log_max_cq_sz[0x8];
u8 disable_local_lb_uc[0x1];
u8 disable_local_lb_mc[0x1];
u8 log_min_hairpin_wq_data_sz[0x5];
- u8 reserved_at_3e8[0x3];
+ u8 reserved_at_3e8[0x2];
+ u8 vhca_state[0x1];
u8 log_max_vlan_list[0x5];
u8 reserved_at_3f0[0x3];
u8 log_max_current_mc_list[0x5];
u8 general_obj_types[0x40];
- u8 reserved_at_440[0x20];
+ u8 reserved_at_440[0x4];
+ u8 steering_format_version[0x4];
+ u8 create_qp_start_hint[0x18];
u8 reserved_at_460[0x3];
u8 log_max_uctx[0x5];
u8 max_num_of_monitor_counters[0x10];
u8 num_ppcnt_monitor_counters[0x10];
- u8 reserved_at_640[0x10];
+ u8 max_num_sf[0x10];
u8 num_q_monitor_counters[0x10];
u8 reserved_at_660[0x20];
MLX5_FLOW_DESTINATION_TYPE_VPORT = 0x0,
MLX5_FLOW_DESTINATION_TYPE_FLOW_TABLE = 0x1,
MLX5_FLOW_DESTINATION_TYPE_TIR = 0x2,
+ MLX5_FLOW_DESTINATION_TYPE_FLOW_SAMPLER = 0x6,
MLX5_FLOW_DESTINATION_TYPE_PORT = 0x99,
MLX5_FLOW_DESTINATION_TYPE_COUNTER = 0x100,
struct mlx5_ifc_fte_match_set_misc3_bits misc_parameters_3;
- u8 reserved_at_a00[0x600];
+ struct mlx5_ifc_fte_match_set_misc4_bits misc_parameters_4;
+
+ u8 reserved_at_c00[0x400];
};
enum {
u8 reserved_at_80[0x10];
u8 hairpin_peer_vhca[0x10];
- u8 reserved_at_a0[0x50];
+ u8 reserved_at_a0[0x20];
+
+ u8 reserved_at_c0[0x8];
+ u8 ts_cqe_to_dest_cqn[0x18];
+ u8 reserved_at_e0[0x10];
u8 packet_pacing_rate_limit_index[0x10];
u8 tis_lst_sz[0x10];
u8 reserved_at_110[0x10];
u8 reserved_at_20[0x10];
u8 op_mod[0x10];
- u8 reserved_at_40[0x40];
+ u8 other_function[0x1];
+ u8 reserved_at_41[0xf];
+ u8 function_id[0x10];
+
+ u8 reserved_at_60[0x20];
union mlx5_ifc_hca_cap_union_bits capability;
};
MLX5_QUERY_FLOW_GROUP_OUT_MATCH_CRITERIA_ENABLE_INNER_HEADERS = 0x2,
MLX5_QUERY_FLOW_GROUP_IN_MATCH_CRITERIA_ENABLE_MISC_PARAMETERS_2 = 0x3,
MLX5_QUERY_FLOW_GROUP_IN_MATCH_CRITERIA_ENABLE_MISC_PARAMETERS_3 = 0x4,
+ MLX5_QUERY_FLOW_GROUP_IN_MATCH_CRITERIA_ENABLE_MISC_PARAMETERS_4 = 0x5,
};
struct mlx5_ifc_query_flow_group_out_bits {
enum {
MLX5_HCA_CAP_GENERAL_OBJECT_TYPES_ENCRYPTION_KEY = BIT(0xc),
MLX5_HCA_CAP_GENERAL_OBJECT_TYPES_IPSEC = BIT(0x13),
+ MLX5_HCA_CAP_GENERAL_OBJECT_TYPES_SAMPLER = BIT(0x20),
};
enum {
MLX5_GENERAL_OBJECT_TYPES_ENCRYPTION_KEY = 0xc,
MLX5_GENERAL_OBJECT_TYPES_IPSEC = 0x13,
+ MLX5_GENERAL_OBJECT_TYPES_SAMPLER = 0x20,
};
enum {
struct mlx5_ifc_encryption_key_obj_bits encryption_key_object;
};
+struct mlx5_ifc_sampler_obj_bits {
+ u8 modify_field_select[0x40];
+
+ u8 table_type[0x8];
+ u8 level[0x8];
+ u8 reserved_at_50[0xf];
+ u8 ignore_flow_level[0x1];
+
+ u8 sample_ratio[0x20];
+
+ u8 reserved_at_80[0x8];
+ u8 sample_table_id[0x18];
+
+ u8 reserved_at_a0[0x8];
+ u8 default_table_id[0x18];
+
+ u8 sw_steering_icm_address_rx[0x40];
+ u8 sw_steering_icm_address_tx[0x40];
+
+ u8 reserved_at_140[0xa0];
+};
+
+struct mlx5_ifc_create_sampler_obj_in_bits {
+ struct mlx5_ifc_general_obj_in_cmd_hdr_bits general_obj_in_cmd_hdr;
+ struct mlx5_ifc_sampler_obj_bits sampler_object;
+};
+
enum {
MLX5_GENERAL_OBJECT_TYPE_ENCRYPTION_KEY_KEY_SIZE_128 = 0x0,
MLX5_GENERAL_OBJECT_TYPE_ENCRYPTION_KEY_KEY_SIZE_256 = 0x1,
#include <linux/workqueue.h>
#include <linux/dynamic_queue_limits.h>
-#include <linux/ethtool.h>
#include <net/net_namespace.h>
#ifdef CONFIG_DCB
#include <net/dcbnl.h>
struct netpoll_info;
struct device;
+struct ethtool_ops;
struct phy_device;
struct dsa_port;
struct ip_tunnel_parm;
};
/**
- * enum net_device_priv_flags - &struct net_device priv_flags
+ * enum netdev_priv_flags - &struct net_device priv_flags
*
* These are the &struct net_device, they are only set internally
* by drivers and used in the kernel. These flags are invisible to
struct net_device *dev_getbyhwaddr_rcu(struct net *net, unsigned short type,
const char *hwaddr);
struct net_device *dev_getfirstbyhwtype(struct net *net, unsigned short type);
-struct net_device *__dev_getfirstbyhwtype(struct net *net, unsigned short type);
void dev_add_pack(struct packet_type *pt);
void dev_remove_pack(struct packet_type *pt);
void __dev_remove_pack(struct packet_type *pt);
struct net_device *sb_dev);
u16 dev_pick_tx_cpu_id(struct net_device *dev, struct sk_buff *skb,
struct net_device *sb_dev);
+
int dev_queue_xmit(struct sk_buff *skb);
int dev_queue_xmit_accel(struct sk_buff *skb, struct net_device *sb_dev);
-int dev_direct_xmit(struct sk_buff *skb, u16 queue_id);
+int __dev_direct_xmit(struct sk_buff *skb, u16 queue_id);
+
+static inline int dev_direct_xmit(struct sk_buff *skb, u16 queue_id)
+{
+ int ret;
+
+ ret = __dev_direct_xmit(skb, queue_id);
+ if (!dev_xmit_complete(ret))
+ kfree_skb(skb);
+ return ret;
+}
+
int register_netdevice(struct net_device *dev);
void unregister_netdevice_queue(struct net_device *dev, struct list_head *head);
void unregister_netdevice_many(struct list_head *head);
return false;
}
+static inline bool dev_has_header(const struct net_device *dev)
+{
+ return dev->header_ops && dev->header_ops->create;
+}
+
typedef int gifconf_func_t(struct net_device * dev, char __user * bufptr,
int len, int size);
int register_gifconf(unsigned int family, gifconf_func_t *gifconf);
}
/**
- * netif_subqueue_stopped - test status of subqueue
+ * __netif_subqueue_stopped - test status of subqueue
* @dev: network device
* @queue_index: sub queue index
*
return netif_tx_queue_stopped(txq);
}
+/**
+ * netif_subqueue_stopped - test status of subqueue
+ * @dev: network device
+ * @skb: sub queue buffer pointer
+ *
+ * Check individual transmit queue of a device with multiple transmit queues.
+ */
static inline bool netif_subqueue_stopped(const struct net_device *dev,
struct sk_buff *skb)
{
#endif
#ifdef CONFIG_NUMA
+#include <linux/printk.h>
+#include <asm/sparsemem.h>
+
/* Generic implementation available */
int numa_map_to_online_node(int node);
-#else
+
+#ifndef memory_add_physaddr_to_nid
+static inline int memory_add_physaddr_to_nid(u64 start)
+{
+ pr_info_once("Unknown online node for memory at 0x%llx, assuming node 0\n",
+ start);
+ return 0;
+}
+#endif
+#ifndef phys_to_target_node
+static inline int phys_to_target_node(u64 start)
+{
+ pr_info_once("Unknown target node for memory at 0x%llx, assuming node 0\n",
+ start);
+ return 0;
+}
+#endif
+#else /* !CONFIG_NUMA */
static inline int numa_map_to_online_node(int node)
{
return NUMA_NO_NODE;
}
+static inline int memory_add_physaddr_to_nid(u64 start)
+{
+ return 0;
+}
+static inline int phys_to_target_node(u64 start)
+{
+ return 0;
+}
#endif
#endif /* _LINUX_NUMA_H */
xas_set(&xas, rac->_index);
rcu_read_lock();
xas_for_each(&xas, page, rac->_index + rac->_nr_pages - 1) {
+ if (xas_retry(&xas, page))
+ continue;
VM_BUG_ON_PAGE(!PageLocked(page), page);
VM_BUG_ON_PAGE(PageTail(page), page);
array[i++] = page;
struct perf_callchain_entry *callchain;
u64 aux_size;
- /*
- * regs_user may point to task_pt_regs or to regs_user_copy, depending
- * on arch details.
- */
struct perf_regs regs_user;
- struct pt_regs regs_user_copy;
-
struct perf_regs regs_intr;
u64 stack_user_size;
extern void perf_event_addr_filters_sync(struct perf_event *event);
extern int perf_output_begin(struct perf_output_handle *handle,
+ struct perf_sample_data *data,
struct perf_event *event, unsigned int size);
extern int perf_output_begin_forward(struct perf_output_handle *handle,
- struct perf_event *event,
- unsigned int size);
+ struct perf_sample_data *data,
+ struct perf_event *event,
+ unsigned int size);
extern int perf_output_begin_backward(struct perf_output_handle *handle,
+ struct perf_sample_data *data,
struct perf_event *event,
unsigned int size);
int perf_reg_validate(u64 mask);
u64 perf_reg_abi(struct task_struct *task);
void perf_get_regs_user(struct perf_regs *regs_user,
- struct pt_regs *regs,
- struct pt_regs *regs_user_copy);
+ struct pt_regs *regs);
#else
#define PERF_REG_EXTENDED_MASK 0
}
static inline void perf_get_regs_user(struct perf_regs *regs_user,
- struct pt_regs *regs,
- struct pt_regs *regs_user_copy)
+ struct pt_regs *regs)
{
regs_user->regs = task_pt_regs(current);
regs_user->abi = perf_reg_abi(current);
#endif /* !__ASSEMBLY__ */
+#if !defined(MAX_POSSIBLE_PHYSMEM_BITS) && !defined(CONFIG_64BIT)
+#ifdef CONFIG_PHYS_ADDR_T_64BIT
+/*
+ * ZSMALLOC needs to know the highest PFN on 32-bit architectures
+ * with physical address space extension, but falls back to
+ * BITS_PER_LONG otherwise.
+ */
+#error Missing MAX_POSSIBLE_PHYSMEM_BITS definition
+#else
+#define MAX_POSSIBLE_PHYSMEM_BITS 32
+#endif
+#endif
+
#ifndef has_transparent_hugepage
#ifdef CONFIG_TRANSPARENT_HUGEPAGE
#define has_transparent_hugepage() 1
/** @read_status: Determines the negotiated speed and duplex */
int (*read_status)(struct phy_device *phydev);
- /** @ack_interrupt: Clears any pending interrupts */
- int (*ack_interrupt)(struct phy_device *phydev);
-
- /** @config_intr: Enables or disables interrupts */
- int (*config_intr)(struct phy_device *phydev);
-
- /**
- * @did_interrupt: Checks if the PHY generated an interrupt.
- * For multi-PHY devices with shared PHY interrupt pin
- * Set interrupt bits have to be cleared.
+ /** @config_intr: Enables or disables interrupts.
+ * It should also clear any pending interrupts prior to enabling the
+ * IRQs and after disabling them.
*/
- int (*did_interrupt)(struct phy_device *phydev);
+ int (*config_intr)(struct phy_device *phydev);
/** @handle_interrupt: Override default interrupt handling */
irqreturn_t (*handle_interrupt)(struct phy_device *phydev);
return __genphy_config_aneg(phydev, false);
}
-static inline int genphy_no_ack_interrupt(struct phy_device *phydev)
-{
- return 0;
-}
static inline int genphy_no_config_intr(struct phy_device *phydev)
{
return 0;
s8 emufree_shift;
};
+#define SYSC_MODULE_QUIRK_ENA_RESETDONE BIT(25)
#define SYSC_MODULE_QUIRK_PRUSS BIT(24)
#define SYSC_MODULE_QUIRK_DSS_RESET BIT(23)
#define SYSC_MODULE_QUIRK_RTC_UNLOCK BIT(22)
return __pm_runtime_resume(dev, RPM_GET_PUT);
}
+/**
+ * pm_runtime_resume_and_get - Bump up usage counter of a device and resume it.
+ * @dev: Target device.
+ *
+ * Resume @dev synchronously and if that is successful, increment its runtime
+ * PM usage counter. Return 0 if the runtime PM usage counter of @dev has been
+ * incremented or a negative error code otherwise.
+ */
+static inline int pm_runtime_resume_and_get(struct device *dev)
+{
+ int ret;
+
+ ret = __pm_runtime_resume(dev, RPM_GET_PUT);
+ if (ret < 0) {
+ pm_runtime_put_noidle(dev);
+ return ret;
+ }
+
+ return 0;
+}
+
/**
* pm_runtime_put - Drop device usage counter and queue up "idle check" if 0.
* @dev: Target device.
#define PTP_CLASS_V2_VLAN (PTP_CLASS_V2 | PTP_CLASS_VLAN)
#define PTP_CLASS_L4 (PTP_CLASS_IPV4 | PTP_CLASS_IPV6)
+#define PTP_MSGTYPE_SYNC 0x0
+#define PTP_MSGTYPE_DELAY_REQ 0x1
+#define PTP_MSGTYPE_PDELAY_REQ 0x2
+#define PTP_MSGTYPE_PDELAY_RESP 0x3
+
#define PTP_EV_PORT 319
#define PTP_GEN_BIT 0x08 /* indicates general message, if set in message type */
/* The return is meaningless. The stub function would not be
* executed since no available header from ptp_parse_header.
*/
- return 0;
+ return PTP_MSGTYPE_SYNC;
}
#endif
#endif /* _PTP_CLASSIFY_H_ */
#include <linux/pps_kernel.h>
#include <linux/ptp_clock.h>
+/**
+ * struct ptp_clock_request - request PTP clock event
+ *
+ * @type: The type of the request.
+ * EXTTS: Configure external trigger timestamping
+ * PEROUT: Configure periodic output signal (e.g. PPS)
+ * PPS: trigger internal PPS event for input
+ * into kernel PPS subsystem
+ * @extts: describes configuration for external trigger timestamping.
+ * This is only valid when event == PTP_CLK_REQ_EXTTS.
+ * @perout: describes configuration for periodic output.
+ * This is only valid when event == PTP_CLK_REQ_PEROUT.
+ */
struct ptp_clock_request {
enum {
#ifndef _QED_IF_H
#define _QED_IF_H
+#include <linux/ethtool.h>
#include <linux/types.h>
#include <linux/interrupt.h>
#include <linux/netdevice.h>
* overruns.
*/
unsigned int dl_throttled : 1;
- unsigned int dl_boosted : 1;
unsigned int dl_yielded : 1;
unsigned int dl_non_contending : 1;
unsigned int dl_overrun : 1;
* time.
*/
struct hrtimer inactive_timer;
+
+#ifdef CONFIG_RT_MUTEXES
+ /*
+ * Priority Inheritance. When a DEADLINE scheduling entity is boosted
+ * pi_se points to the donor, otherwise points to the dl_se it belongs
+ * to (the original one/itself).
+ */
+ struct sched_dl_entity *pi_se;
+#endif
};
#ifdef CONFIG_UCLAMP_TASK
unsigned sched_reset_on_fork:1;
unsigned sched_contributes_to_load:1;
unsigned sched_migrated:1;
- unsigned sched_remote_wakeup:1;
#ifdef CONFIG_PSI
unsigned sched_psi_wake_requeue:1;
#endif
/* Unserialized, strictly 'current' */
+ /*
+ * This field must not be in the scheduler word above due to wakelist
+ * queueing no longer being serialized by p->on_cpu. However:
+ *
+ * p->XXX = X; ttwu()
+ * schedule() if (p->on_rq && ..) // false
+ * smp_mb__after_spinlock(); if (smp_load_acquire(&p->on_cpu) && //true
+ * deactivate_task() ttwu_queue_wakelist())
+ * p->on_rq = 0; p->sched_remote_wakeup = Y;
+ *
+ * guarantees all stores of 'current' are visible before
+ * ->sched_remote_wakeup gets used, so it can be in this word.
+ */
+ unsigned sched_remote_wakeup:1;
+
/* Bit to tell LSMs we're in execve(): */
unsigned in_execve:1;
unsigned in_iowait:1;
+++ /dev/null
-/* SPDX-License-Identifier: GPL-2.0-or-later */
-/*
- * INET An implementation of the TCP/IP protocol suite for the LINUX
- * operating system. INET is implemented using the BSD Socket
- * interface as the means of communication with the user level.
- *
- * Global definitions for the Frame relay interface.
- *
- * Version: @(#)if_ifrad.h 0.20 13 Apr 96
- *
- * Author: Mike McLagan <mike.mclagan@linux.org>
- *
- * Changes:
- * 0.15 Mike McLagan Structure packing
- *
- * 0.20 Mike McLagan New flags for S508 buffer handling
- */
-#ifndef SDLA_H
-#define SDLA_H
-
-#include <uapi/linux/sdla.h>
-
-
-/* important Z80 window addresses */
-#define SDLA_CONTROL_WND 0xE000
-
-#define SDLA_502_CMD_BUF 0xEF60
-#define SDLA_502_RCV_BUF 0xA900
-#define SDLA_502_TXN_AVAIL 0xFFF1
-#define SDLA_502_RCV_AVAIL 0xFFF2
-#define SDLA_502_EVENT_FLAGS 0xFFF3
-#define SDLA_502_MDM_STATUS 0xFFF4
-#define SDLA_502_IRQ_INTERFACE 0xFFFD
-#define SDLA_502_IRQ_PERMISSION 0xFFFE
-#define SDLA_502_DATA_OFS 0x0010
-
-#define SDLA_508_CMD_BUF 0xE000
-#define SDLA_508_TXBUF_INFO 0xF100
-#define SDLA_508_RXBUF_INFO 0xF120
-#define SDLA_508_EVENT_FLAGS 0xF003
-#define SDLA_508_MDM_STATUS 0xF004
-#define SDLA_508_IRQ_INTERFACE 0xF010
-#define SDLA_508_IRQ_PERMISSION 0xF011
-#define SDLA_508_TSE_OFFSET 0xF012
-
-/* Event flags */
-#define SDLA_EVENT_STATUS 0x01
-#define SDLA_EVENT_DLCI_STATUS 0x02
-#define SDLA_EVENT_BAD_DLCI 0x04
-#define SDLA_EVENT_LINK_DOWN 0x40
-
-/* IRQ Trigger flags */
-#define SDLA_INTR_RX 0x01
-#define SDLA_INTR_TX 0x02
-#define SDLA_INTR_MODEM 0x04
-#define SDLA_INTR_COMPLETE 0x08
-#define SDLA_INTR_STATUS 0x10
-#define SDLA_INTR_TIMER 0x20
-
-/* DLCI status bits */
-#define SDLA_DLCI_DELETED 0x01
-#define SDLA_DLCI_ACTIVE 0x02
-#define SDLA_DLCI_WAITING 0x04
-#define SDLA_DLCI_NEW 0x08
-#define SDLA_DLCI_INCLUDED 0x40
-
-/* valid command codes */
-#define SDLA_INFORMATION_WRITE 0x01
-#define SDLA_INFORMATION_READ 0x02
-#define SDLA_ISSUE_IN_CHANNEL_SIGNAL 0x03
-#define SDLA_SET_DLCI_CONFIGURATION 0x10
-#define SDLA_READ_DLCI_CONFIGURATION 0x11
-#define SDLA_DISABLE_COMMUNICATIONS 0x12
-#define SDLA_ENABLE_COMMUNICATIONS 0x13
-#define SDLA_READ_DLC_STATUS 0x14
-#define SDLA_READ_DLC_STATISTICS 0x15
-#define SDLA_FLUSH_DLC_STATISTICS 0x16
-#define SDLA_LIST_ACTIVE_DLCI 0x17
-#define SDLA_FLUSH_INFORMATION_BUFFERS 0x18
-#define SDLA_ADD_DLCI 0x20
-#define SDLA_DELETE_DLCI 0x21
-#define SDLA_ACTIVATE_DLCI 0x22
-#define SDLA_DEACTIVATE_DLCI 0x23
-#define SDLA_READ_MODEM_STATUS 0x30
-#define SDLA_SET_MODEM_STATUS 0x31
-#define SDLA_READ_COMMS_ERR_STATS 0x32
-#define SDLA_FLUSH_COMMS_ERR_STATS 0x33
-#define SDLA_READ_CODE_VERSION 0x40
-#define SDLA_SET_IRQ_TRIGGER 0x50
-#define SDLA_GET_IRQ_TRIGGER 0x51
-
-/* In channel signal types */
-#define SDLA_ICS_LINK_VERIFY 0x02
-#define SDLA_ICS_STATUS_ENQ 0x03
-
-/* modem status flags */
-#define SDLA_MODEM_DTR_HIGH 0x01
-#define SDLA_MODEM_RTS_HIGH 0x02
-#define SDLA_MODEM_DCD_HIGH 0x08
-#define SDLA_MODEM_CTS_HIGH 0x20
-
-/* used for RET_MODEM interpretation */
-#define SDLA_MODEM_DCD_LOW 0x01
-#define SDLA_MODEM_CTS_LOW 0x02
-
-/* return codes */
-#define SDLA_RET_OK 0x00
-#define SDLA_RET_COMMUNICATIONS 0x01
-#define SDLA_RET_CHANNEL_INACTIVE 0x02
-#define SDLA_RET_DLCI_INACTIVE 0x03
-#define SDLA_RET_DLCI_CONFIG 0x04
-#define SDLA_RET_BUF_TOO_BIG 0x05
-#define SDLA_RET_NO_DATA 0x05
-#define SDLA_RET_BUF_OVERSIZE 0x06
-#define SDLA_RET_CIR_OVERFLOW 0x07
-#define SDLA_RET_NO_BUFS 0x08
-#define SDLA_RET_TIMEOUT 0x0A
-#define SDLA_RET_MODEM 0x10
-#define SDLA_RET_CHANNEL_OFF 0x11
-#define SDLA_RET_CHANNEL_ON 0x12
-#define SDLA_RET_DLCI_STATUS 0x13
-#define SDLA_RET_DLCI_UNKNOWN 0x14
-#define SDLA_RET_COMMAND_INVALID 0x1F
-
-/* Configuration flags */
-#define SDLA_DIRECT_RECV 0x0080
-#define SDLA_TX_NO_EXCEPT 0x0020
-#define SDLA_NO_ICF_MSGS 0x1000
-#define SDLA_TX50_RX50 0x0000
-#define SDLA_TX70_RX30 0x2000
-#define SDLA_TX30_RX70 0x4000
-
-/* IRQ selection flags */
-#define SDLA_IRQ_RECEIVE 0x01
-#define SDLA_IRQ_TRANSMIT 0x02
-#define SDLA_IRQ_MODEM_STAT 0x04
-#define SDLA_IRQ_COMMAND 0x08
-#define SDLA_IRQ_CHANNEL 0x10
-#define SDLA_IRQ_TIMER 0x20
-
-/* definitions for PC memory mapping */
-#define SDLA_8K_WINDOW 0x01
-#define SDLA_S502_SEG_A 0x10
-#define SDLA_S502_SEG_C 0x20
-#define SDLA_S502_SEG_D 0x00
-#define SDLA_S502_SEG_E 0x30
-#define SDLA_S507_SEG_A 0x00
-#define SDLA_S507_SEG_B 0x40
-#define SDLA_S507_SEG_C 0x80
-#define SDLA_S507_SEG_E 0xC0
-#define SDLA_S508_SEG_A 0x00
-#define SDLA_S508_SEG_C 0x10
-#define SDLA_S508_SEG_D 0x08
-#define SDLA_S508_SEG_E 0x18
-
-/* SDLA adapter port constants */
-#define SDLA_IO_EXTENTS 0x04
-
-#define SDLA_REG_CONTROL 0x00
-#define SDLA_REG_PC_WINDOW 0x01 /* offset for PC window select latch */
-#define SDLA_REG_Z80_WINDOW 0x02 /* offset for Z80 window select latch */
-#define SDLA_REG_Z80_CONTROL 0x03 /* offset for Z80 control latch */
-
-#define SDLA_S502_STS 0x00 /* status reg for 502, 502E, 507 */
-#define SDLA_S508_GNRL 0x00 /* general purp. reg for 508 */
-#define SDLA_S508_STS 0x01 /* status reg for 508 */
-#define SDLA_S508_IDR 0x02 /* ID reg for 508 */
-
-/* control register flags */
-#define SDLA_S502A_START 0x00 /* start the CPU */
-#define SDLA_S502A_INTREQ 0x02
-#define SDLA_S502A_INTEN 0x04
-#define SDLA_S502A_HALT 0x08 /* halt the CPU */
-#define SDLA_S502A_NMI 0x10 /* issue an NMI to the CPU */
-
-#define SDLA_S502E_CPUEN 0x01
-#define SDLA_S502E_ENABLE 0x02
-#define SDLA_S502E_INTACK 0x04
-
-#define SDLA_S507_ENABLE 0x01
-#define SDLA_S507_IRQ3 0x00
-#define SDLA_S507_IRQ4 0x20
-#define SDLA_S507_IRQ5 0x40
-#define SDLA_S507_IRQ7 0x60
-#define SDLA_S507_IRQ10 0x80
-#define SDLA_S507_IRQ11 0xA0
-#define SDLA_S507_IRQ12 0xC0
-#define SDLA_S507_IRQ15 0xE0
-
-#define SDLA_HALT 0x00
-#define SDLA_CPUEN 0x02
-#define SDLA_MEMEN 0x04
-#define SDLA_S507_EPROMWR 0x08
-#define SDLA_S507_EPROMCLK 0x10
-#define SDLA_S508_INTRQ 0x08
-#define SDLA_S508_INTEN 0x10
-
-struct sdla_cmd {
- char opp_flag;
- char cmd;
- short length;
- char retval;
- short dlci;
- char flags;
- short rxlost_int;
- long rxlost_app;
- char reserve[2];
- char data[SDLA_MAX_DATA]; /* transfer data buffer */
-} __attribute__((packed));
-
-struct intr_info {
- char flags;
- short txlen;
- char irq;
- char flags2;
- short timeout;
-} __attribute__((packed));
-
-/* found in the 508's control window at RXBUF_INFO */
-struct buf_info {
- unsigned short rse_num;
- unsigned long rse_base;
- unsigned long rse_next;
- unsigned long buf_base;
- unsigned short reserved;
- unsigned long buf_top;
-} __attribute__((packed));
-
-/* structure pointed to by rse_base in RXBUF_INFO struct */
-struct buf_entry {
- char opp_flag;
- short length;
- short dlci;
- char flags;
- short timestamp;
- short reserved[2];
- long buf_addr;
-} __attribute__((packed));
-
-#endif
void security_sk_classify_flow(struct sock *sk, struct flowi *fl);
void security_req_classify_flow(const struct request_sock *req, struct flowi *fl);
void security_sock_graft(struct sock*sk, struct socket *parent);
-int security_inet_conn_request(struct sock *sk,
+int security_inet_conn_request(const struct sock *sk,
struct sk_buff *skb, struct request_sock *req);
void security_inet_csk_clone(struct sock *newsk,
const struct request_sock *req);
{
}
-static inline int security_inet_conn_request(struct sock *sk,
+static inline int security_inet_conn_request(const struct sock *sk,
struct sk_buff *skb, struct request_sock *req)
{
return 0;
* @transport_header: Transport layer header
* @network_header: Network layer header
* @mac_header: Link layer header
+ * @kcov_handle: KCOV remote handle for remote coverage collection
* @tail: Tail pointer
* @end: End pointer
* @head: Head of buffer
__u16 network_header;
__u16 mac_header;
+#ifdef CONFIG_KCOV
+ u64 kcov_handle;
+#endif
+
/* private: */
__u32 headers_end[0];
/* public: */
#endif
#if IS_ENABLED(CONFIG_MPTCP)
SKB_EXT_MPTCP,
-#endif
-#if IS_ENABLED(CONFIG_KCOV)
- SKB_EXT_KCOV_HANDLE,
#endif
SKB_EXT_NUM, /* must be last */
};
#endif
}
-#ifdef CONFIG_KCOV
static inline void skb_set_kcov_handle(struct sk_buff *skb,
const u64 kcov_handle)
{
- /* Do not allocate skb extensions only to set kcov_handle to zero
- * (as it is zero by default). However, if the extensions are
- * already allocated, update kcov_handle anyway since
- * skb_set_kcov_handle can be called to zero a previously set
- * value.
- */
- if (skb_has_extensions(skb) || kcov_handle) {
- u64 *kcov_handle_ptr = skb_ext_add(skb, SKB_EXT_KCOV_HANDLE);
-
- if (kcov_handle_ptr)
- *kcov_handle_ptr = kcov_handle;
- }
+#ifdef CONFIG_KCOV
+ skb->kcov_handle = kcov_handle;
+#endif
}
static inline u64 skb_get_kcov_handle(struct sk_buff *skb)
{
- u64 *kcov_handle = skb_ext_find(skb, SKB_EXT_KCOV_HANDLE);
-
- return kcov_handle ? *kcov_handle : 0;
-}
+#ifdef CONFIG_KCOV
+ return skb->kcov_handle;
#else
-static inline void skb_set_kcov_handle(struct sk_buff *skb,
- const u64 kcov_handle) { }
-static inline u64 skb_get_kcov_handle(struct sk_buff *skb) { return 0; }
-#endif /* CONFIG_KCOV */
+ return 0;
+#endif
+}
#endif /* __KERNEL__ */
#endif /* _LINUX_SKBUFF_H */
--- /dev/null
+/* SPDX-License-Identifier: GPL-2.0-only
+ * Copyright (C) 2020 Marvell.
+ */
+
+#ifndef __SOC_OTX2_ASM_H
+#define __SOC_OTX2_ASM_H
+
+#if defined(CONFIG_ARM64)
+/*
+ * otx2_lmt_flush is used for LMT store operation.
+ * On octeontx2 platform CPT instruction enqueue and
+ * NIX packet send are only possible via LMTST
+ * operations and it uses LDEOR instruction targeting
+ * the coprocessor address.
+ */
+#define otx2_lmt_flush(ioaddr) \
+({ \
+ u64 result = 0; \
+ __asm__ volatile(".cpu generic+lse\n" \
+ "ldeor xzr, %x[rf], [%[rs]]" \
+ : [rf]"=r" (result) \
+ : [rs]"r" (ioaddr)); \
+ (result); \
+})
+#else
+#define otx2_lmt_flush(ioaddr) ({ 0; })
+#endif
+
+#endif /* __SOC_OTX2_ASM_H */
return __spi_alloc_controller(host, size, true);
}
+struct spi_controller *__devm_spi_alloc_controller(struct device *dev,
+ unsigned int size,
+ bool slave);
+
+static inline struct spi_controller *devm_spi_alloc_master(struct device *dev,
+ unsigned int size)
+{
+ return __devm_spi_alloc_controller(dev, size, false);
+}
+
+static inline struct spi_controller *devm_spi_alloc_slave(struct device *dev,
+ unsigned int size)
+{
+ if (!IS_ENABLED(CONFIG_SPI_SLAVE))
+ return NULL;
+
+ return __devm_spi_alloc_controller(dev, size, true);
+}
+
extern int spi_register_controller(struct spi_controller *ctlr);
extern int devm_spi_register_controller(struct device *dev,
struct spi_controller *ctlr);
#include <linux/dma-direction.h>
#include <linux/init.h>
#include <linux/types.h>
+#include <linux/limits.h>
struct device;
struct page;
struct netlink_ext_ack *newchain);
struct tcf_chain *tcf_action_set_ctrlact(struct tc_action *a, int action,
struct tcf_chain *newchain);
+
+#ifdef CONFIG_INET
+DECLARE_STATIC_KEY_FALSE(tcf_frag_xmit_count);
+#endif
+
+int tcf_dev_queue_xmit(struct sk_buff *skb, int (*xmit)(struct sk_buff *skb));
#endif /* CONFIG_NET_CLS_ACT */
static inline void tcf_action_stats_update(struct tc_action *a, u64 bytes,
struct rtnl_link_stats64 slave_stats;
};
+static inline struct slave *to_slave(struct kobject *kobj)
+{
+ return container_of(kobj, struct slave, kobj);
+}
+
struct bond_up_slave {
unsigned int count;
struct rcu_head rcu;
/* exported from bond_netlink.c */
extern struct rtnl_link_ops bond_link_ops;
+/* exported from bond_sysfs_slave.c */
+extern const struct sysfs_ops slave_sysfs_ops;
+
static inline netdev_tx_t bond_tx_drop(struct net_device *dev, struct sk_buff *skb)
{
atomic_long_inc(&dev->tx_dropped);
* Copyright (C) 2018-2020 Intel Corporation
*/
+#include <linux/ethtool.h>
#include <linux/netdevice.h>
#include <linux/debugfs.h>
#include <linux/list.h>
struct sock;
-#if defined(CONFIG_COMPAT)
-
#include <linux/compat.h>
struct compat_msghdr {
unsigned short rt_irtt; /* Initial RTT */
};
-#else /* defined(CONFIG_COMPAT) */
-/*
- * To avoid compiler warnings:
- */
-#define compat_msghdr msghdr
-#define compat_mmsghdr mmsghdr
-#endif /* defined(CONFIG_COMPAT) */
-
int __get_compat_msghdr(struct msghdr *kmsg, struct compat_msghdr __user *umsg,
struct sockaddr __user **save_addr, compat_uptr_t *ptr,
compat_size_t *len);
#include <net/flow_offload.h>
#include <uapi/linux/devlink.h>
#include <linux/xarray.h>
+#include <linux/firmware.h>
#define DEVLINK_RELOAD_STATS_ARRAY_SIZE \
(__DEVLINK_RELOAD_LIMIT_MAX * __DEVLINK_RELOAD_ACTION_MAX)
/**
* struct devlink_flash_update_params - Flash Update parameters
- * @file_name: the name of the flash firmware file to update from
+ * @fw: pointer to the firmware data to update from
* @component: the flash component to update
*
- * With the exception of file_name, drivers must opt-in to parameters by
+ * With the exception of fw, drivers must opt-in to parameters by
* setting the appropriate bit in the supported_flash_update_params field in
* their devlink_ops structure.
*/
struct devlink_flash_update_params {
- const char *file_name;
+ const struct firmware *fw;
const char *component;
u32 overwrite_mask;
};
DEVLINK_TRAP_GENERIC_ID_DCCP_PARSING,
DEVLINK_TRAP_GENERIC_ID_GTP_PARSING,
DEVLINK_TRAP_GENERIC_ID_ESP_PARSING,
+ DEVLINK_TRAP_GENERIC_ID_BLACKHOLE_NEXTHOP,
/* Add new generic trap IDs above */
__DEVLINK_TRAP_GENERIC_ID_MAX,
"gtp_parsing"
#define DEVLINK_TRAP_GENERIC_NAME_ESP_PARSING \
"esp_parsing"
-
+#define DEVLINK_TRAP_GENERIC_NAME_BLACKHOLE_NEXTHOP \
+ "blackhole_nexthop"
#define DEVLINK_TRAP_GROUP_GENERIC_NAME_L2_DROPS \
"l2_drops"
enum devlink_reload_limit limit,
u32 actions_performed);
-void devlink_flash_update_begin_notify(struct devlink *devlink);
-void devlink_flash_update_end_notify(struct devlink *devlink);
void devlink_flash_update_status_notify(struct devlink *devlink,
const char *status_msg,
const char *component,
if ((iph->tos & INET_ECN_MASK) != INET_ECN_ECT_0)
return 0;
- check += (__force u16)htons(0x100);
+ check += (__force u16)htons(0x1);
iph->check = (__force __sum16)(check + (check>=0xFFFF));
iph->tos ^= INET_ECN_MASK;
unsigned long high_limit);
int inet_hashinfo2_init_mod(struct inet_hashinfo *h);
-bool inet_ehash_insert(struct sock *sk, struct sock *osk);
-bool inet_ehash_nolisten(struct sock *sk, struct sock *osk);
+bool inet_ehash_insert(struct sock *sk, struct sock *osk, bool *found_dup_sk);
+bool inet_ehash_nolisten(struct sock *sk, struct sock *osk,
+ bool *found_dup_sk);
int __inet_hash(struct sock *sk, struct sock *osk);
int inet_hash(struct sock *sk);
void inet_unhash(struct sock *sk);
const void *from, int len,
__be16 flags)
{
- memcpy(ip_tunnel_info_opts(info), from, len);
info->options_len = len;
- info->key.tun_flags |= flags;
+ if (len > 0) {
+ memcpy(ip_tunnel_info_opts(info), from, len);
+ info->key.tun_flags |= flags;
+ }
}
static inline struct ip_tunnel_info *lwt_tun_info(struct lwtunnel_state *lwtstate)
__be16 flags)
{
info->options_len = 0;
- info->key.tun_flags |= flags;
}
#endif /* CONFIG_INET */
rcu_read_unlock();
inet_frag_put(&fq->q);
}
+
+/* Check if the upper layer header is truncated in the first fragment. */
+static inline bool
+ipv6frag_thdr_truncated(struct sk_buff *skb, int start, u8 *nexthdrp)
+{
+ u8 nexthdr = *nexthdrp;
+ __be16 frag_off;
+ int offset;
+
+ offset = ipv6_skip_exthdr(skb, start, &nexthdr, &frag_off);
+ if (offset < 0 || (frag_off & htons(IP6_OFFSET)))
+ return false;
+ switch (nexthdr) {
+ case NEXTHDR_TCP:
+ offset += sizeof(struct tcphdr);
+ break;
+ case NEXTHDR_UDP:
+ offset += sizeof(struct udphdr);
+ break;
+ case NEXTHDR_ICMP:
+ offset += sizeof(struct icmp6hdr);
+ break;
+ default:
+ offset += 1;
+ }
+ if (offset > skb->len)
+ return true;
+ return false;
+}
+
#endif
#endif
struct mptcp_out_options *opts);
void mptcp_incoming_options(struct sock *sk, struct sk_buff *skb);
-void mptcp_write_options(__be32 *ptr, struct mptcp_out_options *opts);
+void mptcp_write_options(__be32 *ptr, const struct tcp_sock *tp,
+ struct mptcp_out_options *opts);
/* move the skb extension owership, with the assumption that 'to' is
* newly allocated
int (*pconstructor)(struct pneigh_entry *);
void (*pdestructor)(struct pneigh_entry *);
void (*proxy_redo)(struct sk_buff *skb);
+ int (*is_multicast)(const void *pkey);
bool (*allow_add)(const struct net_device *dev,
struct netlink_ext_ack *extack);
char *id;
{
return &net->ct.nf_ct_proto.icmpv6;
}
+
+/* Caller must check nf_ct_protonum(ct) is IPPROTO_TCP before calling. */
+static inline void nf_ct_set_tcp_be_liberal(struct nf_conn *ct)
+{
+ ct->proto.tcp.seen[0].flags |= IP_CT_TCP_FLAG_BE_LIBERAL;
+ ct->proto.tcp.seen[1].flags |= IP_CT_TCP_FLAG_BE_LIBERAL;
+}
+
+/* Caller must check nf_ct_protonum(ct) is IPPROTO_TCP before calling. */
+static inline bool nf_conntrack_tcp_established(const struct nf_conn *ct)
+{
+ return ct->proto.tcp.state == TCP_CONNTRACK_ESTABLISHED &&
+ test_bit(IPS_ASSURED_BIT, &ct->status);
+}
#endif
#ifdef CONFIG_NF_CT_PROTO_DCCP
struct nft_flow_key {
struct flow_dissector_key_basic basic;
+ struct flow_dissector_key_control control;
union {
struct flow_dissector_key_ipv4_addrs ipv4;
struct flow_dissector_key_ipv6_addrs ipv6;
#define NFT_OFFLOAD_F_ACTION (1 << 0)
+void nft_flow_rule_set_addr_type(struct nft_flow_rule *flow,
+ enum flow_dissector_key_id addr_type);
+
struct nft_rule;
struct nft_flow_rule *nft_flow_rule_create(struct net *net, const struct nft_rule *rule);
void nft_flow_rule_destroy(struct nft_flow_rule *flow);
offsetof(struct nft_flow_key, __base.__field); \
(__reg)->len = __len; \
(__reg)->key = __key; \
+
+#define NFT_OFFLOAD_MATCH_EXACT(__key, __base, __field, __len, __reg) \
+ NFT_OFFLOAD_MATCH(__key, __base, __field, __len, __reg) \
memset(&(__reg)->mask, 0xff, (__reg)->len);
int nft_chain_offload_priority(struct nft_base_chain *basechain);
* Attribute Misc:
* nla_memcpy(dest, nla, count) copy attribute into memory
* nla_memcmp(nla, data, size) compare attribute with memory area
- * nla_strlcpy(dst, nla, size) copy attribute to a sized string
+ * nla_strscpy(dst, nla, size) copy attribute to a sized string
* nla_strcmp(nla, str) compare attribute with string
*
* Attribute Parsing:
struct netlink_ext_ack *extack);
int nla_policy_len(const struct nla_policy *, int);
struct nlattr *nla_find(const struct nlattr *head, int len, int attrtype);
-size_t nla_strlcpy(char *dst, const struct nlattr *nla, size_t dstsize);
+ssize_t nla_strscpy(char *dst, const struct nlattr *nla, size_t dstsize);
char *nla_strdup(const struct nlattr *nla, gfp_t flags);
int nla_memcpy(void *dest, const struct nlattr *src, int count);
int nla_memcmp(const struct nlattr *nla, const void *data, size_t size);
struct tcf_chain *tcf_get_next_chain(struct tcf_block *block,
struct tcf_chain *chain);
struct tcf_proto *tcf_get_next_proto(struct tcf_chain *chain,
- struct tcf_proto *tp, bool rtnl_held);
+ struct tcf_proto *tp);
void tcf_block_netif_keep_dst(struct tcf_block *block);
int tcf_block_get(struct tcf_block **p_block,
struct tcf_proto __rcu **p_filter_chain, struct Qdisc *q,
char indev[IFNAMSIZ];
struct net_device *dev;
- if (nla_strlcpy(indev, indev_tlv, IFNAMSIZ) >= IFNAMSIZ) {
+ if (nla_strscpy(indev, indev_tlv, IFNAMSIZ) < 0) {
NL_SET_ERR_MSG_ATTR(extack, indev_tlv,
"Interface name too long");
return -EINVAL;
void mini_qdisc_pair_block_init(struct mini_Qdisc_pair *miniqp,
struct tcf_block *block);
-static inline int skb_tc_reinsert(struct sk_buff *skb, struct tcf_result *res)
-{
- return res->ingress ? netif_receive_skb(skb) : dev_queue_xmit(skb);
-}
+int sch_frag_xmit_hook(struct sk_buff *skb, int (*xmit)(struct sk_buff *skb));
#endif
#include <linux/rculist_nulls.h>
#include <linux/poll.h>
#include <linux/sockptr.h>
-
+#include <linux/indirect_call_wrapper.h>
#include <linux/atomic.h>
#include <linux/refcount.h>
#include <net/dst.h>
#define sk_refcnt_debug_release(sk) do { } while (0)
#endif /* SOCK_REFCNT_DEBUG */
+INDIRECT_CALLABLE_DECLARE(bool tcp_stream_memory_free(const struct sock *sk, int wake));
+
static inline bool __sk_stream_memory_free(const struct sock *sk, int wake)
{
if (READ_ONCE(sk->sk_wmem_queued) >= READ_ONCE(sk->sk_sndbuf))
return false;
+#ifdef CONFIG_INET
+ return sk->sk_prot->stream_memory_free ?
+ INDIRECT_CALL_1(sk->sk_prot->stream_memory_free,
+ tcp_stream_memory_free,
+ sk, wake) : true;
+#else
return sk->sk_prot->stream_memory_free ?
sk->sk_prot->stream_memory_free(sk, wake) : true;
+#endif
}
static inline bool sk_stream_memory_free(const struct sock *sk)
lock_sock_nested(sk, 0);
}
+void __lock_sock(struct sock *sk);
void __release_sock(struct sock *sk);
void release_sock(struct sock *sk);
SINGLE_DEPTH_NESTING)
#define bh_unlock_sock(__sk) spin_unlock(&((__sk)->sk_lock.slock))
-bool lock_sock_fast(struct sock *sk);
+bool lock_sock_fast(struct sock *sk) __acquires(&sk->sk_lock.slock);
+
/**
* unlock_sock_fast - complement of lock_sock_fast
* @sk: socket
* If slow mode is on, we call regular release_sock()
*/
static inline void unlock_sock_fast(struct sock *sk, bool slow)
+ __releases(&sk->sk_lock.slock)
{
- if (slow)
+ if (slow) {
release_sock(sk);
- else
+ __release(&sk->sk_lock.slock);
+ } else {
spin_unlock_bh(&sk->sk_lock.slock);
+ }
}
/* Used by processes to "lock" a socket state, so that
SWITCHDEV_ATTR_ID_PORT_MROUTER,
SWITCHDEV_ATTR_ID_BRIDGE_AGEING_TIME,
SWITCHDEV_ATTR_ID_BRIDGE_VLAN_FILTERING,
+ SWITCHDEV_ATTR_ID_BRIDGE_VLAN_PROTOCOL,
SWITCHDEV_ATTR_ID_BRIDGE_MC_DISABLED,
SWITCHDEV_ATTR_ID_BRIDGE_MROUTER,
#if IS_ENABLED(CONFIG_BRIDGE_MRP)
bool mrouter; /* PORT_MROUTER */
clock_t ageing_time; /* BRIDGE_AGEING_TIME */
bool vlan_filtering; /* BRIDGE_VLAN_FILTERING */
+ u16 vlan_protocol; /* BRIDGE_VLAN_PROTOCOL */
bool mc_disabled; /* MC_DISABLED */
#if IS_ENABLED(CONFIG_BRIDGE_MRP)
u8 mrp_port_state; /* MRP_PORT_STATE */
int tcp_v4_early_demux(struct sk_buff *skb);
int tcp_v4_rcv(struct sk_buff *skb);
+void tcp_remove_empty_skb(struct sock *sk, struct sk_buff *skb);
int tcp_v4_tw_remember_stamp(struct inet_timewait_sock *tw);
int tcp_sendmsg(struct sock *sk, struct msghdr *msg, size_t size);
int tcp_sendmsg_locked(struct sock *sk, struct msghdr *msg, size_t size);
int flags);
int tcp_sendpage_locked(struct sock *sk, struct page *page, int offset,
size_t size, int flags);
+struct sk_buff *tcp_build_frag(struct sock *sk, int size_goal, int flags,
+ struct page *page, int offset, size_t *size);
ssize_t do_tcp_sendpages(struct sock *sk, struct page *page, int offset,
size_t size, int flags);
int tcp_send_mss(struct sock *sk, int *size_goal, int flags);
void tcp_init_metrics(struct sock *sk);
void tcp_metrics_init(void);
bool tcp_peer_is_proven(struct request_sock *req, struct dst_entry *dst);
+void __tcp_close(struct sock *sk, long timeout);
void tcp_close(struct sock *sk, long timeout);
void tcp_init_sock(struct sock *sk);
void tcp_init_transfer(struct sock *sk, int bpf_op, struct sk_buff *skb);
return tp->notsent_lowat ?: net->ipv4.sysctl_tcp_notsent_lowat;
}
-/* @wake is one when sk_stream_write_space() calls us.
- * This sends EPOLLOUT only if notsent_bytes is half the limit.
- * This mimics the strategy used in sock_def_write_space().
- */
-static inline bool tcp_stream_memory_free(const struct sock *sk, int wake)
-{
- const struct tcp_sock *tp = tcp_sk(sk);
- u32 notsent_bytes = READ_ONCE(tp->write_seq) -
- READ_ONCE(tp->snd_nxt);
-
- return (notsent_bytes << wake) < tcp_notsent_lowat(tp);
-}
+bool tcp_stream_memory_free(const struct sock *sk, int wake);
#ifdef CONFIG_PROC_FS
int tcp4_proc_init(void);
const struct sock *sk,
const struct sk_buff *skb);
#endif
- void (*init_req)(struct request_sock *req,
- const struct sock *sk_listener,
- struct sk_buff *skb);
#ifdef CONFIG_SYN_COOKIES
__u32 (*cookie_init_seq)(const struct sk_buff *skb,
__u16 *mss);
#endif
- struct dst_entry *(*route_req)(const struct sock *sk, struct flowi *fl,
- const struct request_sock *req);
+ struct dst_entry *(*route_req)(const struct sock *sk,
+ struct sk_buff *skb,
+ struct flowi *fl,
+ struct request_sock *req);
u32 (*init_seq)(const struct sk_buff *skb);
u32 (*init_ts_off)(const struct net *net, const struct sk_buff *skb);
int (*send_synack)(const struct sock *sk, struct dst_entry *dst,
* to be atomic.
*/
TLS_TX_SYNC_SCHED = 1,
+ /* tls_dev_del was called for the RX side, device state was released,
+ * but tls_ctx->netdev might still be kept, because TX-side driver
+ * resources might not be released yet. Used to prevent the second
+ * tls_dev_del call in tls_device_down if it happens simultaneously.
+ */
+ TLS_RX_DEV_CLOSED = 2,
};
struct cipher_context {
union {
struct tls12_crypto_info_aes_gcm_128 aes_gcm_128;
struct tls12_crypto_info_aes_gcm_256 aes_gcm_256;
+ struct tls12_crypto_info_chacha20_poly1305 chacha20_poly1305;
};
};
#define TLS_DEVICE_RESYNC_ASYNC_LOGMAX 13
struct tls_offload_resync_async {
atomic64_t req;
- u32 loglen;
+ u16 loglen;
+ u16 rcd_delta;
u32 log[TLS_DEVICE_RESYNC_ASYNC_LOGMAX];
};
return (i == -1);
}
+static inline void tls_bigint_subtract(unsigned char *seq, int n)
+{
+ u64 rcd_sn;
+ __be64 *p;
+
+ BUILD_BUG_ON(TLS_MAX_REC_SEQ_SIZE != 8);
+
+ p = (__be64 *)seq;
+ rcd_sn = be64_to_cpu(*p);
+ *p = cpu_to_be64(rcd_sn - n);
+}
+
static inline struct tls_context *tls_get_ctx(const struct sock *sk)
{
struct inet_connection_sock *icsk = inet_csk(sk);
if (tls_bigint_increment(ctx->rec_seq, prot->rec_seq_size))
tls_err_abort(sk, EBADMSG);
- if (prot->version != TLS_1_3_VERSION)
- tls_bigint_increment(ctx->iv + TLS_CIPHER_AES_GCM_128_SALT_SIZE,
+ if (prot->version != TLS_1_3_VERSION &&
+ prot->cipher_type != TLS_CIPHER_CHACHA20_POLY1305)
+ tls_bigint_increment(ctx->iv + prot->salt_size,
prot->iv_size);
}
static inline void tls_fill_prepend(struct tls_context *ctx,
char *buf,
size_t plaintext_len,
- unsigned char record_type,
- int version)
+ unsigned char record_type)
{
struct tls_prot_info *prot = &ctx->prot_info;
size_t pkt_len, iv_size = prot->iv_size;
pkt_len = plaintext_len + prot->tag_size;
- if (version != TLS_1_3_VERSION) {
+ if (prot->version != TLS_1_3_VERSION &&
+ prot->cipher_type != TLS_CIPHER_CHACHA20_POLY1305) {
pkt_len += iv_size;
memcpy(buf + TLS_NONCE_OFFSET,
- ctx->tx.iv + TLS_CIPHER_AES_GCM_128_SALT_SIZE, iv_size);
+ ctx->tx.iv + prot->salt_size, iv_size);
}
/* we cover nonce explicit here as well, so buf should be of
* size KTLS_DTLS_HEADER_SIZE + KTLS_DTLS_NONCE_EXPLICIT_SIZE
*/
- buf[0] = version == TLS_1_3_VERSION ?
+ buf[0] = prot->version == TLS_1_3_VERSION ?
TLS_RECORD_TYPE_DATA : record_type;
/* Note that VERSION must be TLS_1_2 for both TLS1.2 and TLS1.3 */
buf[1] = TLS_1_2_VERSION_MINOR;
static inline void tls_make_aad(char *buf,
size_t size,
char *record_sequence,
- int record_sequence_size,
unsigned char record_type,
- int version)
+ struct tls_prot_info *prot)
{
- if (version != TLS_1_3_VERSION) {
- memcpy(buf, record_sequence, record_sequence_size);
+ if (prot->version != TLS_1_3_VERSION) {
+ memcpy(buf, record_sequence, prot->rec_seq_size);
buf += 8;
} else {
- size += TLS_CIPHER_AES_GCM_128_TAG_SIZE;
+ size += prot->tag_size;
}
- buf[0] = version == TLS_1_3_VERSION ?
+ buf[0] = prot->version == TLS_1_3_VERSION ?
TLS_RECORD_TYPE_DATA : record_type;
buf[1] = TLS_1_2_VERSION_MAJOR;
buf[2] = TLS_1_2_VERSION_MINOR;
buf[4] = size & 0xFF;
}
-static inline void xor_iv_with_seq(int version, char *iv, char *seq)
+static inline void xor_iv_with_seq(struct tls_prot_info *prot, char *iv, char *seq)
{
int i;
- if (version == TLS_1_3_VERSION) {
+ if (prot->version == TLS_1_3_VERSION ||
+ prot->cipher_type == TLS_CIPHER_CHACHA20_POLY1305) {
for (i = 0; i < 8; i++)
iv[i + 4] ^= seq[i];
}
atomic64_set(&rx_ctx->resync_async->req, ((u64)ntohl(seq) << 32) |
((u64)len << 16) | RESYNC_REQ | RESYNC_REQ_ASYNC);
rx_ctx->resync_async->loglen = 0;
+ rx_ctx->resync_async->rcd_delta = 0;
}
static inline void
struct page **pgs;
int id;
struct list_head xsk_dma_list;
+ struct work_struct work;
};
struct xsk_map {
#ifndef IB_ADDR_H
#define IB_ADDR_H
+#include <linux/ethtool.h>
#include <linux/in.h>
#include <linux/in6.h>
#include <linux/if_arp.h>
#ifndef IB_VERBS_H
#define IB_VERBS_H
+#include <linux/ethtool.h>
#include <linux/types.h>
#include <linux/device.h>
#include <linux/dma-mapping.h>
void *dd_data; /* driver/transport data */
};
+/* invalid scsi_task pointer */
+#define INVALID_SCSI_TASK (struct iscsi_task *)-1l
+
static inline int iscsi_task_has_unsol_data(struct iscsi_task *task)
{
return task->unsol_r2t.data_length > task->unsol_r2t.sent;
--- /dev/null
+/* SPDX-License-Identifier: GPL-2.0-only */
+/*
+ * linux/sound/rt1015.h -- Platform data for RT1015
+ *
+ * Copyright 2020 Realtek Microelectronics
+ */
+
+#ifndef __LINUX_SND_RT1015_H
+#define __LINUX_SND_RT1015_H
+
+struct rt1015_platform_data {
+ unsigned int power_up_delay_ms;
+};
+
+#endif
TP_fast_assign(
__entry->task_id = task->tk_pid;
- __entry->client_id = task->tk_client->cl_clid;
+ __entry->client_id = task->tk_client ?
+ task->tk_client->cl_clid : -1;
__entry->head_base = xdr->head[0].iov_base;
__entry->head_len = xdr->head[0].iov_len;
__entry->tail_base = xdr->tail[0].iov_base;
),
TP_fast_assign(
- strncpy(__entry->name, bdi_dev_name(inode_to_bdi(inode)), 32);
+ strscpy_pad(__entry->name, bdi_dev_name(inode_to_bdi(inode)), 32);
__entry->ino = inode->i_ino;
__entry->cgroup_ino = __trace_wbc_assign_cgroup(wbc);
__entry->history = history;
),
TP_fast_assign(
- strncpy(__entry->name, bdi_dev_name(old_wb->bdi), 32);
+ strscpy_pad(__entry->name, bdi_dev_name(old_wb->bdi), 32);
__entry->ino = inode->i_ino;
__entry->old_cgroup_ino = __trace_wb_assign_cgroup(old_wb);
__entry->new_cgroup_ino = __trace_wb_assign_cgroup(new_wb);
struct address_space *mapping = page_mapping(page);
struct inode *inode = mapping ? mapping->host : NULL;
- strncpy(__entry->name, bdi_dev_name(wb->bdi), 32);
+ strscpy_pad(__entry->name, bdi_dev_name(wb->bdi), 32);
__entry->bdi_id = wb->bdi->id;
__entry->ino = inode ? inode->i_ino : 0;
__entry->memcg_id = wb->memcg_css->id;
),
TP_fast_assign(
- strncpy(__entry->name, bdi_dev_name(wb->bdi), 32);
+ strscpy_pad(__entry->name, bdi_dev_name(wb->bdi), 32);
__entry->cgroup_ino = __trace_wb_assign_cgroup(wb);
__entry->frn_bdi_id = frn_bdi_id;
__entry->frn_memcg_id = frn_memcg_id;
/* CAN payload length and DLC definitions according to ISO 11898-1 */
#define CAN_MAX_DLC 8
+#define CAN_MAX_RAW_DLC 15
#define CAN_MAX_DLEN 8
/* CAN FD payload length and DLC definitions according to ISO 11898-7 */
#define CANFD_MAX_DLEN 64
/**
- * struct can_frame - basic CAN frame structure
- * @can_id: CAN ID of the frame and CAN_*_FLAG flags, see canid_t definition
- * @can_dlc: frame payload length in byte (0 .. 8) aka data length code
- * N.B. the DLC field from ISO 11898-1 Chapter 8.4.2.3 has a 1:1
- * mapping of the 'data length code' to the real payload length
- * @__pad: padding
- * @__res0: reserved / padding
- * @__res1: reserved / padding
- * @data: CAN frame payload (up to 8 byte)
+ * struct can_frame - Classical CAN frame structure (aka CAN 2.0B)
+ * @can_id: CAN ID of the frame and CAN_*_FLAG flags, see canid_t definition
+ * @len: CAN frame payload length in byte (0 .. 8)
+ * @can_dlc: deprecated name for CAN frame payload length in byte (0 .. 8)
+ * @__pad: padding
+ * @__res0: reserved / padding
+ * @len8_dlc: optional DLC value (9 .. 15) at 8 byte payload length
+ * len8_dlc contains values from 9 .. 15 when the payload length is
+ * 8 bytes but the DLC value (see ISO 11898-1) is greater then 8.
+ * CAN_CTRLMODE_CC_LEN8_DLC flag has to be enabled in CAN driver.
+ * @data: CAN frame payload (up to 8 byte)
*/
struct can_frame {
canid_t can_id; /* 32 bit CAN_ID + EFF/RTR/ERR flags */
- __u8 can_dlc; /* frame payload length in byte (0 .. CAN_MAX_DLEN) */
- __u8 __pad; /* padding */
- __u8 __res0; /* reserved / padding */
- __u8 __res1; /* reserved / padding */
- __u8 data[CAN_MAX_DLEN] __attribute__((aligned(8)));
+ union {
+ /* CAN frame payload length in byte (0 .. CAN_MAX_DLEN)
+ * was previously named can_dlc so we need to carry that
+ * name for legacy support
+ */
+ __u8 len;
+ __u8 can_dlc; /* deprecated */
+ };
+ __u8 __pad; /* padding */
+ __u8 __res0; /* reserved / padding */
+ __u8 len8_dlc; /* optional DLC for 8 byte payload length (9 .. 15) */
+ __u8 data[CAN_MAX_DLEN] __attribute__((aligned(8)));
};
/*
/* CAN frame elements that are affected by curr. 3 CAN frame modifications */
#define CGW_MOD_ID 0x01
-#define CGW_MOD_DLC 0x02 /* contains the data length in bytes */
-#define CGW_MOD_LEN CGW_MOD_DLC /* CAN FD length representation */
+#define CGW_MOD_DLC 0x02 /* Classical CAN data length code */
+#define CGW_MOD_LEN CGW_MOD_DLC /* CAN FD (plain) data length */
#define CGW_MOD_DATA 0x04
#define CGW_MOD_FLAGS 0x08 /* CAN FD flags */
#define CAN_CTRLMODE_FD 0x20 /* CAN FD mode */
#define CAN_CTRLMODE_PRESUME_ACK 0x40 /* Ignore missing CAN ACKs */
#define CAN_CTRLMODE_FD_NON_ISO 0x80 /* CAN FD in non-ISO mode */
+#define CAN_CTRLMODE_CC_LEN8_DLC 0x100 /* Classic CAN DLC option */
/*
* CAN device statistics
DEVLINK_ATTR_RELOAD_STATS_LIMIT, /* u8 */
DEVLINK_ATTR_RELOAD_STATS_VALUE, /* u32 */
DEVLINK_ATTR_REMOTE_RELOAD_STATS, /* nested */
+ DEVLINK_ATTR_RELOAD_ACTION_INFO, /* nested */
+ DEVLINK_ATTR_RELOAD_ACTION_STATS, /* nested */
/* add new attributes above here, update the policy in devlink.c */
* struct gpiochip_info - Information about a certain GPIO chip
* @name: the Linux kernel name of this GPIO chip
* @label: a functional name for this GPIO chip, such as a product
- * number, may be empty
+ * number, may be empty (i.e. label[0] == '\0')
* @lines: number of GPIO lines on this chip
*/
struct gpiochip_info {
* identifying which field of the attribute union is in use.
* @GPIO_V2_LINE_ATTR_ID_FLAGS: flags field is in use
* @GPIO_V2_LINE_ATTR_ID_OUTPUT_VALUES: values field is in use
- * @GPIO_V2_LINE_ATTR_ID_DEBOUNCE: debounce_period_us is in use
+ * @GPIO_V2_LINE_ATTR_ID_DEBOUNCE: debounce_period_us field is in use
*/
enum gpio_v2_line_attr_id {
GPIO_V2_LINE_ATTR_ID_FLAGS = 1,
* struct gpio_v2_line_attribute - a configurable attribute of a line
* @id: attribute identifier with value from &enum gpio_v2_line_attr_id
* @padding: reserved for future use and must be zero filled
- * @flags: if id is GPIO_V2_LINE_ATTR_ID_FLAGS, the flags for the GPIO
- * line, with values from enum gpio_v2_line_flag, such as
- * GPIO_V2_LINE_FLAG_ACTIVE_LOW, GPIO_V2_LINE_FLAG_OUTPUT etc, OR:ed
+ * @flags: if id is %GPIO_V2_LINE_ATTR_ID_FLAGS, the flags for the GPIO
+ * line, with values from &enum gpio_v2_line_flag, such as
+ * %GPIO_V2_LINE_FLAG_ACTIVE_LOW, %GPIO_V2_LINE_FLAG_OUTPUT etc, added
* together. This overrides the default flags contained in the &struct
* gpio_v2_line_config for the associated line.
- * @values: if id is GPIO_V2_LINE_ATTR_ID_OUTPUT_VALUES, a bitmap
+ * @values: if id is %GPIO_V2_LINE_ATTR_ID_OUTPUT_VALUES, a bitmap
* containing the values to which the lines will be set, with each bit
* number corresponding to the index into &struct
* gpio_v2_line_request.offsets.
- * @debounce_period_us: if id is GPIO_V2_LINE_ATTR_ID_DEBOUNCE, the desired
- * debounce period, in microseconds
+ * @debounce_period_us: if id is %GPIO_V2_LINE_ATTR_ID_DEBOUNCE, the
+ * desired debounce period, in microseconds
*/
struct gpio_v2_line_attribute {
__u32 id;
/**
* struct gpio_v2_line_config - Configuration for GPIO lines
- * @flags: flags for the GPIO lines, with values from enum
- * gpio_v2_line_flag, such as GPIO_V2_LINE_FLAG_ACTIVE_LOW,
- * GPIO_V2_LINE_FLAG_OUTPUT etc, OR:ed together. This is the default for
+ * @flags: flags for the GPIO lines, with values from &enum
+ * gpio_v2_line_flag, such as %GPIO_V2_LINE_FLAG_ACTIVE_LOW,
+ * %GPIO_V2_LINE_FLAG_OUTPUT etc, added together. This is the default for
* all requested lines but may be overridden for particular lines using
- * attrs.
- * @num_attrs: the number of attributes in attrs
+ * @attrs.
+ * @num_attrs: the number of attributes in @attrs
* @padding: reserved for future use and must be zero filled
* @attrs: the configuration attributes associated with the requested
* lines. Any attribute should only be associated with a particular line
* "my-bitbanged-relay"
* @config: requested configuration for the lines.
* @num_lines: number of lines requested in this request, i.e. the number
- * of valid fields in the GPIO_V2_LINES_MAX sized arrays, set to 1 to
+ * of valid fields in the %GPIO_V2_LINES_MAX sized arrays, set to 1 to
* request a single line
* @event_buffer_size: a suggested minimum number of line events that the
* kernel should buffer. This is only relevant if edge detection is
* enabled in the configuration. Note that this is only a suggested value
* and the kernel may allocate a larger buffer or cap the size of the
* buffer. If this field is zero then the buffer size defaults to a minimum
- * of num_lines*16.
+ * of @num_lines * 16.
* @padding: reserved for future use and must be zero filled
* @fd: if successful this field will contain a valid anonymous file handle
- * after a GPIO_GET_LINE_IOCTL operation, zero or negative value means
+ * after a %GPIO_GET_LINE_IOCTL operation, zero or negative value means
* error
*/
struct gpio_v2_line_request {
* struct gpio_v2_line_info - Information about a certain GPIO line
* @name: the name of this GPIO line, such as the output pin of the line on
* the chip, a rail or a pin header name on a board, as specified by the
- * GPIO chip, may be empty
+ * GPIO chip, may be empty (i.e. name[0] == '\0')
* @consumer: a functional name for the consumer of this GPIO line as set
* by whatever is using it, will be empty if there is no current user but
* may also be empty if the consumer doesn't set this up
- * @flags: flags for the GPIO line, such as GPIO_V2_LINE_FLAG_ACTIVE_LOW,
- * GPIO_V2_LINE_FLAG_OUTPUT etc, OR:ed together
* @offset: the local offset on this GPIO chip, fill this in when
* requesting the line information from the kernel
- * @num_attrs: the number of attributes in attrs
+ * @num_attrs: the number of attributes in @attrs
+ * @flags: flags for the GPIO lines, with values from &enum
+ * gpio_v2_line_flag, such as %GPIO_V2_LINE_FLAG_ACTIVE_LOW,
+ * %GPIO_V2_LINE_FLAG_OUTPUT etc, added together.
* @attrs: the configuration attributes associated with the line
* @padding: reserved for future use
*/
* of a GPIO line
* @info: updated line information
* @timestamp_ns: estimate of time of status change occurrence, in nanoseconds
- * @event_type: the type of change with a value from enum
+ * @event_type: the type of change with a value from &enum
* gpio_v2_line_changed_type
* @padding: reserved for future use
*/
/**
* struct gpio_v2_line_event - The actual event being pushed to userspace
* @timestamp_ns: best estimate of time of event occurrence, in nanoseconds.
- * The timestamp_ns is read from CLOCK_MONOTONIC and is intended to allow the
- * accurate measurement of the time between events. It does not provide
+ * The @timestamp_ns is read from %CLOCK_MONOTONIC and is intended to allow
+ * the accurate measurement of the time between events. It does not provide
* the wall-clock time.
- * @id: event identifier with value from enum gpio_v2_line_event_id
+ * @id: event identifier with value from &enum gpio_v2_line_event_id
* @offset: the offset of the line that triggered the event
* @seqno: the sequence number for this event in the sequence of events for
* all the lines in this line request
};
/*
- * ABI v1
+ * ABI v1
*
* This version of the ABI is deprecated.
* Use the latest version of the ABI, defined above, instead.
* @flags: various flags for this line
* @name: the name of this GPIO line, such as the output pin of the line on the
* chip, a rail or a pin header name on a board, as specified by the gpio
- * chip, may be empty
+ * chip, may be empty (i.e. name[0] == '\0')
* @consumer: a functional name for the consumer of this GPIO line as set by
* whatever is using it, will be empty if there is no current user but may
* also be empty if the consumer doesn't set this up
*
- * This struct is part of ABI v1 and is deprecated.
- * Use struct gpio_v2_line_info instead.
+ * Note: This struct is part of ABI v1 and is deprecated.
+ * Use &struct gpio_v2_line_info instead.
*/
struct gpioline_info {
__u32 line_offset;
* of a GPIO line
* @info: updated line information
* @timestamp: estimate of time of status change occurrence, in nanoseconds
- * @event_type: one of GPIOLINE_CHANGED_REQUESTED, GPIOLINE_CHANGED_RELEASED
- * and GPIOLINE_CHANGED_CONFIG
+ * @event_type: one of %GPIOLINE_CHANGED_REQUESTED,
+ * %GPIOLINE_CHANGED_RELEASED and %GPIOLINE_CHANGED_CONFIG
+ * @padding: reserved for future use
*
- * Note: struct gpioline_info embedded here has 32-bit alignment on its own,
+ * The &struct gpioline_info embedded here has 32-bit alignment on its own,
* but it works fine with 64-bit alignment too. With its 72 byte size, we can
* guarantee there are no implicit holes between it and subsequent members.
* The 20-byte padding at the end makes sure we don't add any implicit padding
* at the end of the structure on 64-bit architectures.
*
- * This struct is part of ABI v1 and is deprecated.
- * Use struct gpio_v2_line_info_changed instead.
+ * Note: This struct is part of ABI v1 and is deprecated.
+ * Use &struct gpio_v2_line_info_changed instead.
*/
struct gpioline_info_changed {
struct gpioline_info info;
* @lineoffsets: an array of desired lines, specified by offset index for the
* associated GPIO device
* @flags: desired flags for the desired GPIO lines, such as
- * GPIOHANDLE_REQUEST_OUTPUT, GPIOHANDLE_REQUEST_ACTIVE_LOW etc, OR:ed
+ * %GPIOHANDLE_REQUEST_OUTPUT, %GPIOHANDLE_REQUEST_ACTIVE_LOW etc, added
* together. Note that even if multiple lines are requested, the same flags
* must be applicable to all of them, if you want lines with individual
* flags set, request them one by one. It is possible to select
* a batch of input or output lines, but they must all have the same
* characteristics, i.e. all inputs or all outputs, all active low etc
- * @default_values: if the GPIOHANDLE_REQUEST_OUTPUT is set for a requested
+ * @default_values: if the %GPIOHANDLE_REQUEST_OUTPUT is set for a requested
* line, this specifies the default output value, should be 0 (low) or
* 1 (high), anything else than 0 or 1 will be interpreted as 1 (high)
* @consumer_label: a desired consumer label for the selected GPIO line(s)
* @lines: number of lines requested in this request, i.e. the number of
* valid fields in the above arrays, set to 1 to request a single line
* @fd: if successful this field will contain a valid anonymous file handle
- * after a GPIO_GET_LINEHANDLE_IOCTL operation, zero or negative value
+ * after a %GPIO_GET_LINEHANDLE_IOCTL operation, zero or negative value
* means error
*
- * This struct is part of ABI v1 and is deprecated.
- * Use struct gpio_v2_line_request instead.
+ * Note: This struct is part of ABI v1 and is deprecated.
+ * Use &struct gpio_v2_line_request instead.
*/
struct gpiohandle_request {
__u32 lineoffsets[GPIOHANDLES_MAX];
/**
* struct gpiohandle_config - Configuration for a GPIO handle request
* @flags: updated flags for the requested GPIO lines, such as
- * GPIOHANDLE_REQUEST_OUTPUT, GPIOHANDLE_REQUEST_ACTIVE_LOW etc, OR:ed
+ * %GPIOHANDLE_REQUEST_OUTPUT, %GPIOHANDLE_REQUEST_ACTIVE_LOW etc, added
* together
- * @default_values: if the GPIOHANDLE_REQUEST_OUTPUT is set in flags,
+ * @default_values: if the %GPIOHANDLE_REQUEST_OUTPUT is set in flags,
* this specifies the default output value, should be 0 (low) or
* 1 (high), anything else than 0 or 1 will be interpreted as 1 (high)
* @padding: reserved for future use and should be zero filled
*
- * This struct is part of ABI v1 and is deprecated.
- * Use struct gpio_v2_line_config instead.
+ * Note: This struct is part of ABI v1 and is deprecated.
+ * Use &struct gpio_v2_line_config instead.
*/
struct gpiohandle_config {
__u32 flags;
* state of a line, when setting the state of lines these should contain
* the desired target state
*
- * This struct is part of ABI v1 and is deprecated.
- * Use struct gpio_v2_line_values instead.
+ * Note: This struct is part of ABI v1 and is deprecated.
+ * Use &struct gpio_v2_line_values instead.
*/
struct gpiohandle_data {
__u8 values[GPIOHANDLES_MAX];
* @lineoffset: the desired line to subscribe to events from, specified by
* offset index for the associated GPIO device
* @handleflags: desired handle flags for the desired GPIO line, such as
- * GPIOHANDLE_REQUEST_ACTIVE_LOW or GPIOHANDLE_REQUEST_OPEN_DRAIN
+ * %GPIOHANDLE_REQUEST_ACTIVE_LOW or %GPIOHANDLE_REQUEST_OPEN_DRAIN
* @eventflags: desired flags for the desired GPIO event line, such as
- * GPIOEVENT_REQUEST_RISING_EDGE or GPIOEVENT_REQUEST_FALLING_EDGE
+ * %GPIOEVENT_REQUEST_RISING_EDGE or %GPIOEVENT_REQUEST_FALLING_EDGE
* @consumer_label: a desired consumer label for the selected GPIO line(s)
* such as "my-listener"
* @fd: if successful this field will contain a valid anonymous file handle
- * after a GPIO_GET_LINEEVENT_IOCTL operation, zero or negative value
+ * after a %GPIO_GET_LINEEVENT_IOCTL operation, zero or negative value
* means error
*
- * This struct is part of ABI v1 and is deprecated.
- * Use struct gpio_v2_line_request instead.
+ * Note: This struct is part of ABI v1 and is deprecated.
+ * Use &struct gpio_v2_line_request instead.
*/
struct gpioevent_request {
__u32 lineoffset;
int fd;
};
-/**
+/*
* GPIO event types
*/
#define GPIOEVENT_EVENT_RISING_EDGE 0x01
* @timestamp: best estimate of time of event occurrence, in nanoseconds
* @id: event identifier
*
- * This struct is part of ABI v1 and is deprecated.
- * Use struct gpio_v2_line_event instead.
+ * Note: This struct is part of ABI v1 and is deprecated.
+ * Use &struct gpio_v2_line_event instead.
*/
struct gpioevent_data {
__u64 timestamp;
+++ /dev/null
-/* SPDX-License-Identifier: GPL-2.0+ WITH Linux-syscall-note */
-/*
- * DLCI/FRAD Definitions for Frame Relay Access Devices. DLCI devices are
- * created for each DLCI associated with a FRAD. The FRAD driver
- * is not truly a network device, but the lower level device
- * handler. This allows other FRAD manufacturers to use the DLCI
- * code, including its RFC1490 encapsulation alongside the current
- * implementation for the Sangoma cards.
- *
- * Version: @(#)if_ifrad.h 0.15 31 Mar 96
- *
- * Author: Mike McLagan <mike.mclagan@linux.org>
- *
- * Changes:
- * 0.15 Mike McLagan changed structure defs (packed)
- * re-arranged flags
- * added DLCI_RET vars
- *
- * This program is free software; you can redistribute it and/or
- * modify it under the terms of the GNU General Public License
- * as published by the Free Software Foundation; either version
- * 2 of the License, or (at your option) any later version.
- */
-
-#ifndef _UAPI_FRAD_H_
-#define _UAPI_FRAD_H_
-
-#include <linux/if.h>
-
-/* Structures and constants associated with the DLCI device driver */
-
-struct dlci_add
-{
- char devname[IFNAMSIZ];
- short dlci;
-};
-
-#define DLCI_GET_CONF (SIOCDEVPRIVATE + 2)
-#define DLCI_SET_CONF (SIOCDEVPRIVATE + 3)
-
-/*
- * These are related to the Sangoma SDLA and should remain in order.
- * Code within the SDLA module is based on the specifics of this
- * structure. Change at your own peril.
- */
-struct dlci_conf {
- short flags;
- short CIR_fwd;
- short Bc_fwd;
- short Be_fwd;
- short CIR_bwd;
- short Bc_bwd;
- short Be_bwd;
-
-/* these are part of the status read */
- short Tc_fwd;
- short Tc_bwd;
- short Tf_max;
- short Tb_max;
-
-/* add any new fields here above is a mirror of sdla_dlci_conf */
-};
-
-#define DLCI_GET_SLAVE (SIOCDEVPRIVATE + 4)
-
-/* configuration flags for DLCI */
-#define DLCI_IGNORE_CIR_OUT 0x0001
-#define DLCI_ACCOUNT_CIR_IN 0x0002
-#define DLCI_BUFFER_IF 0x0008
-
-#define DLCI_VALID_FLAGS 0x000B
-
-/* defines for the actual Frame Relay hardware */
-#define FRAD_GET_CONF (SIOCDEVPRIVATE)
-#define FRAD_SET_CONF (SIOCDEVPRIVATE + 1)
-
-#define FRAD_LAST_IOCTL FRAD_SET_CONF
-
-/*
- * Based on the setup for the Sangoma SDLA. If changes are
- * necessary to this structure, a routine will need to be
- * added to that module to copy fields.
- */
-struct frad_conf
-{
- short station;
- short flags;
- short kbaud;
- short clocking;
- short mtu;
- short T391;
- short T392;
- short N391;
- short N392;
- short N393;
- short CIR_fwd;
- short Bc_fwd;
- short Be_fwd;
- short CIR_bwd;
- short Bc_bwd;
- short Be_bwd;
-
-/* Add new fields here, above is a mirror of the sdla_conf */
-
-};
-
-#define FRAD_STATION_CPE 0x0000
-#define FRAD_STATION_NODE 0x0001
-
-#define FRAD_TX_IGNORE_CIR 0x0001
-#define FRAD_RX_ACCOUNT_CIR 0x0002
-#define FRAD_DROP_ABORTED 0x0004
-#define FRAD_BUFFERIF 0x0008
-#define FRAD_STATS 0x0010
-#define FRAD_MCI 0x0100
-#define FRAD_AUTODLCI 0x8000
-#define FRAD_VALID_FLAGS 0x811F
-
-#define FRAD_CLOCK_INT 0x0001
-#define FRAD_CLOCK_EXT 0x0000
-
-
-#endif /* _UAPI_FRAD_H_ */
IFLA_MACVLAN_MACADDR,
IFLA_MACVLAN_MACADDR_DATA,
IFLA_MACVLAN_MACADDR_COUNT,
+ IFLA_MACVLAN_BC_QUEUE_LEN,
+ IFLA_MACVLAN_BC_QUEUE_LEN_USED,
__IFLA_MACVLAN_MAX,
};
BR_MRP_TLV_HEADER_IN_TOPO = 0x7,
BR_MRP_TLV_HEADER_IN_LINK_DOWN = 0x8,
BR_MRP_TLV_HEADER_IN_LINK_UP = 0x9,
+ BR_MRP_TLV_HEADER_IN_LINK_STATUS = 0xa,
BR_MRP_TLV_HEADER_OPTION = 0x7f,
};
__OVS_DEC_TTL_ATTR_MAX
};
+#define OVS_DEC_TTL_ATTR_MAX (__OVS_DEC_TTL_ATTR_MAX - 1)
+
#endif /* _LINUX_OPENVSWITCH_H */
#define TA_PAYLOAD(n) NLMSG_PAYLOAD(n,sizeof(struct tcamsg))
/* tcamsg flags stored in attribute TCA_ROOT_FLAGS
*
- * TCA_FLAG_LARGE_DUMP_ON user->kernel to request for larger than TCA_ACT_MAX_PRIO
- * actions in a dump. All dump responses will contain the number of actions
- * being dumped stored in for user app's consumption in TCA_ROOT_COUNT
+ * TCA_ACT_FLAG_LARGE_DUMP_ON user->kernel to request for larger than
+ * TCA_ACT_MAX_PRIO actions in a dump. All dump responses will contain the
+ * number of actions being dumped stored in for user app's consumption in
+ * TCA_ROOT_COUNT
*
- * TCA_FLAG_TERSE_DUMP user->kernel to request terse (brief) dump that only
+ * TCA_ACT_FLAG_TERSE_DUMP user->kernel to request terse (brief) dump that only
* includes essential action info (kind, index, etc.)
*
*/
#define TCA_FLAG_LARGE_DUMP_ON (1 << 0)
-#define TCA_FLAG_TERSE_DUMP (1 << 1)
+#define TCA_ACT_FLAG_LARGE_DUMP_ON TCA_FLAG_LARGE_DUMP_ON
+#define TCA_ACT_FLAG_TERSE_DUMP (1 << 1)
/* New extended info filters for IFLA_EXT_MASK */
#define RTEXT_FILTER_VF (1 << 0)
+++ /dev/null
-/* SPDX-License-Identifier: GPL-2.0+ WITH Linux-syscall-note */
-/*
- * INET An implementation of the TCP/IP protocol suite for the LINUX
- * operating system. INET is implemented using the BSD Socket
- * interface as the means of communication with the user level.
- *
- * Global definitions for the Frame relay interface.
- *
- * Version: @(#)if_ifrad.h 0.20 13 Apr 96
- *
- * Author: Mike McLagan <mike.mclagan@linux.org>
- *
- * Changes:
- * 0.15 Mike McLagan Structure packing
- *
- * 0.20 Mike McLagan New flags for S508 buffer handling
- *
- * This program is free software; you can redistribute it and/or
- * modify it under the terms of the GNU General Public License
- * as published by the Free Software Foundation; either version
- * 2 of the License, or (at your option) any later version.
- */
-
-#ifndef _UAPISDLA_H
-#define _UAPISDLA_H
-
-/* adapter type */
-#define SDLA_TYPES
-#define SDLA_S502A 5020
-#define SDLA_S502E 5021
-#define SDLA_S503 5030
-#define SDLA_S507 5070
-#define SDLA_S508 5080
-#define SDLA_S509 5090
-#define SDLA_UNKNOWN -1
-
-/* port selection flags for the S508 */
-#define SDLA_S508_PORT_V35 0x00
-#define SDLA_S508_PORT_RS232 0x02
-
-/* Z80 CPU speeds */
-#define SDLA_CPU_3M 0x00
-#define SDLA_CPU_5M 0x01
-#define SDLA_CPU_7M 0x02
-#define SDLA_CPU_8M 0x03
-#define SDLA_CPU_10M 0x04
-#define SDLA_CPU_16M 0x05
-#define SDLA_CPU_12M 0x06
-
-/* some private IOCTLs */
-#define SDLA_IDENTIFY (FRAD_LAST_IOCTL + 1)
-#define SDLA_CPUSPEED (FRAD_LAST_IOCTL + 2)
-#define SDLA_PROTOCOL (FRAD_LAST_IOCTL + 3)
-
-#define SDLA_CLEARMEM (FRAD_LAST_IOCTL + 4)
-#define SDLA_WRITEMEM (FRAD_LAST_IOCTL + 5)
-#define SDLA_READMEM (FRAD_LAST_IOCTL + 6)
-
-struct sdla_mem {
- int addr;
- int len;
- void __user *data;
-};
-
-#define SDLA_START (FRAD_LAST_IOCTL + 7)
-#define SDLA_STOP (FRAD_LAST_IOCTL + 8)
-
-/* some offsets in the Z80's memory space */
-#define SDLA_NMIADDR 0x0000
-#define SDLA_CONF_ADDR 0x0010
-#define SDLA_S502A_NMIADDR 0x0066
-#define SDLA_CODE_BASEADDR 0x0100
-#define SDLA_WINDOW_SIZE 0x2000
-#define SDLA_ADDR_MASK 0x1FFF
-
-/* largest handleable block of data */
-#define SDLA_MAX_DATA 4080
-#define SDLA_MAX_MTU 4072 /* MAX_DATA - sizeof(fradhdr) */
-#define SDLA_MAX_DLCI 24
-
-/* this should be the same as frad_conf */
-struct sdla_conf {
- short station;
- short config;
- short kbaud;
- short clocking;
- short max_frm;
- short T391;
- short T392;
- short N391;
- short N392;
- short N393;
- short CIR_fwd;
- short Bc_fwd;
- short Be_fwd;
- short CIR_bwd;
- short Bc_bwd;
- short Be_bwd;
-};
-
-/* this should be the same as dlci_conf */
-struct sdla_dlci_conf {
- short config;
- short CIR_fwd;
- short Bc_fwd;
- short Be_fwd;
- short CIR_bwd;
- short Bc_bwd;
- short Be_bwd;
- short Tc_fwd;
- short Tc_bwd;
- short Tf_max;
- short Tb_max;
-};
-
-
-#endif /* _UAPISDLA_H */
#define SMCR_GENL_FAMILY_NAME "SMC_PNETID"
#define SMCR_GENL_FAMILY_VERSION 1
+/* gennetlink interface to access non-socket information from SMC module */
+#define SMC_GENL_FAMILY_NAME "SMC_GEN_NETLINK"
+#define SMC_GENL_FAMILY_VERSION 1
+
+#define SMC_PCI_ID_STR_LEN 16 /* Max length of pci id string */
+
+/* SMC_GENL_FAMILY commands */
+enum {
+ SMC_NETLINK_GET_SYS_INFO = 1,
+ SMC_NETLINK_GET_LGR_SMCR,
+ SMC_NETLINK_GET_LINK_SMCR,
+ SMC_NETLINK_GET_LGR_SMCD,
+ SMC_NETLINK_GET_DEV_SMCD,
+ SMC_NETLINK_GET_DEV_SMCR,
+};
+
+/* SMC_GENL_FAMILY top level attributes */
+enum {
+ SMC_GEN_UNSPEC,
+ SMC_GEN_SYS_INFO, /* nest */
+ SMC_GEN_LGR_SMCR, /* nest */
+ SMC_GEN_LINK_SMCR, /* nest */
+ SMC_GEN_LGR_SMCD, /* nest */
+ SMC_GEN_DEV_SMCD, /* nest */
+ SMC_GEN_DEV_SMCR, /* nest */
+ __SMC_GEN_MAX,
+ SMC_GEN_MAX = __SMC_GEN_MAX - 1
+};
+
+/* SMC_GEN_SYS_INFO attributes */
+enum {
+ SMC_NLA_SYS_UNSPEC,
+ SMC_NLA_SYS_VER, /* u8 */
+ SMC_NLA_SYS_REL, /* u8 */
+ SMC_NLA_SYS_IS_ISM_V2, /* u8 */
+ SMC_NLA_SYS_LOCAL_HOST, /* string */
+ SMC_NLA_SYS_SEID, /* string */
+ __SMC_NLA_SYS_MAX,
+ SMC_NLA_SYS_MAX = __SMC_NLA_SYS_MAX - 1
+};
+
+/* SMC_NLA_LGR_V2 nested attributes */
+enum {
+ SMC_NLA_LGR_V2_VER, /* u8 */
+ SMC_NLA_LGR_V2_REL, /* u8 */
+ SMC_NLA_LGR_V2_OS, /* u8 */
+ SMC_NLA_LGR_V2_NEG_EID, /* string */
+ SMC_NLA_LGR_V2_PEER_HOST, /* string */
+};
+
+/* SMC_GEN_LGR_SMCR attributes */
+enum {
+ SMC_NLA_LGR_R_UNSPEC,
+ SMC_NLA_LGR_R_ID, /* u32 */
+ SMC_NLA_LGR_R_ROLE, /* u8 */
+ SMC_NLA_LGR_R_TYPE, /* u8 */
+ SMC_NLA_LGR_R_PNETID, /* string */
+ SMC_NLA_LGR_R_VLAN_ID, /* u8 */
+ SMC_NLA_LGR_R_CONNS_NUM, /* u32 */
+ __SMC_NLA_LGR_R_MAX,
+ SMC_NLA_LGR_R_MAX = __SMC_NLA_LGR_R_MAX - 1
+};
+
+/* SMC_GEN_LINK_SMCR attributes */
+enum {
+ SMC_NLA_LINK_UNSPEC,
+ SMC_NLA_LINK_ID, /* u8 */
+ SMC_NLA_LINK_IB_DEV, /* string */
+ SMC_NLA_LINK_IB_PORT, /* u8 */
+ SMC_NLA_LINK_GID, /* string */
+ SMC_NLA_LINK_PEER_GID, /* string */
+ SMC_NLA_LINK_CONN_CNT, /* u32 */
+ SMC_NLA_LINK_NET_DEV, /* u32 */
+ SMC_NLA_LINK_UID, /* u32 */
+ SMC_NLA_LINK_PEER_UID, /* u32 */
+ SMC_NLA_LINK_STATE, /* u32 */
+ __SMC_NLA_LINK_MAX,
+ SMC_NLA_LINK_MAX = __SMC_NLA_LINK_MAX - 1
+};
+
+/* SMC_GEN_LGR_SMCD attributes */
+enum {
+ SMC_NLA_LGR_D_UNSPEC,
+ SMC_NLA_LGR_D_ID, /* u32 */
+ SMC_NLA_LGR_D_GID, /* u64 */
+ SMC_NLA_LGR_D_PEER_GID, /* u64 */
+ SMC_NLA_LGR_D_VLAN_ID, /* u8 */
+ SMC_NLA_LGR_D_CONNS_NUM, /* u32 */
+ SMC_NLA_LGR_D_PNETID, /* string */
+ SMC_NLA_LGR_D_CHID, /* u16 */
+ SMC_NLA_LGR_D_PAD, /* flag */
+ SMC_NLA_LGR_V2, /* nest */
+ __SMC_NLA_LGR_D_MAX,
+ SMC_NLA_LGR_D_MAX = __SMC_NLA_LGR_D_MAX - 1
+};
+
+/* SMC_NLA_DEV_PORT nested attributes */
+enum {
+ SMC_NLA_DEV_PORT_UNSPEC,
+ SMC_NLA_DEV_PORT_PNET_USR, /* u8 */
+ SMC_NLA_DEV_PORT_PNETID, /* string */
+ SMC_NLA_DEV_PORT_NETDEV, /* u32 */
+ SMC_NLA_DEV_PORT_STATE, /* u8 */
+ SMC_NLA_DEV_PORT_VALID, /* u8 */
+ SMC_NLA_DEV_PORT_LNK_CNT, /* u32 */
+ __SMC_NLA_DEV_PORT_MAX,
+ SMC_NLA_DEV_PORT_MAX = __SMC_NLA_DEV_PORT_MAX - 1
+};
+
+/* SMC_GEN_DEV_SMCD and SMC_GEN_DEV_SMCR attributes */
+enum {
+ SMC_NLA_DEV_UNSPEC,
+ SMC_NLA_DEV_USE_CNT, /* u32 */
+ SMC_NLA_DEV_IS_CRIT, /* u8 */
+ SMC_NLA_DEV_PCI_FID, /* u32 */
+ SMC_NLA_DEV_PCI_CHID, /* u16 */
+ SMC_NLA_DEV_PCI_VENDOR, /* u16 */
+ SMC_NLA_DEV_PCI_DEVICE, /* u16 */
+ SMC_NLA_DEV_PCI_ID, /* string */
+ SMC_NLA_DEV_PORT, /* nest */
+ SMC_NLA_DEV_PORT2, /* nest */
+ SMC_NLA_DEV_IB_NAME, /* string */
+ __SMC_NLA_DEV_MAX,
+ SMC_NLA_DEV_MAX = __SMC_NLA_DEV_MAX - 1
+};
+
#endif /* _UAPI_LINUX_SMC_H */
* be of use to ordinary userspace programs such as GUIs or ls rather than
* specialised tools.
*
- * Note that the flags marked [I] correspond to generic FS_IOC_FLAGS
+ * Note that the flags marked [I] correspond to the FS_IOC_SETFLAGS flags
* semantically. Where possible, the numerical value is picked to correspond
- * also.
+ * also. Note that the DAX attribute indicates that the file is in the CPU
+ * direct access state. It does not correspond to the per-inode flag that
+ * some filesystems support.
+ *
*/
#define STATX_ATTR_COMPRESSED 0x00000004 /* [I] File is compressed by the fs */
#define STATX_ATTR_IMMUTABLE 0x00000010 /* [I] File is marked immutable */
#define STATX_ATTR_AUTOMOUNT 0x00001000 /* Dir: Automount trigger */
#define STATX_ATTR_MOUNT_ROOT 0x00002000 /* Root of a mount */
#define STATX_ATTR_VERITY 0x00100000 /* [I] Verity protected file */
-#define STATX_ATTR_DAX 0x00002000 /* [I] File is DAX */
+#define STATX_ATTR_DAX 0x00200000 /* File is currently in DAX state */
#endif /* _UAPI_LINUX_STAT_H */
#define TLS_CIPHER_AES_CCM_128_TAG_SIZE 16
#define TLS_CIPHER_AES_CCM_128_REC_SEQ_SIZE 8
+#define TLS_CIPHER_CHACHA20_POLY1305 54
+#define TLS_CIPHER_CHACHA20_POLY1305_IV_SIZE 12
+#define TLS_CIPHER_CHACHA20_POLY1305_KEY_SIZE 32
+#define TLS_CIPHER_CHACHA20_POLY1305_SALT_SIZE 0
+#define TLS_CIPHER_CHACHA20_POLY1305_TAG_SIZE 16
+#define TLS_CIPHER_CHACHA20_POLY1305_REC_SEQ_SIZE 8
+
#define TLS_SET_RECORD_TYPE 1
#define TLS_GET_RECORD_TYPE 2
unsigned char rec_seq[TLS_CIPHER_AES_CCM_128_REC_SEQ_SIZE];
};
+struct tls12_crypto_info_chacha20_poly1305 {
+ struct tls_crypto_info info;
+ unsigned char iv[TLS_CIPHER_CHACHA20_POLY1305_IV_SIZE];
+ unsigned char key[TLS_CIPHER_CHACHA20_POLY1305_KEY_SIZE];
+ unsigned char salt[TLS_CIPHER_CHACHA20_POLY1305_SALT_SIZE];
+ unsigned char rec_seq[TLS_CIPHER_CHACHA20_POLY1305_REC_SEQ_SIZE];
+};
+
enum {
TLS_INFO_UNSPEC,
TLS_INFO_VERSION,
MLX5_IB_ATTR_QUERY_CONTEXT_RESP_UCTX = (1U << UVERBS_ID_NS_SHIFT),
};
-#define MLX5_IB_DW_MATCH_PARAM 0x80
+#define MLX5_IB_DW_MATCH_PARAM 0x90
struct mlx5_ib_match_params {
__u32 match_params[MLX5_IB_DW_MATCH_PARAM];
with more CPUs. Therefore this value is used only when the sum of
contributions is greater than the half of the default kernel ring
buffer as defined by LOG_BUF_SHIFT. The default values are set
- so that more than 64 CPUs are needed to trigger the allocation.
+ so that more than 16 CPUs are needed to trigger the allocation.
Also this option is ignored when "log_buf_len" kernel parameter is
used as it forces an exact (power of two) size of the ring buffer.
u32 size, csum;
char *data;
u32 *hdr;
+ int i;
if (!initrd_end)
return NULL;
data = (char *)initrd_end - BOOTCONFIG_MAGIC_LEN;
- if (memcmp(data, BOOTCONFIG_MAGIC, BOOTCONFIG_MAGIC_LEN))
- return NULL;
+ /*
+ * Since Grub may align the size of initrd to 4, we must
+ * check the preceding 3 bytes as well.
+ */
+ for (i = 0; i < 4; i++) {
+ if (!memcmp(data, BOOTCONFIG_MAGIC, BOOTCONFIG_MAGIC_LEN))
+ goto found;
+ data--;
+ }
+ return NULL;
+found:
hdr = (u32 *)(data - 8);
- size = hdr[0];
- csum = hdr[1];
+ size = le32_to_cpu(hdr[0]);
+ csum = le32_to_cpu(hdr[1]);
data = ((void *)hdr) - size;
if ((unsigned long)data < initrd_start) {
struct tnum range = tnum_range(0, 1);
enum bpf_prog_type prog_type = resolve_prog_type(env->prog);
int err;
+ const bool is_subprog = env->cur_state->frame[0]->subprogno;
/* LSM and struct_ops func-ptr's return type could be "void" */
- if ((prog_type == BPF_PROG_TYPE_STRUCT_OPS ||
+ if (!is_subprog &&
+ (prog_type == BPF_PROG_TYPE_STRUCT_OPS ||
prog_type == BPF_PROG_TYPE_LSM) &&
!prog->aux->attach_func_proto->type)
return 0;
return -EACCES;
}
+ reg = cur_regs(env) + BPF_REG_0;
+ if (is_subprog) {
+ if (reg->type != SCALAR_VALUE) {
+ verbose(env, "At subprogram exit the register R0 is not a scalar value (%s)\n",
+ reg_type_str[reg->type]);
+ return -EINVAL;
+ }
+ return 0;
+ }
+
switch (prog_type) {
case BPF_PROG_TYPE_CGROUP_SOCK_ADDR:
if (env->prog->expected_attach_type == BPF_CGROUP_UDP4_RECVMSG ||
return 0;
}
- reg = cur_regs(env) + BPF_REG_0;
if (reg->type != SCALAR_VALUE) {
verbose(env, "At program exit the register R0 is not a known value (%s)\n",
reg_type_str[reg->type]);
struct bpf_insn *insn,
struct bpf_insn_aux_data *aux)
{
- u32 datasec_id, type, id = insn->imm;
const struct btf_var_secinfo *vsi;
const struct btf_type *datasec;
const struct btf_type *t;
const char *sym_name;
bool percpu = false;
+ u32 type, id = insn->imm;
+ s32 datasec_id;
u64 addr;
int i;
event_sched_out(event, cpuctx, ctx);
perf_pmu_enable(ctx->pmu);
-
- if (group_event->attr.exclusive)
- cpuctx->exclusive = 0;
}
#define DETACH_GROUP 0x01UL
pmu->start_txn(pmu, PERF_PMU_TXN_ADD);
- if (event_sched_in(group_event, cpuctx, ctx)) {
- pmu->cancel_txn(pmu);
- perf_mux_hrtimer_restart(cpuctx);
- return -EAGAIN;
- }
+ if (event_sched_in(group_event, cpuctx, ctx))
+ goto error;
/*
* Schedule in siblings as one group (if any):
}
event_sched_out(group_event, cpuctx, ctx);
+error:
pmu->cancel_txn(pmu);
-
- perf_mux_hrtimer_restart(cpuctx);
-
return -EAGAIN;
}
* If this group is exclusive and there are already
* events on the CPU, it can't go on.
*/
- if (event->attr.exclusive && cpuctx->active_oncpu)
+ if (event->attr.exclusive && !list_empty(get_event_list(event)))
return 0;
/*
* Otherwise, try to add it if all previous groups were able
*can_add_hw = 0;
ctx->rotate_necessary = 1;
+ perf_mux_hrtimer_restart(cpuctx);
}
return 0;
}
static void perf_sample_regs_user(struct perf_regs *regs_user,
- struct pt_regs *regs,
- struct pt_regs *regs_user_copy)
+ struct pt_regs *regs)
{
if (user_mode(regs)) {
regs_user->abi = perf_reg_abi(current);
regs_user->regs = regs;
} else if (!(current->flags & PF_KTHREAD)) {
- perf_get_regs_user(regs_user, regs, regs_user_copy);
+ perf_get_regs_user(regs_user, regs);
} else {
regs_user->abi = PERF_SAMPLE_REGS_ABI_NONE;
regs_user->regs = NULL;
}
if (sample_type & (PERF_SAMPLE_REGS_USER | PERF_SAMPLE_STACK_USER))
- perf_sample_regs_user(&data->regs_user, regs,
- &data->regs_user_copy);
+ perf_sample_regs_user(&data->regs_user, regs);
if (sample_type & PERF_SAMPLE_REGS_USER) {
/* regs dump ABI info */
struct perf_sample_data *data,
struct pt_regs *regs,
int (*output_begin)(struct perf_output_handle *,
+ struct perf_sample_data *,
struct perf_event *,
unsigned int))
{
perf_prepare_sample(&header, data, event, regs);
- err = output_begin(&handle, event, header.size);
+ err = output_begin(&handle, data, event, header.size);
if (err)
goto exit;
int ret;
perf_event_header__init_id(&read_event.header, &sample, event);
- ret = perf_output_begin(&handle, event, read_event.header.size);
+ ret = perf_output_begin(&handle, &sample, event, read_event.header.size);
if (ret)
return;
perf_event_header__init_id(&task_event->event_id.header, &sample, event);
- ret = perf_output_begin(&handle, event,
+ ret = perf_output_begin(&handle, &sample, event,
task_event->event_id.header.size);
if (ret)
goto out;
return;
perf_event_header__init_id(&comm_event->event_id.header, &sample, event);
- ret = perf_output_begin(&handle, event,
+ ret = perf_output_begin(&handle, &sample, event,
comm_event->event_id.header.size);
if (ret)
perf_event_header__init_id(&namespaces_event->event_id.header,
&sample, event);
- ret = perf_output_begin(&handle, event,
+ ret = perf_output_begin(&handle, &sample, event,
namespaces_event->event_id.header.size);
if (ret)
goto out;
perf_event_header__init_id(&cgroup_event->event_id.header,
&sample, event);
- ret = perf_output_begin(&handle, event,
+ ret = perf_output_begin(&handle, &sample, event,
cgroup_event->event_id.header.size);
if (ret)
goto out;
}
perf_event_header__init_id(&mmap_event->event_id.header, &sample, event);
- ret = perf_output_begin(&handle, event,
+ ret = perf_output_begin(&handle, &sample, event,
mmap_event->event_id.header.size);
if (ret)
goto out;
int ret;
perf_event_header__init_id(&rec.header, &sample, event);
- ret = perf_output_begin(&handle, event, rec.header.size);
+ ret = perf_output_begin(&handle, &sample, event, rec.header.size);
if (ret)
return;
perf_event_header__init_id(&lost_samples_event.header, &sample, event);
- ret = perf_output_begin(&handle, event,
+ ret = perf_output_begin(&handle, &sample, event,
lost_samples_event.header.size);
if (ret)
return;
perf_event_header__init_id(&se->event_id.header, &sample, event);
- ret = perf_output_begin(&handle, event, se->event_id.header.size);
+ ret = perf_output_begin(&handle, &sample, event, se->event_id.header.size);
if (ret)
return;
perf_event_header__init_id(&throttle_event.header, &sample, event);
- ret = perf_output_begin(&handle, event,
+ ret = perf_output_begin(&handle, &sample, event,
throttle_event.header.size);
if (ret)
return;
perf_event_header__init_id(&ksymbol_event->event_id.header,
&sample, event);
- ret = perf_output_begin(&handle, event,
+ ret = perf_output_begin(&handle, &sample, event,
ksymbol_event->event_id.header.size);
if (ret)
return;
perf_event_header__init_id(&bpf_event->event_id.header,
&sample, event);
- ret = perf_output_begin(&handle, event,
+ ret = perf_output_begin(&handle, data, event,
bpf_event->event_id.header.size);
if (ret)
return;
perf_event_header__init_id(&text_poke_event->event_id.header, &sample, event);
- ret = perf_output_begin(&handle, event, text_poke_event->event_id.header.size);
+ ret = perf_output_begin(&handle, &sample, event,
+ text_poke_event->event_id.header.size);
if (ret)
return;
rec.tid = perf_event_tid(event, current);
perf_event_header__init_id(&rec.header, &sample, event);
- ret = perf_output_begin(&handle, event, rec.header.size);
+ ret = perf_output_begin(&handle, &sample, event, rec.header.size);
if (ret)
return;
static inline int get_recursion_context(int *recursion)
{
- int rctx;
-
- if (unlikely(in_nmi()))
- rctx = 3;
- else if (in_irq())
- rctx = 2;
- else if (in_softirq())
- rctx = 1;
- else
- rctx = 0;
+ unsigned int pc = preempt_count();
+ unsigned char rctx = 0;
+
+ rctx += !!(pc & (NMI_MASK));
+ rctx += !!(pc & (NMI_MASK | HARDIRQ_MASK));
+ rctx += !!(pc & (NMI_MASK | HARDIRQ_MASK | SOFTIRQ_OFFSET));
if (recursion[rctx])
return -1;
static __always_inline int
__perf_output_begin(struct perf_output_handle *handle,
+ struct perf_sample_data *data,
struct perf_event *event, unsigned int size,
bool backward)
{
handle->size = (1UL << page_shift) - offset;
if (unlikely(have_lost)) {
- struct perf_sample_data sample_data;
-
lost_event.header.size = sizeof(lost_event);
lost_event.header.type = PERF_RECORD_LOST;
lost_event.header.misc = 0;
lost_event.id = event->id;
lost_event.lost = local_xchg(&rb->lost, 0);
- perf_event_header__init_id(&lost_event.header,
- &sample_data, event);
+ /* XXX mostly redundant; @data is already fully initializes */
+ perf_event_header__init_id(&lost_event.header, data, event);
perf_output_put(handle, lost_event);
- perf_event__output_id_sample(event, handle, &sample_data);
+ perf_event__output_id_sample(event, handle, data);
}
return 0;
}
int perf_output_begin_forward(struct perf_output_handle *handle,
- struct perf_event *event, unsigned int size)
+ struct perf_sample_data *data,
+ struct perf_event *event, unsigned int size)
{
- return __perf_output_begin(handle, event, size, false);
+ return __perf_output_begin(handle, data, event, size, false);
}
int perf_output_begin_backward(struct perf_output_handle *handle,
+ struct perf_sample_data *data,
struct perf_event *event, unsigned int size)
{
- return __perf_output_begin(handle, event, size, true);
+ return __perf_output_begin(handle, data, event, size, true);
}
int perf_output_begin(struct perf_output_handle *handle,
+ struct perf_sample_data *data,
struct perf_event *event, unsigned int size)
{
- return __perf_output_begin(handle, event, size,
+ return __perf_output_begin(handle, data, event, size,
unlikely(is_write_backward(event)));
}
if (copy_from_user(buf, buffer, count)) {
ret = -EFAULT;
- goto out;
+ goto out_free;
}
buf[count] = '\0';
sym = strstrip(buf);
ret = count;
}
out:
- kfree(buf);
mutex_unlock(&fei_lock);
+out_free:
+ kfree(buf);
return ret;
}
*/
if (pi_state->owner) {
struct task_struct *owner;
+ unsigned long flags;
- raw_spin_lock_irq(&pi_state->pi_mutex.wait_lock);
+ raw_spin_lock_irqsave(&pi_state->pi_mutex.wait_lock, flags);
owner = pi_state->owner;
if (owner) {
raw_spin_lock(&owner->pi_lock);
raw_spin_unlock(&owner->pi_lock);
}
rt_mutex_proxy_unlock(&pi_state->pi_mutex, owner);
- raw_spin_unlock_irq(&pi_state->pi_mutex.wait_lock);
+ raw_spin_unlock_irqrestore(&pi_state->pi_mutex.wait_lock, flags);
}
if (current->pi_state_cache) {
{
DEBUG_LOCKS_WARN_ON(!irqs_disabled());
+ __this_cpu_inc(lockdep_recursion);
arch_spin_lock(&__lock);
__owner = current;
- __this_cpu_inc(lockdep_recursion);
}
static inline void lockdep_unlock(void)
{
+ DEBUG_LOCKS_WARN_ON(!irqs_disabled());
+
if (debug_locks && DEBUG_LOCKS_WARN_ON(__owner != current))
return;
- __this_cpu_dec(lockdep_recursion);
__owner = NULL;
arch_spin_unlock(&__lock);
+ __this_cpu_dec(lockdep_recursion);
}
static inline bool lockdep_assert_locked(void)
* (Note that this has to be done separately, because the graph cannot
* detect such classes of deadlocks.)
*
- * Returns: 0 on deadlock detected, 1 on OK, 2 on recursive read
+ * Returns: 0 on deadlock detected, 1 on OK, 2 if another lock with the same
+ * lock class is held but nest_lock is also held, i.e. we rely on the
+ * nest_lock to avoid the deadlock.
*/
static int
check_deadlock(struct task_struct *curr, struct held_lock *next)
* lock class (i.e. read_lock(lock)+read_lock(lock)):
*/
if ((next->read == 2) && prev->read)
- return 2;
+ continue;
/*
* We're holding the nest_lock, which serializes this lock's
if (!ret)
return 0;
- /*
- * Mark recursive read, as we jump over it when
- * building dependencies (just like we jump over
- * trylock entries):
- */
- if (ret == 2)
- hlock->read = 2;
/*
* Add dependency only if this lock is not the head
- * of the chain, and if it's not a secondary read-lock:
+ * of the chain, and if the new lock introduces no more
+ * lock dependency (because we already hold a lock with the
+ * same lock class) nor deadlock (because the nest_lock
+ * serializes nesting locks), see the comments for
+ * check_deadlock().
*/
if (!chain_head && ret != 2) {
if (!check_prevs_add(curr, hlock))
panic("panic_on_warn set ...\n");
}
- dump_stack();
+ if (!regs)
+ dump_stack();
print_irqtrace_events(current);
if (dev_info)
memcpy(&r.info->dev_info, dev_info, sizeof(r.info->dev_info));
- /* insert message */
- if ((flags & LOG_CONT) || !(flags & LOG_NEWLINE))
+ /* A message without a trailing newline can be continued. */
+ if (!(flags & LOG_NEWLINE))
prb_commit(&e);
else
prb_final_commit(&e);
head_id = atomic_long_read(&desc_ring->head_id); /* LMM(desc_reserve:A) */
do {
- desc = to_desc(desc_ring, head_id);
-
id = DESC_ID(head_id + 1);
id_prev_wrap = DESC_ID_PREV_WRAP(desc_ring, id);
return ret;
}
-static bool ptrace_has_cap(const struct cred *cred, struct user_namespace *ns,
- unsigned int mode)
+static bool ptrace_has_cap(struct user_namespace *ns, unsigned int mode)
{
- int ret;
-
if (mode & PTRACE_MODE_NOAUDIT)
- ret = security_capable(cred, ns, CAP_SYS_PTRACE, CAP_OPT_NOAUDIT);
- else
- ret = security_capable(cred, ns, CAP_SYS_PTRACE, CAP_OPT_NONE);
-
- return ret == 0;
+ return ns_capable_noaudit(ns, CAP_SYS_PTRACE);
+ return ns_capable(ns, CAP_SYS_PTRACE);
}
/* Returns 0 on success, -errno on denial. */
gid_eq(caller_gid, tcred->sgid) &&
gid_eq(caller_gid, tcred->gid))
goto ok;
- if (ptrace_has_cap(cred, tcred->user_ns, mode))
+ if (ptrace_has_cap(tcred->user_ns, mode))
goto ok;
rcu_read_unlock();
return -EPERM;
mm = task->mm;
if (mm &&
((get_dumpable(mm) != SUID_DUMP_USER) &&
- !ptrace_has_cap(cred, mm->user_ns, mode)))
+ !ptrace_has_cap(mm->user_ns, mode)))
return -EPERM;
return security_ptrace_access_check(task, mode);
smp_mb(); /* Ensure RCU read-side usage follows above initialization. */
}
-#ifdef CONFIG_HOTPLUG_CPU
/*
* The outgoing function has no further need of RCU, so remove it from
* the rcu_node tree's ->qsmaskinitnext bit masks.
rdp->cpu_started = false;
}
+#ifdef CONFIG_HOTPLUG_CPU
/*
* The outgoing CPU has just passed through the dying-idle state, and we
* are being invoked from the CPU that was IPIed to continue the offline
/*
* Scan the current list of tasks blocked within RCU read-side critical
- * sections, printing out the tid of each.
+ * sections, printing out the tid of each of the first few of them.
*/
-static int rcu_print_task_stall(struct rcu_node *rnp)
+static int rcu_print_task_stall(struct rcu_node *rnp, unsigned long flags)
+ __releases(rnp->lock)
{
+ int i = 0;
int ndetected = 0;
struct rcu_stall_chk_rdr rscr;
struct task_struct *t;
+ struct task_struct *ts[8];
if (!rcu_preempt_blocked_readers_cgp(rnp))
return 0;
t = list_entry(rnp->gp_tasks->prev,
struct task_struct, rcu_node_entry);
list_for_each_entry_continue(t, &rnp->blkd_tasks, rcu_node_entry) {
+ get_task_struct(t);
+ ts[i++] = t;
+ if (i >= ARRAY_SIZE(ts))
+ break;
+ }
+ raw_spin_unlock_irqrestore_rcu_node(rnp, flags);
+ for (i--; i; i--) {
+ t = ts[i];
if (!try_invoke_on_locked_down_task(t, check_slow_task, &rscr))
pr_cont(" P%d", t->pid);
else
".q"[rscr.rs.b.need_qs],
".e"[rscr.rs.b.exp_hint],
".l"[rscr.on_blkd_list]);
+ put_task_struct(t);
ndetected++;
}
pr_cont("\n");
* Because preemptible RCU does not exist, we never have to check for
* tasks blocked within RCU read-side critical sections.
*/
-static int rcu_print_task_stall(struct rcu_node *rnp)
+static int rcu_print_task_stall(struct rcu_node *rnp, unsigned long flags)
{
+ raw_spin_unlock_irqrestore_rcu_node(rnp, flags);
return 0;
}
#endif /* #else #ifdef CONFIG_PREEMPT_RCU */
pr_err("INFO: %s detected stalls on CPUs/tasks:\n", rcu_state.name);
rcu_for_each_leaf_node(rnp) {
raw_spin_lock_irqsave_rcu_node(rnp, flags);
- ndetected += rcu_print_task_stall(rnp);
if (rnp->qsmask != 0) {
for_each_leaf_node_possible_cpu(rnp, cpu)
if (rnp->qsmask & leaf_node_cpu_bit(rnp, cpu)) {
ndetected++;
}
}
- raw_spin_unlock_irqrestore_rcu_node(rnp, flags);
+ ndetected += rcu_print_task_stall(rnp, flags); // Releases rnp->lock.
}
for_each_possible_cpu(cpu)
break;
case 's':
- {
- int rc;
-
- if (isdigit(*(str+1))) {
- rc = kstrtoint(str+1, 0, &reboot_cpu);
- if (rc)
- return rc;
- } else if (str[1] == 'm' && str[2] == 'p' &&
- isdigit(*(str+3))) {
- rc = kstrtoint(str+3, 0, &reboot_cpu);
- if (rc)
- return rc;
- } else
+ if (isdigit(*(str+1)))
+ reboot_cpu = simple_strtoul(str+1, NULL, 0);
+ else if (str[1] == 'm' && str[2] == 'p' &&
+ isdigit(*(str+3)))
+ reboot_cpu = simple_strtoul(str+3, NULL, 0);
+ else
*mode = REBOOT_SOFT;
+ if (reboot_cpu >= num_possible_cpus()) {
+ pr_err("Ignoring the CPU number in reboot= option. "
+ "CPU %d exceeds possible cpu number %d\n",
+ reboot_cpu, num_possible_cpus());
+ reboot_cpu = 0;
+ break;
+ }
break;
- }
+
case 'g':
*mode = REBOOT_GPIO;
break;
#ifdef CONFIG_SMP
if (wake_flags & WF_MIGRATED)
en_flags |= ENQUEUE_MIGRATED;
+ else
#endif
+ if (p->in_iowait) {
+ delayacct_blkio_end(p);
+ atomic_dec(&task_rq(p)->nr_iowait);
+ }
activate_task(rq, p, en_flags);
ttwu_do_wakeup(rq, p, wake_flags, rf);
if (READ_ONCE(p->on_rq) && ttwu_runnable(p, wake_flags))
goto unlock;
- if (p->in_iowait) {
- delayacct_blkio_end(p);
- atomic_dec(&task_rq(p)->nr_iowait);
- }
-
#ifdef CONFIG_SMP
/*
* Ensure we load p->on_cpu _after_ p->on_rq, otherwise it would be
cpu = select_task_rq(p, p->wake_cpu, SD_BALANCE_WAKE, wake_flags);
if (task_cpu(p) != cpu) {
+ if (p->in_iowait) {
+ delayacct_blkio_end(p);
+ atomic_dec(&task_rq(p)->nr_iowait);
+ }
+
wake_flags |= WF_MIGRATED;
psi_ttwu_dequeue(p);
set_task_cpu(p, cpu);
if (!dl_prio(p->normal_prio) ||
(pi_task && dl_prio(pi_task->prio) &&
dl_entity_preempt(&pi_task->dl, &p->dl))) {
- p->dl.dl_boosted = 1;
+ p->dl.pi_se = pi_task->dl.pi_se;
queue_flag |= ENQUEUE_REPLENISH;
- } else
- p->dl.dl_boosted = 0;
+ } else {
+ p->dl.pi_se = &p->dl;
+ }
p->sched_class = &dl_sched_class;
} else if (rt_prio(prio)) {
if (dl_prio(oldprio))
- p->dl.dl_boosted = 0;
+ p->dl.pi_se = &p->dl;
if (oldprio < prio)
queue_flag |= ENQUEUE_HEAD;
p->sched_class = &rt_sched_class;
} else {
if (dl_prio(oldprio))
- p->dl.dl_boosted = 0;
+ p->dl.pi_se = &p->dl;
if (rt_prio(oldprio))
p->rt.timeout = 0;
p->sched_class = &fair_sched_class;
return !RB_EMPTY_NODE(&dl_se->rb_node);
}
+#ifdef CONFIG_RT_MUTEXES
+static inline struct sched_dl_entity *pi_of(struct sched_dl_entity *dl_se)
+{
+ return dl_se->pi_se;
+}
+
+static inline bool is_dl_boosted(struct sched_dl_entity *dl_se)
+{
+ return pi_of(dl_se) != dl_se;
+}
+#else
+static inline struct sched_dl_entity *pi_of(struct sched_dl_entity *dl_se)
+{
+ return dl_se;
+}
+
+static inline bool is_dl_boosted(struct sched_dl_entity *dl_se)
+{
+ return false;
+}
+#endif
+
#ifdef CONFIG_SMP
static inline struct dl_bw *dl_bw_of(int i)
{
struct dl_rq *dl_rq = dl_rq_of_se(dl_se);
struct rq *rq = rq_of_dl_rq(dl_rq);
- WARN_ON(dl_se->dl_boosted);
+ WARN_ON(is_dl_boosted(dl_se));
WARN_ON(dl_time_before(rq_clock(rq), dl_se->deadline));
/*
* could happen are, typically, a entity voluntarily trying to overcome its
* runtime, or it just underestimated it during sched_setattr().
*/
-static void replenish_dl_entity(struct sched_dl_entity *dl_se,
- struct sched_dl_entity *pi_se)
+static void replenish_dl_entity(struct sched_dl_entity *dl_se)
{
struct dl_rq *dl_rq = dl_rq_of_se(dl_se);
struct rq *rq = rq_of_dl_rq(dl_rq);
- BUG_ON(pi_se->dl_runtime <= 0);
+ BUG_ON(pi_of(dl_se)->dl_runtime <= 0);
/*
* This could be the case for a !-dl task that is boosted.
* Just go with full inherited parameters.
*/
if (dl_se->dl_deadline == 0) {
- dl_se->deadline = rq_clock(rq) + pi_se->dl_deadline;
- dl_se->runtime = pi_se->dl_runtime;
+ dl_se->deadline = rq_clock(rq) + pi_of(dl_se)->dl_deadline;
+ dl_se->runtime = pi_of(dl_se)->dl_runtime;
}
if (dl_se->dl_yielded && dl_se->runtime > 0)
* arbitrary large.
*/
while (dl_se->runtime <= 0) {
- dl_se->deadline += pi_se->dl_period;
- dl_se->runtime += pi_se->dl_runtime;
+ dl_se->deadline += pi_of(dl_se)->dl_period;
+ dl_se->runtime += pi_of(dl_se)->dl_runtime;
}
/*
*/
if (dl_time_before(dl_se->deadline, rq_clock(rq))) {
printk_deferred_once("sched: DL replenish lagged too much\n");
- dl_se->deadline = rq_clock(rq) + pi_se->dl_deadline;
- dl_se->runtime = pi_se->dl_runtime;
+ dl_se->deadline = rq_clock(rq) + pi_of(dl_se)->dl_deadline;
+ dl_se->runtime = pi_of(dl_se)->dl_runtime;
}
if (dl_se->dl_yielded)
* task with deadline equal to period this is the same of using
* dl_period instead of dl_deadline in the equation above.
*/
-static bool dl_entity_overflow(struct sched_dl_entity *dl_se,
- struct sched_dl_entity *pi_se, u64 t)
+static bool dl_entity_overflow(struct sched_dl_entity *dl_se, u64 t)
{
u64 left, right;
* of anything below microseconds resolution is actually fiction
* (but still we want to give the user that illusion >;).
*/
- left = (pi_se->dl_deadline >> DL_SCALE) * (dl_se->runtime >> DL_SCALE);
+ left = (pi_of(dl_se)->dl_deadline >> DL_SCALE) * (dl_se->runtime >> DL_SCALE);
right = ((dl_se->deadline - t) >> DL_SCALE) *
- (pi_se->dl_runtime >> DL_SCALE);
+ (pi_of(dl_se)->dl_runtime >> DL_SCALE);
return dl_time_before(right, left);
}
* Please refer to the comments update_dl_revised_wakeup() function to find
* more about the Revised CBS rule.
*/
-static void update_dl_entity(struct sched_dl_entity *dl_se,
- struct sched_dl_entity *pi_se)
+static void update_dl_entity(struct sched_dl_entity *dl_se)
{
struct dl_rq *dl_rq = dl_rq_of_se(dl_se);
struct rq *rq = rq_of_dl_rq(dl_rq);
if (dl_time_before(dl_se->deadline, rq_clock(rq)) ||
- dl_entity_overflow(dl_se, pi_se, rq_clock(rq))) {
+ dl_entity_overflow(dl_se, rq_clock(rq))) {
if (unlikely(!dl_is_implicit(dl_se) &&
!dl_time_before(dl_se->deadline, rq_clock(rq)) &&
- !dl_se->dl_boosted)){
+ !is_dl_boosted(dl_se))) {
update_dl_revised_wakeup(dl_se, rq);
return;
}
- dl_se->deadline = rq_clock(rq) + pi_se->dl_deadline;
- dl_se->runtime = pi_se->dl_runtime;
+ dl_se->deadline = rq_clock(rq) + pi_of(dl_se)->dl_deadline;
+ dl_se->runtime = pi_of(dl_se)->dl_runtime;
}
}
* The task might have been boosted by someone else and might be in the
* boosting/deboosting path, its not throttled.
*/
- if (dl_se->dl_boosted)
+ if (is_dl_boosted(dl_se))
goto unlock;
/*
* but do not enqueue -- wait for our wakeup to do that.
*/
if (!task_on_rq_queued(p)) {
- replenish_dl_entity(dl_se, dl_se);
+ replenish_dl_entity(dl_se);
goto unlock;
}
if (dl_time_before(dl_se->deadline, rq_clock(rq)) &&
dl_time_before(rq_clock(rq), dl_next_period(dl_se))) {
- if (unlikely(dl_se->dl_boosted || !start_dl_timer(p)))
+ if (unlikely(is_dl_boosted(dl_se) || !start_dl_timer(p)))
return;
dl_se->dl_throttled = 1;
if (dl_se->runtime > 0)
dl_se->dl_overrun = 1;
__dequeue_task_dl(rq, curr, 0);
- if (unlikely(dl_se->dl_boosted || !start_dl_timer(curr)))
+ if (unlikely(is_dl_boosted(dl_se) || !start_dl_timer(curr)))
enqueue_task_dl(rq, curr, ENQUEUE_REPLENISH);
if (!is_leftmost(curr, &rq->dl))
}
static void
-enqueue_dl_entity(struct sched_dl_entity *dl_se,
- struct sched_dl_entity *pi_se, int flags)
+enqueue_dl_entity(struct sched_dl_entity *dl_se, int flags)
{
BUG_ON(on_dl_rq(dl_se));
*/
if (flags & ENQUEUE_WAKEUP) {
task_contending(dl_se, flags);
- update_dl_entity(dl_se, pi_se);
+ update_dl_entity(dl_se);
} else if (flags & ENQUEUE_REPLENISH) {
- replenish_dl_entity(dl_se, pi_se);
+ replenish_dl_entity(dl_se);
} else if ((flags & ENQUEUE_RESTORE) &&
dl_time_before(dl_se->deadline,
rq_clock(rq_of_dl_rq(dl_rq_of_se(dl_se))))) {
static void enqueue_task_dl(struct rq *rq, struct task_struct *p, int flags)
{
- struct task_struct *pi_task = rt_mutex_get_top_task(p);
- struct sched_dl_entity *pi_se = &p->dl;
-
- /*
- * Use the scheduling parameters of the top pi-waiter task if:
- * - we have a top pi-waiter which is a SCHED_DEADLINE task AND
- * - our dl_boosted is set (i.e. the pi-waiter's (absolute) deadline is
- * smaller than our deadline OR we are a !SCHED_DEADLINE task getting
- * boosted due to a SCHED_DEADLINE pi-waiter).
- * Otherwise we keep our runtime and deadline.
- */
- if (pi_task && dl_prio(pi_task->normal_prio) && p->dl.dl_boosted) {
- pi_se = &pi_task->dl;
+ if (is_dl_boosted(&p->dl)) {
/*
* Because of delays in the detection of the overrun of a
* thread's runtime, it might be the case that a thread
* the throttle.
*/
p->dl.dl_throttled = 0;
- BUG_ON(!p->dl.dl_boosted || flags != ENQUEUE_REPLENISH);
+ BUG_ON(!is_dl_boosted(&p->dl) || flags != ENQUEUE_REPLENISH);
return;
}
return;
}
- enqueue_dl_entity(&p->dl, pi_se, flags);
+ enqueue_dl_entity(&p->dl, flags);
if (!task_current(rq, p) && p->nr_cpus_allowed > 1)
enqueue_pushable_dl_task(rq, p);
dl_se->dl_bw = 0;
dl_se->dl_density = 0;
- dl_se->dl_boosted = 0;
dl_se->dl_throttled = 0;
dl_se->dl_yielded = 0;
dl_se->dl_non_contending = 0;
dl_se->dl_overrun = 0;
+
+#ifdef CONFIG_RT_MUTEXES
+ dl_se->pi_se = dl_se;
+#endif
}
bool dl_param_changed(struct task_struct *p, const struct sched_attr *attr)
unsigned long flags = *(unsigned long *)table->data;
size_t data_size = 0;
size_t len = 0;
- char *tmp;
+ char *tmp, *buf;
int idx;
if (write)
return 0;
}
- tmp = kcalloc(data_size + 1, sizeof(*tmp), GFP_KERNEL);
- if (!tmp)
+ buf = kcalloc(data_size + 1, sizeof(*buf), GFP_KERNEL);
+ if (!buf)
return -ENOMEM;
for_each_set_bit(idx, &flags, __SD_FLAG_CNT) {
char *name = sd_flag_debug[idx].name;
- len += snprintf(tmp + len, strlen(name) + 2, "%s ", name);
+ len += snprintf(buf + len, strlen(name) + 2, "%s ", name);
}
- tmp += *ppos;
+ tmp = buf + *ppos;
len -= *ppos;
if (len > *lenp)
*lenp = len;
*ppos += len;
- kfree(tmp);
+ kfree(buf);
return 0;
}
struct cfs_rq *cfs_rq;
struct sched_entity *se = &p->se;
int idle_h_nr_running = task_has_idle_policy(p);
+ int task_new = !(flags & ENQUEUE_WAKEUP);
/*
* The code below (indirectly) updates schedutil which looks at
* into account, but that is not straightforward to implement,
* and the following generally works well enough in practice.
*/
- if (flags & ENQUEUE_WAKEUP)
+ if (!task_new)
update_overutilized_status(rq);
enqueue_throttle:
static int
select_idle_capacity(struct task_struct *p, struct sched_domain *sd, int target)
{
- unsigned long best_cap = 0;
+ unsigned long task_util, best_cap = 0;
int cpu, best_cpu = -1;
struct cpumask *cpus;
- sync_entity_load_avg(&p->se);
-
cpus = this_cpu_cpumask_var_ptr(select_idle_mask);
cpumask_and(cpus, sched_domain_span(sd), p->cpus_ptr);
+ task_util = uclamp_task_util(p);
+
for_each_cpu_wrap(cpu, cpus, target) {
unsigned long cpu_cap = capacity_of(cpu);
if (!available_idle_cpu(cpu) && !sched_idle_cpu(cpu))
continue;
- if (task_fits_capacity(p, cpu_cap))
+ if (fits_capacity(task_util, cpu_cap))
return cpu;
if (cpu_cap > best_cap) {
return best_cpu;
}
+static inline bool asym_fits_capacity(int task_util, int cpu)
+{
+ if (static_branch_unlikely(&sched_asym_cpucapacity))
+ return fits_capacity(task_util, capacity_of(cpu));
+
+ return true;
+}
+
/*
* Try and locate an idle core/thread in the LLC cache domain.
*/
static int select_idle_sibling(struct task_struct *p, int prev, int target)
{
struct sched_domain *sd;
+ unsigned long task_util;
int i, recent_used_cpu;
/*
- * For asymmetric CPU capacity systems, our domain of interest is
- * sd_asym_cpucapacity rather than sd_llc.
+ * On asymmetric system, update task utilization because we will check
+ * that the task fits with cpu's capacity.
*/
if (static_branch_unlikely(&sched_asym_cpucapacity)) {
- sd = rcu_dereference(per_cpu(sd_asym_cpucapacity, target));
- /*
- * On an asymmetric CPU capacity system where an exclusive
- * cpuset defines a symmetric island (i.e. one unique
- * capacity_orig value through the cpuset), the key will be set
- * but the CPUs within that cpuset will not have a domain with
- * SD_ASYM_CPUCAPACITY. These should follow the usual symmetric
- * capacity path.
- */
- if (!sd)
- goto symmetric;
-
- i = select_idle_capacity(p, sd, target);
- return ((unsigned)i < nr_cpumask_bits) ? i : target;
+ sync_entity_load_avg(&p->se);
+ task_util = uclamp_task_util(p);
}
-symmetric:
- if (available_idle_cpu(target) || sched_idle_cpu(target))
+ if ((available_idle_cpu(target) || sched_idle_cpu(target)) &&
+ asym_fits_capacity(task_util, target))
return target;
/*
* If the previous CPU is cache affine and idle, don't be stupid:
*/
if (prev != target && cpus_share_cache(prev, target) &&
- (available_idle_cpu(prev) || sched_idle_cpu(prev)))
+ (available_idle_cpu(prev) || sched_idle_cpu(prev)) &&
+ asym_fits_capacity(task_util, prev))
return prev;
/*
recent_used_cpu != target &&
cpus_share_cache(recent_used_cpu, target) &&
(available_idle_cpu(recent_used_cpu) || sched_idle_cpu(recent_used_cpu)) &&
- cpumask_test_cpu(p->recent_used_cpu, p->cpus_ptr)) {
+ cpumask_test_cpu(p->recent_used_cpu, p->cpus_ptr) &&
+ asym_fits_capacity(task_util, recent_used_cpu)) {
/*
* Replace recent_used_cpu with prev as it is a potential
* candidate for the next wake:
return recent_used_cpu;
}
+ /*
+ * For asymmetric CPU capacity systems, our domain of interest is
+ * sd_asym_cpucapacity rather than sd_llc.
+ */
+ if (static_branch_unlikely(&sched_asym_cpucapacity)) {
+ sd = rcu_dereference(per_cpu(sd_asym_cpucapacity, target));
+ /*
+ * On an asymmetric CPU capacity system where an exclusive
+ * cpuset defines a symmetric island (i.e. one unique
+ * capacity_orig value through the cpuset), the key will be set
+ * but the CPUs within that cpuset will not have a domain with
+ * SD_ASYM_CPUCAPACITY. These should follow the usual symmetric
+ * capacity path.
+ */
+ if (sd) {
+ i = select_idle_capacity(p, sd, target);
+ return ((unsigned)i < nr_cpumask_bits) ? i : target;
+ }
+ }
+
sd = rcu_dereference(per_cpu(sd_llc, target));
if (!sd)
return target;
* emptying busiest.
*/
if (local->group_type == group_has_spare) {
- if (busiest->group_type > group_fully_busy) {
+ if ((busiest->group_type > group_fully_busy) &&
+ !(env->sd->flags & SD_SHARE_PKG_RESOURCES)) {
/*
* If busiest is overloaded, try to fill spare
* capacity. This might end up creating spare capacity
void __weak arch_cpu_idle(void)
{
cpu_idle_force_poll = 1;
- local_irq_enable();
+ raw_local_irq_enable();
}
/**
trace_cpu_idle(1, smp_processor_id());
stop_critical_timings();
+
+ /*
+ * arch_cpu_idle() is supposed to enable IRQs, however
+ * we can't do that because of RCU and tracing.
+ *
+ * Trace IRQs enable here, then switch off RCU, and have
+ * arch_cpu_idle() use raw_local_irq_enable(). Note that
+ * rcu_idle_enter() relies on lockdep IRQ state, so switch that
+ * last -- this is very similar to the entry code.
+ */
+ trace_hardirqs_on_prepare();
+ lockdep_hardirqs_on_prepare(_THIS_IP_);
rcu_idle_enter();
+ lockdep_hardirqs_on(_THIS_IP_);
+
arch_cpu_idle();
+
+ /*
+ * OK, so IRQs are enabled here, but RCU needs them disabled to
+ * turn itself back on.. funny thing is that disabling IRQs
+ * will cause tracing, which needs RCU. Jump through hoops to
+ * make it 'work'.
+ */
+ raw_local_irq_disable();
+ lockdep_hardirqs_off(_THIS_IP_);
rcu_idle_exit();
+ lockdep_hardirqs_on(_THIS_IP_);
+ raw_local_irq_enable();
+
start_critical_timings();
trace_cpu_idle(PWR_EVENT_EXIT, smp_processor_id());
}
#include <linux/filter.h>
#include <linux/pid.h>
#include <linux/ptrace.h>
-#include <linux/security.h>
+#include <linux/capability.h>
#include <linux/tracehook.h>
#include <linux/uaccess.h>
#include <linux/anon_inodes.h>
* behavior of privileged children.
*/
if (!task_no_new_privs(current) &&
- security_capable(current_cred(), current_user_ns(),
- CAP_SYS_ADMIN, CAP_OPT_NOAUDIT) != 0)
+ !ns_capable_noaudit(current_user_ns(), CAP_SYS_ADMIN))
return ERR_PTR(-EACCES);
/* Allocate a new seccomp_filter */
data = kmalloc(len, GFP_KERNEL);
if (!data)
return -ENOMEM;
- nla_strlcpy(data, na, len);
+ nla_strscpy(data, na, len);
ret = cpulist_parse(data, mask);
kfree(data);
return ret;
config DYNAMIC_FTRACE_WITH_DIRECT_CALLS
def_bool y
- depends on DYNAMIC_FTRACE
+ depends on DYNAMIC_FTRACE_WITH_REGS
depends on HAVE_DYNAMIC_FTRACE_WITH_DIRECT_CALLS
config FUNCTION_PROFILER
{
int ret;
+ /*
+ * NB: We rely on strncpy_from_user() not copying junk past the NUL
+ * terminator into `dst`.
+ *
+ * strncpy_from_user() does long-sized strides in the fast path. If the
+ * strncpy does not mask out the bytes after the NUL in `unsafe_ptr`,
+ * then there could be junk after the NUL in `dst`. If user takes `dst`
+ * and keys a hash map with it, then semantically identical strings can
+ * occupy multiple entries in the map.
+ */
ret = strncpy_from_user_nofault(dst, unsafe_ptr, size);
if (unlikely(ret < 0))
memset(dst, 0, size);
*btf = bpf_get_btf_vmlinux();
if (IS_ERR_OR_NULL(*btf))
- return PTR_ERR(*btf);
+ return IS_ERR(*btf) ? PTR_ERR(*btf) : -EINVAL;
if (ptr->type_id > 0)
*btf_id = ptr->type_id;
static struct ftrace_ops *
ftrace_find_tramp_ops_any(struct dyn_ftrace *rec);
static struct ftrace_ops *
+ftrace_find_tramp_ops_any_other(struct dyn_ftrace *rec, struct ftrace_ops *op_exclude);
+static struct ftrace_ops *
ftrace_find_tramp_ops_next(struct dyn_ftrace *rec, struct ftrace_ops *ops);
static bool __ftrace_hash_rec_update(struct ftrace_ops *ops,
* to it.
*/
if (ftrace_rec_count(rec) == 1 &&
- ftrace_find_tramp_ops_any(rec))
+ ftrace_find_tramp_ops_any_other(rec, ops))
rec->flags |= FTRACE_FL_TRAMP;
else
rec->flags &= ~FTRACE_FL_TRAMP;
return NULL;
}
+static struct ftrace_ops *
+ftrace_find_tramp_ops_any_other(struct dyn_ftrace *rec, struct ftrace_ops *op_exclude)
+{
+ struct ftrace_ops *op;
+ unsigned long ip = rec->ip;
+
+ do_for_each_ftrace_op(op, ftrace_ops_list) {
+
+ if (op == op_exclude || !op->trampoline)
+ continue;
+
+ if (hash_contains_ip(ip, op->func_hash))
+ return op;
+ } while_for_each_ftrace_op(op);
+
+ return NULL;
+}
+
static struct ftrace_ops *
ftrace_find_tramp_ops_next(struct dyn_ftrace *rec,
struct ftrace_ops *op)
/* See if we shot pass the end of this buffer page */
if (unlikely(write > BUF_PAGE_SIZE)) {
- if (tail != w) {
- /* before and after may now different, fix it up*/
- b_ok = rb_time_read(&cpu_buffer->before_stamp, &info->before);
- a_ok = rb_time_read(&cpu_buffer->write_stamp, &info->after);
- if (a_ok && b_ok && info->before != info->after)
- (void)rb_time_cmpxchg(&cpu_buffer->before_stamp,
- info->before, info->after);
- }
+ /* before and after may now different, fix it up*/
+ b_ok = rb_time_read(&cpu_buffer->before_stamp, &info->before);
+ a_ok = rb_time_read(&cpu_buffer->write_stamp, &info->after);
+ if (a_ok && b_ok && info->before != info->after)
+ (void)rb_time_cmpxchg(&cpu_buffer->before_stamp,
+ info->before, info->after);
return rb_move_tail(cpu_buffer, tail, info);
}
ts = rb_time_stamp(cpu_buffer->buffer);
barrier();
/*E*/ if (write == (local_read(&tail_page->write) & RB_WRITE_MASK) &&
- info->after < ts) {
+ info->after < ts &&
+ rb_time_cmpxchg(&cpu_buffer->write_stamp,
+ info->after, ts)) {
/* Nothing came after this event between C and E */
info->delta = ts - info->after;
- (void)rb_time_cmpxchg(&cpu_buffer->write_stamp,
- info->after, info->ts);
info->ts = ts;
} else {
/*
}
#define STATIC_TEMP_BUF_SIZE 128
-static char static_temp_buf[STATIC_TEMP_BUF_SIZE];
+static char static_temp_buf[STATIC_TEMP_BUF_SIZE] __aligned(4);
/* Find the next real entry, without updating the iterator itself */
struct trace_entry *trace_find_next_entry(struct trace_iterator *iter,
struct task_struct *kthread;
int next_cpu;
- if (WARN_ON(hwlat_kthread))
+ if (hwlat_kthread)
return 0;
/* Just pick the first CPU on first iteration */
int __read_mostly watchdog_thresh = 10;
static int __read_mostly nmi_watchdog_available;
-static struct cpumask watchdog_allowed_mask __read_mostly;
-
struct cpumask watchdog_cpumask __read_mostly;
unsigned long *watchdog_cpumask_bits = cpumask_bits(&watchdog_cpumask);
int __read_mostly sysctl_softlockup_all_cpu_backtrace;
#endif
+static struct cpumask watchdog_allowed_mask __read_mostly;
+
/* Global variables, exported for sysctl */
unsigned int __read_mostly softlockup_panic =
CONFIG_BOOTPARAM_SOFTLOCKUP_PANIC_VALUE;
depends on CC_HAS_SANCOV_TRACE_PC || GCC_PLUGINS
select DEBUG_FS
select GCC_PLUGIN_SANCOV if !CC_HAS_SANCOV_TRACE_PC
- select SKB_EXTENSIONS if NET
help
KCOV exposes kernel code coverage information in a form suitable
for coverage-guided fuzzing (randomized testing).
EXPORT_SYMBOL(nla_find);
/**
- * nla_strlcpy - Copy string attribute payload into a sized buffer
- * @dst: where to copy the string to
- * @nla: attribute to copy the string from
- * @dstsize: size of destination buffer
+ * nla_strscpy - Copy string attribute payload into a sized buffer
+ * @dst: Where to copy the string to.
+ * @nla: Attribute to copy the string from.
+ * @dstsize: Size of destination buffer.
*
* Copies at most dstsize - 1 bytes into the destination buffer.
- * The result is always a valid NUL-terminated string. Unlike
- * strlcpy the destination buffer is always padded out.
+ * Unlike strlcpy the destination buffer is always padded out.
*
- * Returns the length of the source buffer.
+ * Return:
+ * * srclen - Returns @nla length (not including the trailing %NUL).
+ * * -E2BIG - If @dstsize is 0 or greater than U16_MAX or @nla length greater
+ * than @dstsize.
*/
-size_t nla_strlcpy(char *dst, const struct nlattr *nla, size_t dstsize)
+ssize_t nla_strscpy(char *dst, const struct nlattr *nla, size_t dstsize)
{
size_t srclen = nla_len(nla);
char *src = nla_data(nla);
+ ssize_t ret;
+ size_t len;
+
+ if (dstsize == 0 || WARN_ON_ONCE(dstsize > U16_MAX))
+ return -E2BIG;
if (srclen > 0 && src[srclen - 1] == '\0')
srclen--;
- if (dstsize > 0) {
- size_t len = (srclen >= dstsize) ? dstsize - 1 : srclen;
-
- memset(dst, 0, dstsize);
- memcpy(dst, src, len);
+ if (srclen >= dstsize) {
+ len = dstsize - 1;
+ ret = -E2BIG;
+ } else {
+ len = srclen;
+ ret = len;
}
- return srclen;
+ memcpy(dst, src, len);
+ /* Zero pad end of dst. */
+ memset(dst + len, 0, dstsize - len);
+
+ return ret;
}
-EXPORT_SYMBOL(nla_strlcpy);
+EXPORT_SYMBOL(nla_strscpy);
/**
* nla_strdup - Copy string attribute payload into a newly allocated buffer
goto byte_at_a_time;
while (max >= sizeof(unsigned long)) {
- unsigned long c, data;
+ unsigned long c, data, mask;
/* Fall back to byte-at-a-time if we get a page fault */
unsafe_get_user(c, (unsigned long __user *)(src+res), byte_at_a_time);
- *(unsigned long *)(dst+res) = c;
+ /*
+ * Note that we mask out the bytes following the NUL. This is
+ * important to do because string oblivious code may read past
+ * the NUL. For those routines, we don't want to give them
+ * potentially random bytes after the NUL in `src`.
+ *
+ * One example of such code is BPF map keys. BPF treats map keys
+ * as an opaque set of bytes. Without the post-NUL mask, any BPF
+ * maps keyed by strings returned from strncpy_from_user() may
+ * have multiple entries for semantically identical strings.
+ */
if (has_zero(c, &data, &constants)) {
data = prep_zero_mask(c, data, &constants);
data = create_zero_mask(data);
+ mask = zero_bytemask(data);
+ *(unsigned long *)(dst+res) = c & mask;
return res + find_zero(data);
}
+
+ *(unsigned long *)(dst+res) = c;
+
res += sizeof(unsigned long);
max -= sizeof(unsigned long);
}
static int collect_syscall(struct task_struct *target, struct syscall_info *info)
{
+ unsigned long args[6] = { };
struct pt_regs *regs;
if (!try_get_task_stack(target)) {
info->data.nr = syscall_get_nr(target, regs);
if (info->data.nr != -1L)
- syscall_get_arguments(target, regs,
- (unsigned long *)&info->data.args[0]);
+ syscall_get_arguments(target, regs, args);
+
+ info->data.args[0] = args[0];
+ info->data.args[1] = args[1];
+ info->data.args[2] = args[2];
+ info->data.args[3] = args[3];
+ info->data.args[4] = args[4];
+ info->data.args[5] = args[5];
put_task_stack(target);
return 0;
* delay for some time until fewer pages are isolated
*/
while (unlikely(too_many_isolated(pgdat))) {
+ /* stop isolation if there are still pages not migrated */
+ if (cc->nr_migratepages)
+ return 0;
+
/* async migration should just abort */
if (cc->mode == MIGRATE_ASYNC)
return 0;
isolate_success:
list_add(&page->lru, &cc->migratepages);
- cc->nr_migratepages++;
- nr_isolated++;
+ cc->nr_migratepages += compound_nr(page);
+ nr_isolated += compound_nr(page);
/*
* Avoid isolating too much unless this block is being
* or a lock is contended. For contention, isolate quickly to
* potentially remove one source of contention.
*/
- if (cc->nr_migratepages == COMPACT_CLUSTER_MAX &&
+ if (cc->nr_migratepages >= COMPACT_CLUSTER_MAX &&
!cc->rescan && !cc->contended) {
++low_pfn;
break;
if (!pfn)
break;
- if (cc->nr_migratepages == COMPACT_CLUSTER_MAX)
+ if (cc->nr_migratepages >= COMPACT_CLUSTER_MAX)
break;
}
rotate_reclaimable_page(page);
}
+ /*
+ * Writeback does not hold a page reference of its own, relying
+ * on truncation to wait for the clearing of PG_writeback.
+ * But here we must make sure that the page is not freed and
+ * reused before the wake_up_page().
+ */
+ get_page(page);
if (!test_clear_page_writeback(page))
BUG();
smp_mb__after_atomic();
wake_up_page(page, PG_writeback);
+ put_page(page);
}
EXPORT_SYMBOL(end_page_writeback);
page_not_up_to_date:
/* Get exclusive access to the page ... */
- if (iocb->ki_flags & IOCB_WAITQ)
+ if (iocb->ki_flags & IOCB_WAITQ) {
+ if (written) {
+ put_page(page);
+ goto out;
+ }
error = lock_page_async(page, iocb->ki_waitq);
- else
+ } else {
error = lock_page_killable(page);
+ }
if (unlikely(error))
goto readpage_error;
}
if (!PageUptodate(page)) {
- if (iocb->ki_flags & IOCB_WAITQ)
+ if (iocb->ki_flags & IOCB_WAITQ) {
+ if (written) {
+ put_page(page);
+ goto out;
+ }
error = lock_page_async(page, iocb->ki_waitq);
- else
+ } else {
error = lock_page_killable(page);
+ }
if (unlikely(error))
goto readpage_error;
/*
* drop the above get_user_pages reference.
*/
- for (i = 0; i < nr_pages; i++)
- put_page(pages[i]);
+ if (gup_flags & FOLL_PIN)
+ unpin_user_pages(pages, nr_pages);
+ else
+ for (i = 0; i < nr_pages; i++)
+ put_page(pages[i]);
if (migrate_pages(&cma_page_list, alloc_migration_target, NULL,
(unsigned long)&mtc, MIGRATE_SYNC, MR_CONTIG_RANGE)) {
goto out;
if (check_dax_vmas(vmas_tmp, rc)) {
- for (i = 0; i < rc; i++)
- put_page(pages[i]);
+ if (gup_flags & FOLL_PIN)
+ unpin_user_pages(pages, rc);
+ else
+ for (i = 0; i < rc; i++)
+ put_page(pages[i]);
rc = -EOPNOTSUPP;
goto out;
}
transparent_hugepage_use_zero_page()) {
pgtable_t pgtable;
struct page *zero_page;
- bool set;
vm_fault_t ret;
pgtable = pte_alloc_one(vma->vm_mm);
if (unlikely(!pgtable))
}
vmf->ptl = pmd_lock(vma->vm_mm, vmf->pmd);
ret = 0;
- set = false;
if (pmd_none(*vmf->pmd)) {
ret = check_stable_address_space(vma->vm_mm);
if (ret) {
spin_unlock(vmf->ptl);
+ pte_free(vma->vm_mm, pgtable);
} else if (userfaultfd_missing(vma)) {
spin_unlock(vmf->ptl);
+ pte_free(vma->vm_mm, pgtable);
ret = handle_userfault(vmf, VM_UFFD_MISSING);
VM_BUG_ON(ret & VM_FAULT_FALLBACK);
} else {
set_huge_zero_page(pgtable, vma->vm_mm, vma,
haddr, vmf->pmd, zero_page);
spin_unlock(vmf->ptl);
- set = true;
}
- } else
+ } else {
spin_unlock(vmf->ptl);
- if (!set)
pte_free(vma->vm_mm, pgtable);
+ }
return ret;
}
gfp = alloc_hugepage_direct_gfpmask(vma);
return page_head[1].compound_dtor == HUGETLB_PAGE_DTOR;
}
-/*
- * Find address_space associated with hugetlbfs page.
- * Upon entry page is locked and page 'was' mapped although mapped state
- * could change. If necessary, use anon_vma to find vma and associated
- * address space. The returned mapping may be stale, but it can not be
- * invalid as page lock (which is held) is required to destroy mapping.
- */
-static struct address_space *_get_hugetlb_page_mapping(struct page *hpage)
-{
- struct anon_vma *anon_vma;
- pgoff_t pgoff_start, pgoff_end;
- struct anon_vma_chain *avc;
- struct address_space *mapping = page_mapping(hpage);
-
- /* Simple file based mapping */
- if (mapping)
- return mapping;
-
- /*
- * Even anonymous hugetlbfs mappings are associated with an
- * underlying hugetlbfs file (see hugetlb_file_setup in mmap
- * code). Find a vma associated with the anonymous vma, and
- * use the file pointer to get address_space.
- */
- anon_vma = page_lock_anon_vma_read(hpage);
- if (!anon_vma)
- return mapping; /* NULL */
-
- /* Use first found vma */
- pgoff_start = page_to_pgoff(hpage);
- pgoff_end = pgoff_start + pages_per_huge_page(page_hstate(hpage)) - 1;
- anon_vma_interval_tree_foreach(avc, &anon_vma->rb_root,
- pgoff_start, pgoff_end) {
- struct vm_area_struct *vma = avc->vma;
-
- mapping = vma->vm_file->f_mapping;
- break;
- }
-
- anon_vma_unlock_read(anon_vma);
- return mapping;
-}
-
/*
* Find and lock address space (mapping) in write mode.
*
- * Upon entry, the page is locked which allows us to find the mapping
- * even in the case of an anon page. However, locking order dictates
- * the i_mmap_rwsem be acquired BEFORE the page lock. This is hugetlbfs
- * specific. So, we first try to lock the sema while still holding the
- * page lock. If this works, great! If not, then we need to drop the
- * page lock and then acquire i_mmap_rwsem and reacquire page lock. Of
- * course, need to revalidate state along the way.
+ * Upon entry, the page is locked which means that page_mapping() is
+ * stable. Due to locking order, we can only trylock_write. If we can
+ * not get the lock, simply return NULL to caller.
*/
struct address_space *hugetlb_page_mapping_lock_write(struct page *hpage)
{
- struct address_space *mapping, *mapping2;
+ struct address_space *mapping = page_mapping(hpage);
- mapping = _get_hugetlb_page_mapping(hpage);
-retry:
if (!mapping)
return mapping;
- /*
- * If no contention, take lock and return
- */
if (i_mmap_trylock_write(mapping))
return mapping;
- /*
- * Must drop page lock and wait on mapping sema.
- * Note: Once page lock is dropped, mapping could become invalid.
- * As a hack, increase map count until we lock page again.
- */
- atomic_inc(&hpage->_mapcount);
- unlock_page(hpage);
- i_mmap_lock_write(mapping);
- lock_page(hpage);
- atomic_add_negative(-1, &hpage->_mapcount);
-
- /* verify page is still mapped */
- if (!page_mapped(hpage)) {
- i_mmap_unlock_write(mapping);
- return NULL;
- }
-
- /*
- * Get address space again and verify it is the same one
- * we locked. If not, drop lock and retry.
- */
- mapping2 = _get_hugetlb_page_mapping(hpage);
- if (mapping2 != mapping) {
- i_mmap_unlock_write(mapping);
- mapping = mapping2;
- goto retry;
- }
-
- return mapping;
+ return NULL;
}
pgoff_t __basepage_index(struct page *page)
struct address_space *mapping)
{
XA_STATE(xas, &mapping->i_pages, linear_page_index(vma, start));
- pgoff_t end_index = end / PAGE_SIZE;
+ pgoff_t end_index = linear_page_index(vma, end + PAGE_SIZE - 1);
struct page *page;
rcu_read_lock();
ret = total_len - iov_iter_count(&iter);
mmput(mm);
- return ret;
-
release_task:
put_task_struct(task);
put_pid:
rcu_read_lock();
memcg = mem_cgroup_from_obj(p);
- /* Untracked pages have no memcg, no lruvec. Update only the node */
- if (!memcg || memcg == root_mem_cgroup) {
+ /*
+ * Untracked pages have no memcg, no lruvec. Update only the
+ * node. If we reparent the slab objects to the root memcg,
+ * when we free the slab object, we need to update the per-memcg
+ * vmstats to keep it correct for the root memcg.
+ */
+ if (!memcg) {
__mod_node_page_state(pgdat, idx, val);
} else {
lruvec = mem_cgroup_lruvec(memcg, pgdat);
if (!PageHuge(hpage)) {
unmap_success = try_to_unmap(hpage, ttu);
} else {
- /*
- * For hugetlb pages, try_to_unmap could potentially call
- * huge_pmd_unshare. Because of this, take semaphore in
- * write mode here and set TTU_RMAP_LOCKED to indicate we
- * have taken the lock at this higer level.
- *
- * Note that the call to hugetlb_page_mapping_lock_write
- * is necessary even if mapping is already set. It handles
- * ugliness of potentially having to drop page lock to obtain
- * i_mmap_rwsem.
- */
- mapping = hugetlb_page_mapping_lock_write(hpage);
-
- if (mapping) {
- unmap_success = try_to_unmap(hpage,
+ if (!PageAnon(hpage)) {
+ /*
+ * For hugetlb pages in shared mappings, try_to_unmap
+ * could potentially call huge_pmd_unshare. Because of
+ * this, take semaphore in write mode here and set
+ * TTU_RMAP_LOCKED to indicate we have taken the lock
+ * at this higer level.
+ */
+ mapping = hugetlb_page_mapping_lock_write(hpage);
+ if (mapping) {
+ unmap_success = try_to_unmap(hpage,
ttu|TTU_RMAP_LOCKED);
- i_mmap_unlock_write(mapping);
+ i_mmap_unlock_write(mapping);
+ } else {
+ pr_info("Memory failure: %#lx: could not lock mapping for mapped huge page\n", pfn);
+ unmap_success = false;
+ }
} else {
- pr_info("Memory failure: %#lx: could not find mapping for mapped huge page\n",
- pfn);
- unmap_success = false;
+ unmap_success = try_to_unmap(hpage, ttu);
}
}
if (!unmap_success)
return err;
}
-#ifdef CONFIG_NUMA
-int __weak memory_add_physaddr_to_nid(u64 start)
-{
- pr_info_once("Unknown online node for memory at 0x%llx, assuming node 0\n",
- start);
- return 0;
-}
-EXPORT_SYMBOL_GPL(memory_add_physaddr_to_nid);
-
-int __weak phys_to_target_node(u64 start)
-{
- pr_info_once("Unknown target node for memory at 0x%llx, assuming node 0\n",
- start);
- return 0;
-}
-EXPORT_SYMBOL_GPL(phys_to_target_node);
-#endif
-
/* find the smallest valid pfn in the range [start_pfn, end_pfn) */
static unsigned long find_smallest_section_pfn(int nid, struct zone *zone,
unsigned long start_pfn,
goto put_anon;
if (page_mapped(hpage)) {
- /*
- * try_to_unmap could potentially call huge_pmd_unshare.
- * Because of this, take semaphore in write mode here and
- * set TTU_RMAP_LOCKED to let lower levels know we have
- * taken the lock.
- */
- mapping = hugetlb_page_mapping_lock_write(hpage);
- if (unlikely(!mapping))
- goto unlock_put_anon;
+ bool mapping_locked = false;
+ enum ttu_flags ttu = TTU_MIGRATION|TTU_IGNORE_MLOCK|
+ TTU_IGNORE_ACCESS;
+
+ if (!PageAnon(hpage)) {
+ /*
+ * In shared mappings, try_to_unmap could potentially
+ * call huge_pmd_unshare. Because of this, take
+ * semaphore in write mode here and set TTU_RMAP_LOCKED
+ * to let lower levels know we have taken the lock.
+ */
+ mapping = hugetlb_page_mapping_lock_write(hpage);
+ if (unlikely(!mapping))
+ goto unlock_put_anon;
+
+ mapping_locked = true;
+ ttu |= TTU_RMAP_LOCKED;
+ }
- try_to_unmap(hpage,
- TTU_MIGRATION|TTU_IGNORE_MLOCK|TTU_IGNORE_ACCESS|
- TTU_RMAP_LOCKED);
+ try_to_unmap(hpage, ttu);
page_was_mapped = 1;
- /*
- * Leave mapping locked until after subsequent call to
- * remove_migration_ptes()
- */
+
+ if (mapping_locked)
+ i_mmap_unlock_write(mapping);
}
if (!page_mapped(hpage))
rc = move_to_new_page(new_hpage, hpage, mode);
- if (page_was_mapped) {
+ if (page_was_mapped)
remove_migration_ptes(hpage,
- rc == MIGRATEPAGE_SUCCESS ? new_hpage : hpage, true);
- i_mmap_unlock_write(mapping);
- }
+ rc == MIGRATEPAGE_SUCCESS ? new_hpage : hpage, false);
unlock_put_anon:
unlock_page(new_hpage);
} else {
ret = TestClearPageWriteback(page);
}
- /*
- * NOTE: Page might be free now! Writeback doesn't hold a page
- * reference on its own, it relies on truncation to wait for
- * the clearing of PG_writeback. The below can only access
- * page state that is static across allocation cycles.
- */
if (ret) {
dec_lruvec_state(lruvec, NR_WRITEBACK);
dec_zone_page_state(page, NR_ZONE_WRITE_PENDING);
if (!page_ref_sub_and_test(page, nc->pagecnt_bias))
goto refill;
+ if (unlikely(nc->pfmemalloc)) {
+ free_the_page(page, compound_order(page));
+ goto refill;
+ }
+
#if (PAGE_SIZE < PAGE_FRAG_CACHE_MAX_SIZE)
/* if size can vary use size else just use PAGE_SIZE */
size = nc->size;
region_size = ALIGN(start_offset + map_size, lcm_align);
/* allocate chunk */
- alloc_size = sizeof(struct pcpu_chunk) +
- BITS_TO_LONGS(region_size >> PAGE_SHIFT) * sizeof(unsigned long);
+ alloc_size = struct_size(chunk, populated,
+ BITS_TO_LONGS(region_size >> PAGE_SHIFT));
chunk = memblock_alloc(alloc_size, SMP_CACHE_BYTES);
if (!chunk)
panic("%s: Failed to allocate %zu bytes\n", __func__,
pcpu_unit_pages = ai->unit_size >> PAGE_SHIFT;
pcpu_unit_size = pcpu_unit_pages << PAGE_SHIFT;
pcpu_atom_size = ai->atom_size;
- pcpu_chunk_struct_size = sizeof(struct pcpu_chunk) +
- BITS_TO_LONGS(pcpu_unit_pages) * sizeof(unsigned long);
+ pcpu_chunk_struct_size = struct_size(chunk, populated,
+ BITS_TO_LONGS(pcpu_unit_pages));
pcpu_stats_save_ai(ai);
/*
* If sharing is possible, start and end will be adjusted
* accordingly.
- *
- * If called for a huge page, caller must hold i_mmap_rwsem
- * in write mode as it is possible to call huge_pmd_unshare.
*/
adjust_range_if_pmd_sharing_possible(vma, &range.start,
&range.end);
subpage = page - page_to_pfn(page) + pte_pfn(*pvmw.pte);
address = pvmw.address;
- if (PageHuge(page)) {
+ if (PageHuge(page) && !PageAnon(page)) {
/*
* To call huge_pmd_unshare, i_mmap_rwsem must be
* held in write mode. Caller needs to explicitly
object = c->freelist;
page = c->page;
- if (unlikely(!object || !node_match(page, node))) {
+ if (unlikely(!object || !page || !node_match(page, node))) {
object = __slab_alloc(s, gfpflags, node, addr, c);
} else {
void *next_object = get_freepointer_safe(s, object);
nr_reclaimed = shrink_page_list(&clean_pages, zone->zone_pgdat, &sc,
TTU_IGNORE_ACCESS, &stat, true);
list_splice(&clean_pages, page_list);
- mod_node_page_state(zone->zone_pgdat, NR_ISOLATED_FILE, -nr_reclaimed);
+ mod_node_page_state(zone->zone_pgdat, NR_ISOLATED_FILE,
+ -(long)nr_reclaimed);
/*
* Since lazyfree pages are isolated from file LRU from the beginning,
* they will rotate back to anonymous LRU in the end if it failed to
mod_node_page_state(zone->zone_pgdat, NR_ISOLATED_ANON,
stat.nr_lazyfree_fail);
mod_node_page_state(zone->zone_pgdat, NR_ISOLATED_FILE,
- -stat.nr_lazyfree_fail);
+ -(long)stat.nr_lazyfree_fail);
return nr_reclaimed;
}
kfree_skb(skb);
return -EADDRNOTAVAIL;
}
+ if (vcc->dev->ops->send_bh)
+ return vcc->dev->ops->send_bh(vcc, skb);
return vcc->dev->ops->send(vcc, skb);
}
vcc->push = atm_push_raw;
vcc->pop = atm_pop_raw;
vcc->push_oam = NULL;
- vcc->send = vcc->dev->ops->send;
+ if (vcc->dev->ops->send_bh)
+ vcc->send = vcc->dev->ops->send_bh;
+ else
+ vcc->send = vcc->dev->ops->send;
return 0;
}
vcc->push = atm_push_raw;
vcc->pop = atm_pop_raw;
vcc->push_oam = NULL;
- vcc->send = vcc->dev->ops->send;
+ if (vcc->dev->ops->send_bh)
+ vcc->send = vcc->dev->ops->send_bh;
+ else
+ vcc->send = vcc->dev->ops->send;
return 0;
}
EXPORT_SYMBOL(atm_init_aal5);
/**
* batadv_frag_create() - create a fragment from skb
+ * @net_dev: outgoing device for fragment
* @skb: skb to create fragment from
* @frag_head: header to use in new fragment
* @fragment_size: size of new fragment
*
* Return: the new fragment, NULL on error.
*/
-static struct sk_buff *batadv_frag_create(struct sk_buff *skb,
+static struct sk_buff *batadv_frag_create(struct net_device *net_dev,
+ struct sk_buff *skb,
struct batadv_frag_packet *frag_head,
unsigned int fragment_size)
{
+ unsigned int ll_reserved = LL_RESERVED_SPACE(net_dev);
+ unsigned int tailroom = net_dev->needed_tailroom;
struct sk_buff *skb_fragment;
unsigned int header_size = sizeof(*frag_head);
unsigned int mtu = fragment_size + header_size;
- skb_fragment = netdev_alloc_skb(NULL, mtu + ETH_HLEN);
+ skb_fragment = dev_alloc_skb(ll_reserved + mtu + tailroom);
if (!skb_fragment)
goto err;
skb_fragment->priority = skb->priority;
/* Eat the last mtu-bytes of the skb */
- skb_reserve(skb_fragment, header_size + ETH_HLEN);
+ skb_reserve(skb_fragment, ll_reserved + header_size);
skb_split(skb, skb_fragment, skb->len - fragment_size);
/* Add the header */
struct batadv_orig_node *orig_node,
struct batadv_neigh_node *neigh_node)
{
+ struct net_device *net_dev = neigh_node->if_incoming->net_dev;
struct batadv_priv *bat_priv;
struct batadv_hard_iface *primary_if = NULL;
struct batadv_frag_packet frag_header;
struct sk_buff *skb_fragment;
- unsigned int mtu = neigh_node->if_incoming->net_dev->mtu;
+ unsigned int mtu = net_dev->mtu;
unsigned int header_size = sizeof(frag_header);
unsigned int max_fragment_size, num_fragments;
int ret;
goto put_primary_if;
}
- skb_fragment = batadv_frag_create(skb, &frag_header,
+ skb_fragment = batadv_frag_create(net_dev, skb, &frag_header,
max_fragment_size);
if (!skb_fragment) {
ret = -ENOMEM;
frag_header.no++;
}
- /* Make room for the fragment header. */
- if (batadv_skb_head_push(skb, header_size) < 0 ||
- pskb_expand_head(skb, header_size + ETH_HLEN, 0, GFP_ATOMIC) < 0) {
- ret = -ENOMEM;
+ /* make sure that there is at least enough head for the fragmentation
+ * and ethernet headers
+ */
+ ret = skb_cow_head(skb, ETH_HLEN + header_size);
+ if (ret < 0)
goto put_primary_if;
- }
+ skb_push(skb, header_size);
memcpy(skb->data, &frag_header, header_size);
/* Send the last fragment */
needed_headroom = lower_headroom + (lower_header_len - ETH_HLEN);
needed_headroom += batadv_max_header_len();
+ /* fragmentation headers don't strip the unicast/... header */
+ needed_headroom += sizeof(struct batadv_frag_packet);
+
soft_iface->needed_headroom = needed_headroom;
soft_iface->needed_tailroom = lower_tailroom;
}
.read = batadv_log_read,
.poll = batadv_log_poll,
.llseek = no_llseek,
+ .owner = THIS_MODULE,
};
/**
struct net_bridge *br = netdev_priv(dev);
struct net_bridge_fdb_entry *dst;
struct net_bridge_mdb_entry *mdst;
- struct pcpu_sw_netstats *brstats = this_cpu_ptr(br->stats);
const struct nf_br_ops *nf_ops;
u8 state = BR_STATE_FORWARDING;
const unsigned char *dest;
return NETDEV_TX_OK;
}
- u64_stats_update_begin(&brstats->syncp);
- brstats->tx_packets++;
- brstats->tx_bytes += skb->len;
- u64_stats_update_end(&brstats->syncp);
+ dev_sw_netstats_tx_add(dev, 1, skb->len);
br_switchdev_frame_unmark(skb);
BR_INPUT_SKB_CB(skb)->brdev = dev;
struct net_bridge *br = netdev_priv(dev);
int err;
- br->stats = netdev_alloc_pcpu_stats(struct pcpu_sw_netstats);
- if (!br->stats)
+ dev->tstats = netdev_alloc_pcpu_stats(struct pcpu_sw_netstats);
+ if (!dev->tstats)
return -ENOMEM;
err = br_fdb_hash_init(br);
if (err) {
- free_percpu(br->stats);
+ free_percpu(dev->tstats);
return err;
}
err = br_mdb_hash_init(br);
if (err) {
- free_percpu(br->stats);
+ free_percpu(dev->tstats);
br_fdb_hash_fini(br);
return err;
}
err = br_vlan_init(br);
if (err) {
- free_percpu(br->stats);
+ free_percpu(dev->tstats);
br_mdb_hash_fini(br);
br_fdb_hash_fini(br);
return err;
err = br_multicast_init_stats(br);
if (err) {
- free_percpu(br->stats);
+ free_percpu(dev->tstats);
br_vlan_flush(br);
br_mdb_hash_fini(br);
br_fdb_hash_fini(br);
br_vlan_flush(br);
br_mdb_hash_fini(br);
br_fdb_hash_fini(br);
- free_percpu(br->stats);
+ free_percpu(dev->tstats);
}
static int br_dev_open(struct net_device *dev)
return 0;
}
-static void br_get_stats64(struct net_device *dev,
- struct rtnl_link_stats64 *stats)
-{
- struct net_bridge *br = netdev_priv(dev);
-
- dev_fetch_sw_netstats(stats, br->stats);
-}
-
static int br_change_mtu(struct net_device *dev, int new_mtu)
{
struct net_bridge *br = netdev_priv(dev);
.ndo_init = br_dev_init,
.ndo_uninit = br_dev_uninit,
.ndo_start_xmit = br_dev_xmit,
- .ndo_get_stats64 = br_get_stats64,
+ .ndo_get_stats64 = dev_get_tstats64,
.ndo_set_mac_address = br_set_mac_address,
.ndo_set_rx_mode = br_dev_set_multicast_list,
.ndo_change_rx_flags = br_dev_change_rx_flags,
struct net_device *indev, *brdev = BR_INPUT_SKB_CB(skb)->brdev;
struct net_bridge *br = netdev_priv(brdev);
struct net_bridge_vlan_group *vg;
- struct pcpu_sw_netstats *brstats = this_cpu_ptr(br->stats);
- u64_stats_update_begin(&brstats->syncp);
- brstats->rx_packets++;
- brstats->rx_bytes += skb->len;
- u64_stats_update_end(&brstats->syncp);
+ dev_sw_netstats_rx_add(brdev, skb->len);
vg = br_vlan_group_rcu(br);
/* Bridge is just like any other port. Make sure the
if (hdr->type == BR_MRP_TLV_HEADER_IN_TEST ||
hdr->type == BR_MRP_TLV_HEADER_IN_TOPO ||
hdr->type == BR_MRP_TLV_HEADER_IN_LINK_DOWN ||
- hdr->type == BR_MRP_TLV_HEADER_IN_LINK_UP)
+ hdr->type == BR_MRP_TLV_HEADER_IN_LINK_UP ||
+ hdr->type == BR_MRP_TLV_HEADER_IN_LINK_STATUS)
return true;
return false;
goto no_forward;
}
} else {
- /* MIM should forward IntLinkChange and
+ /* MIM should forward IntLinkChange/Status and
* IntTopoChange between ring ports but MIM
- * should not forward IntLinkChange and
+ * should not forward IntLinkChange/Status and
* IntTopoChange if the frame was received at
* the interconnect port
*/
in_type == BR_MRP_TLV_HEADER_IN_LINK_DOWN))
goto forward;
+ /* MIC should forward IntLinkStatus frames only to
+ * interconnect port if it was received on a ring port.
+ * If it is received on interconnect port then, it
+ * should be forward on both ring ports
+ */
+ if (br_mrp_is_ring_port(p_port, s_port, p) &&
+ in_type == BR_MRP_TLV_HEADER_IN_LINK_STATUS) {
+ p_dst = NULL;
+ s_dst = NULL;
+ }
+
/* Should forward the InTopo frames only between the
* ring ports
*/
mtu_reserved = nf_bridge_mtu_reduction(skb);
mtu = skb->dev->mtu;
+ if (nf_bridge->pkt_otherhost) {
+ skb->pkt_type = PACKET_OTHERHOST;
+ nf_bridge->pkt_otherhost = false;
+ }
+
if (nf_bridge->frag_max_size && nf_bridge->frag_max_size < mtu)
mtu = nf_bridge->frag_max_size;
else
return NF_ACCEPT;
- /* We assume any code from br_dev_queue_push_xmit onwards doesn't care
- * about the value of skb->pkt_type. */
if (skb->pkt_type == PACKET_OTHERHOST) {
skb->pkt_type = PACKET_HOST;
nf_bridge->pkt_otherhost = true;
pvid = br_get_pvid(vg);
list_for_each_entry(v, &vg->vlan_list, vlist) {
struct bridge_vlan_xstats vxi;
- struct br_vlan_stats stats;
+ struct pcpu_sw_netstats stats;
if (++vl_idx < *prividx)
continue;
};
#endif
-struct br_vlan_stats {
- u64 rx_bytes;
- u64 rx_packets;
- u64 tx_bytes;
- u64 tx_packets;
- struct u64_stats_sync syncp;
-};
-
struct br_tunnel_info {
__be64 tunnel_id;
struct metadata_dst *tunnel_dst;
u16 flags;
u16 priv_flags;
u8 state;
- struct br_vlan_stats __percpu *stats;
+ struct pcpu_sw_netstats __percpu *stats;
union {
struct net_bridge *br;
struct net_bridge_port *port;
spinlock_t hash_lock;
struct hlist_head frame_type_list;
struct net_device *dev;
- struct pcpu_sw_netstats __percpu *stats;
unsigned long options;
/* These fields are accessed on each packet */
#ifdef CONFIG_BRIDGE_VLAN_FILTERING
int nbp_vlan_init(struct net_bridge_port *port, struct netlink_ext_ack *extack);
int nbp_get_num_vlan_infos(struct net_bridge_port *p, u32 filter_mask);
void br_vlan_get_stats(const struct net_bridge_vlan *v,
- struct br_vlan_stats *stats);
+ struct pcpu_sw_netstats *stats);
void br_vlan_port_event(struct net_bridge_port *p, unsigned long event);
int br_vlan_bridge_event(struct net_device *dev, unsigned long event,
void *ptr);
}
static inline void br_vlan_get_stats(const struct net_bridge_vlan *v,
- struct br_vlan_stats *stats)
+ struct pcpu_sw_netstats *stats)
{
}
goto out_filt;
v->brvlan = masterv;
if (br_opt_get(br, BROPT_VLAN_STATS_PER_PORT)) {
- v->stats = netdev_alloc_pcpu_stats(struct br_vlan_stats);
+ v->stats =
+ netdev_alloc_pcpu_stats(struct pcpu_sw_netstats);
if (!v->stats) {
err = -ENOMEM;
goto out_filt;
struct net_bridge_vlan_group *vg,
struct sk_buff *skb)
{
- struct br_vlan_stats *stats;
+ struct pcpu_sw_netstats *stats;
struct net_bridge_vlan *v;
u16 vid;
struct sk_buff *skb, u16 *vid,
u8 *state)
{
- struct br_vlan_stats *stats;
+ struct pcpu_sw_netstats *stats;
struct net_bridge_vlan *v;
bool tagged;
if (!vlan)
return -ENOMEM;
- vlan->stats = netdev_alloc_pcpu_stats(struct br_vlan_stats);
+ vlan->stats = netdev_alloc_pcpu_stats(struct pcpu_sw_netstats);
if (!vlan->stats) {
kfree(vlan);
return -ENOMEM;
int __br_vlan_set_proto(struct net_bridge *br, __be16 proto)
{
+ struct switchdev_attr attr = {
+ .orig_dev = br->dev,
+ .id = SWITCHDEV_ATTR_ID_BRIDGE_VLAN_PROTOCOL,
+ .flags = SWITCHDEV_F_SKIP_EOPNOTSUPP,
+ .u.vlan_protocol = ntohs(proto),
+ };
int err = 0;
struct net_bridge_port *p;
struct net_bridge_vlan *vlan;
struct net_bridge_vlan_group *vg;
- __be16 oldproto;
+ __be16 oldproto = br->vlan_proto;
if (br->vlan_proto == proto)
return 0;
+ err = switchdev_port_attr_set(br->dev, &attr);
+ if (err && err != -EOPNOTSUPP)
+ return err;
+
/* Add VLANs for the new proto to the device filter. */
list_for_each_entry(p, &br->port_list, list) {
vg = nbp_vlan_group(p);
}
}
- oldproto = br->vlan_proto;
br->vlan_proto = proto;
recalculate_group_addr(br);
return 0;
err_filt:
+ attr.u.vlan_protocol = ntohs(oldproto);
+ switchdev_port_attr_set(br->dev, &attr);
+
list_for_each_entry_continue_reverse(vlan, &vg->vlan_list, vlist)
vlan_vid_del(p->dev, proto, vlan->vid);
}
void br_vlan_get_stats(const struct net_bridge_vlan *v,
- struct br_vlan_stats *stats)
+ struct pcpu_sw_netstats *stats)
{
int i;
memset(stats, 0, sizeof(*stats));
for_each_possible_cpu(i) {
u64 rxpackets, rxbytes, txpackets, txbytes;
- struct br_vlan_stats *cpu_stats;
+ struct pcpu_sw_netstats *cpu_stats;
unsigned int start;
cpu_stats = per_cpu_ptr(v->stats, i);
static bool br_vlan_stats_fill(struct sk_buff *skb,
const struct net_bridge_vlan *v)
{
- struct br_vlan_stats stats;
+ struct pcpu_sw_netstats stats;
struct nlattr *nest;
nest = nla_nest_start(skb, BRIDGE_VLANDB_ENTRY_STATS);
/* Check for bugs in CAN protocol implementations using af_can.c:
* 'rcv' will be NULL if no matching list item was found for removal.
+ * As this case may potentially happen when closing a socket while
+ * the notifier for removing the CAN netdev is running we just print
+ * a warning here.
*/
if (!rcv) {
- WARN(1, "BUG: receive list entry not found for dev %s, id %03X, mask %03X\n",
- DNAME(dev), can_id, mask);
+ pr_warn("can: receive list entry not found for dev %s, id %03X, mask %03X\n",
+ DNAME(dev), can_id, mask);
goto out;
}
{
struct canfd_frame *cfd = (struct canfd_frame *)skb->data;
- if (unlikely(dev->type != ARPHRD_CAN || skb->len != CAN_MTU ||
- cfd->len > CAN_MAX_DLEN)) {
- pr_warn_once("PF_CAN: dropped non conform CAN skbuf: dev type %d, len %d, datalen %d\n",
+ if (unlikely(dev->type != ARPHRD_CAN || skb->len != CAN_MTU)) {
+ pr_warn_once("PF_CAN: dropped non conform CAN skbuff: dev type %d, len %d\n",
+ dev->type, skb->len);
+ goto free_skb;
+ }
+
+ /* This check is made separately since cfd->len would be uninitialized if skb->len = 0. */
+ if (unlikely(cfd->len > CAN_MAX_DLEN)) {
+ pr_warn_once("PF_CAN: dropped non conform CAN skbuff: dev type %d, len %d, datalen %d\n",
dev->type, skb->len, cfd->len);
- kfree_skb(skb);
- return NET_RX_DROP;
+ goto free_skb;
}
can_receive(skb, dev);
return NET_RX_SUCCESS;
+
+free_skb:
+ kfree_skb(skb);
+ return NET_RX_DROP;
}
static int canfd_rcv(struct sk_buff *skb, struct net_device *dev,
{
struct canfd_frame *cfd = (struct canfd_frame *)skb->data;
- if (unlikely(dev->type != ARPHRD_CAN || skb->len != CANFD_MTU ||
- cfd->len > CANFD_MAX_DLEN)) {
- pr_warn_once("PF_CAN: dropped non conform CAN FD skbuf: dev type %d, len %d, datalen %d\n",
+ if (unlikely(dev->type != ARPHRD_CAN || skb->len != CANFD_MTU)) {
+ pr_warn_once("PF_CAN: dropped non conform CAN FD skbuff: dev type %d, len %d\n",
+ dev->type, skb->len);
+ goto free_skb;
+ }
+
+ /* This check is made separately since cfd->len would be uninitialized if skb->len = 0. */
+ if (unlikely(cfd->len > CANFD_MAX_DLEN)) {
+ pr_warn_once("PF_CAN: dropped non conform CAN FD skbuff: dev type %d, len %d, datalen %d\n",
dev->type, skb->len, cfd->len);
- kfree_skb(skb);
- return NET_RX_DROP;
+ goto free_skb;
}
can_receive(skb, dev);
return NET_RX_SUCCESS;
+
+free_skb:
+ kfree_skb(skb);
+ return NET_RX_DROP;
}
/* af_can protocol functions */
int err;
/* check for correct padding to be able to use the structs similarly */
- BUILD_BUG_ON(offsetof(struct can_frame, can_dlc) !=
+ BUILD_BUG_ON(offsetof(struct can_frame, len) !=
offsetof(struct canfd_frame, len) ||
offsetof(struct can_frame, data) !=
offsetof(struct canfd_frame, data));
memcpy(cf->data, mod->modframe.set.data, CANFD_MAX_DLEN);
}
+/* retrieve valid CC DLC value and store it into 'len' */
+static void mod_retrieve_ccdlc(struct canfd_frame *cf)
+{
+ struct can_frame *ccf = (struct can_frame *)cf;
+
+ /* len8_dlc is only valid if len == CAN_MAX_DLEN */
+ if (ccf->len != CAN_MAX_DLEN)
+ return;
+
+ /* do we have a valid len8_dlc value from 9 .. 15 ? */
+ if (ccf->len8_dlc > CAN_MAX_DLEN && ccf->len8_dlc <= CAN_MAX_RAW_DLC)
+ ccf->len = ccf->len8_dlc;
+}
+
+/* convert valid CC DLC value in 'len' into struct can_frame elements */
+static void mod_store_ccdlc(struct canfd_frame *cf)
+{
+ struct can_frame *ccf = (struct can_frame *)cf;
+
+ /* clear potential leftovers */
+ ccf->len8_dlc = 0;
+
+ /* plain data length 0 .. 8 - that was easy */
+ if (ccf->len <= CAN_MAX_DLEN)
+ return;
+
+ /* potentially broken values are catched in can_can_gw_rcv() */
+ if (ccf->len > CAN_MAX_RAW_DLC)
+ return;
+
+ /* we have a valid dlc value from 9 .. 15 in ccf->len */
+ ccf->len8_dlc = ccf->len;
+ ccf->len = CAN_MAX_DLEN;
+}
+
+static void mod_and_ccdlc(struct canfd_frame *cf, struct cf_mod *mod)
+{
+ mod_retrieve_ccdlc(cf);
+ mod_and_len(cf, mod);
+ mod_store_ccdlc(cf);
+}
+
+static void mod_or_ccdlc(struct canfd_frame *cf, struct cf_mod *mod)
+{
+ mod_retrieve_ccdlc(cf);
+ mod_or_len(cf, mod);
+ mod_store_ccdlc(cf);
+}
+
+static void mod_xor_ccdlc(struct canfd_frame *cf, struct cf_mod *mod)
+{
+ mod_retrieve_ccdlc(cf);
+ mod_xor_len(cf, mod);
+ mod_store_ccdlc(cf);
+}
+
+static void mod_set_ccdlc(struct canfd_frame *cf, struct cf_mod *mod)
+{
+ mod_set_len(cf, mod);
+ mod_store_ccdlc(cf);
+}
+
static void canframecpy(struct canfd_frame *dst, struct can_frame *src)
{
/* Copy the struct members separately to ensure that no uninitialized
*/
dst->can_id = src->can_id;
- dst->len = src->can_dlc;
+ dst->len = src->len;
*(u64 *)dst->data = *(u64 *)src->data;
}
if (mb.modtype & CGW_MOD_ID)
mod->modfunc[modidx++] = mod_and_id;
- if (mb.modtype & CGW_MOD_LEN)
- mod->modfunc[modidx++] = mod_and_len;
+ if (mb.modtype & CGW_MOD_DLC)
+ mod->modfunc[modidx++] = mod_and_ccdlc;
if (mb.modtype & CGW_MOD_DATA)
mod->modfunc[modidx++] = mod_and_data;
if (mb.modtype & CGW_MOD_ID)
mod->modfunc[modidx++] = mod_or_id;
- if (mb.modtype & CGW_MOD_LEN)
- mod->modfunc[modidx++] = mod_or_len;
+ if (mb.modtype & CGW_MOD_DLC)
+ mod->modfunc[modidx++] = mod_or_ccdlc;
if (mb.modtype & CGW_MOD_DATA)
mod->modfunc[modidx++] = mod_or_data;
if (mb.modtype & CGW_MOD_ID)
mod->modfunc[modidx++] = mod_xor_id;
- if (mb.modtype & CGW_MOD_LEN)
- mod->modfunc[modidx++] = mod_xor_len;
+ if (mb.modtype & CGW_MOD_DLC)
+ mod->modfunc[modidx++] = mod_xor_ccdlc;
if (mb.modtype & CGW_MOD_DATA)
mod->modfunc[modidx++] = mod_xor_data;
if (mb.modtype & CGW_MOD_ID)
mod->modfunc[modidx++] = mod_set_id;
- if (mb.modtype & CGW_MOD_LEN)
- mod->modfunc[modidx++] = mod_set_len;
+ if (mb.modtype & CGW_MOD_DLC)
+ mod->modfunc[modidx++] = mod_set_ccdlc;
if (mb.modtype & CGW_MOD_DATA)
mod->modfunc[modidx++] = mod_set_data;
skb_pull(skb, J1939_CAN_HDR);
/* fix length, set to dlc, with 8 maximum */
- skb_trim(skb, min_t(uint8_t, cf->can_dlc, 8));
+ skb_trim(skb, min_t(uint8_t, cf->len, 8));
/* set addr */
skcb = j1939_skb_to_cb(skb);
canid |= skcb->addr.da << 8;
cf->can_id = canid;
- cf->can_dlc = dlc;
+ cf->len = dlc;
return can_send(skb, 1);
EXPORT_SYMBOL(zerocopy_sg_from_iter);
/**
- * skb_copy_and_csum_datagram_iter - Copy datagram to an iovec iterator
+ * skb_copy_and_csum_datagram - Copy datagram to an iovec iterator
* and update a checksum.
* @skb: buffer to copy
* @offset: offset in the buffer to start copying from
}
EXPORT_SYMBOL(dev_getbyhwaddr_rcu);
-struct net_device *__dev_getfirstbyhwtype(struct net *net, unsigned short type)
-{
- struct net_device *dev;
-
- ASSERT_RTNL();
- for_each_netdev(net, dev)
- if (dev->type == type)
- return dev;
-
- return NULL;
-}
-EXPORT_SYMBOL(__dev_getfirstbyhwtype);
-
struct net_device *dev_getfirstbyhwtype(struct net *net, unsigned short type)
{
struct net_device *dev, *ret = NULL;
if (gso_segs > dev->gso_max_segs)
return features & ~NETIF_F_GSO_MASK;
+ if (!skb_shinfo(skb)->gso_type) {
+ skb_warn_bad_offload(skb);
+ return features & ~NETIF_F_GSO_MASK;
+ }
+
/* Support for GSO partial features requires software
* intervention before we can actually process the packets
* so we need to strip support for any partial features now
return skb;
/* qdisc_skb_cb(skb)->pkt_len was already set by the caller. */
+ qdisc_skb_cb(skb)->mru = 0;
mini_qdisc_bstats_cpu_update(miniq, skb);
switch (tcf_classify(skb, miniq->filter_list, &cl_res, false)) {
}
EXPORT_SYMBOL(dev_queue_xmit_accel);
-int dev_direct_xmit(struct sk_buff *skb, u16 queue_id)
+int __dev_direct_xmit(struct sk_buff *skb, u16 queue_id)
{
struct net_device *dev = skb->dev;
struct sk_buff *orig_skb = skb;
dev_xmit_recursion_dec();
local_bh_enable();
-
- if (!dev_xmit_complete(ret))
- kfree_skb(skb);
-
return ret;
drop:
atomic_long_inc(&dev->tx_dropped);
kfree_skb_list(skb);
return NET_XMIT_DROP;
}
-EXPORT_SYMBOL(dev_direct_xmit);
+EXPORT_SYMBOL(__dev_direct_xmit);
/*************************************************************************
* Receiver routines
}
qdisc_skb_cb(skb)->pkt_len = skb->len;
+ qdisc_skb_cb(skb)->mru = 0;
skb->tc_at_ingress = 1;
mini_qdisc_bstats_cpu_update(miniq, skb);
EXPORT_SYMBOL(netdev_has_upper_dev);
/**
- * netdev_has_upper_dev_all - Check if device is linked to an upper device
+ * netdev_has_upper_dev_all_rcu - Check if device is linked to an upper device
* @dev: device
* @upper_dev: upper device to check
*
/**
- * netdev_lower_change - Dispatch event about lower device state change
+ * netdev_lower_state_changed - Dispatch event about lower device state change
* @lower_dev: device
* @lower_state_info: state to dispatch
*
return test_bit(limit, &devlink->ops->reload_limits);
}
-static int devlink_reload_stat_put(struct sk_buff *msg, enum devlink_reload_action action,
+static int devlink_reload_stat_put(struct sk_buff *msg,
enum devlink_reload_limit limit, u32 value)
{
struct nlattr *reload_stats_entry;
if (!reload_stats_entry)
return -EMSGSIZE;
- if (nla_put_u8(msg, DEVLINK_ATTR_RELOAD_ACTION, action) ||
- nla_put_u8(msg, DEVLINK_ATTR_RELOAD_STATS_LIMIT, limit) ||
+ if (nla_put_u8(msg, DEVLINK_ATTR_RELOAD_STATS_LIMIT, limit) ||
nla_put_u32(msg, DEVLINK_ATTR_RELOAD_STATS_VALUE, value))
goto nla_put_failure;
nla_nest_end(msg, reload_stats_entry);
static int devlink_reload_stats_put(struct sk_buff *msg, struct devlink *devlink, bool is_remote)
{
- struct nlattr *reload_stats_attr;
+ struct nlattr *reload_stats_attr, *act_info, *act_stats;
int i, j, stat_idx;
u32 value;
if (!reload_stats_attr)
return -EMSGSIZE;
- for (j = 0; j <= DEVLINK_RELOAD_LIMIT_MAX; j++) {
- /* Remote stats are shown even if not locally supported. Stats
- * of actions with unspecified limit are shown though drivers
- * don't need to register unspecified limit.
- */
- if (!is_remote && j != DEVLINK_RELOAD_LIMIT_UNSPEC &&
- !devlink_reload_limit_is_supported(devlink, j))
+ for (i = 0; i <= DEVLINK_RELOAD_ACTION_MAX; i++) {
+ if ((!is_remote &&
+ !devlink_reload_action_is_supported(devlink, i)) ||
+ i == DEVLINK_RELOAD_ACTION_UNSPEC)
continue;
- for (i = 0; i <= DEVLINK_RELOAD_ACTION_MAX; i++) {
- if ((!is_remote && !devlink_reload_action_is_supported(devlink, i)) ||
- i == DEVLINK_RELOAD_ACTION_UNSPEC ||
+ act_info = nla_nest_start(msg, DEVLINK_ATTR_RELOAD_ACTION_INFO);
+ if (!act_info)
+ goto nla_put_failure;
+
+ if (nla_put_u8(msg, DEVLINK_ATTR_RELOAD_ACTION, i))
+ goto action_info_nest_cancel;
+ act_stats = nla_nest_start(msg, DEVLINK_ATTR_RELOAD_ACTION_STATS);
+ if (!act_stats)
+ goto action_info_nest_cancel;
+
+ for (j = 0; j <= DEVLINK_RELOAD_LIMIT_MAX; j++) {
+ /* Remote stats are shown even if not locally supported.
+ * Stats of actions with unspecified limit are shown
+ * though drivers don't need to register unspecified
+ * limit.
+ */
+ if ((!is_remote && j != DEVLINK_RELOAD_LIMIT_UNSPEC &&
+ !devlink_reload_limit_is_supported(devlink, j)) ||
devlink_reload_combination_is_invalid(i, j))
continue;
value = devlink->stats.reload_stats[stat_idx];
else
value = devlink->stats.remote_reload_stats[stat_idx];
- if (devlink_reload_stat_put(msg, i, j, value))
- goto nla_put_failure;
+ if (devlink_reload_stat_put(msg, j, value))
+ goto action_stats_nest_cancel;
}
+ nla_nest_end(msg, act_stats);
+ nla_nest_end(msg, act_info);
}
nla_nest_end(msg, reload_stats_attr);
return 0;
+action_stats_nest_cancel:
+ nla_nest_cancel(msg, act_stats);
+action_info_nest_cancel:
+ nla_nest_cancel(msg, act_info);
nla_put_failure:
nla_nest_cancel(msg, reload_stats_attr);
return -EMSGSIZE;
if (nla_put_u32(msg, DEVLINK_ATTR_PORT_INDEX, devlink_port->index))
goto nla_put_failure;
+ /* Hold rtnl lock while accessing port's netdev attributes. */
+ rtnl_lock();
spin_lock_bh(&devlink_port->type_lock);
if (nla_put_u16(msg, DEVLINK_ATTR_PORT_TYPE, devlink_port->type))
goto nla_put_failure_type_locked;
devlink_port->desired_type))
goto nla_put_failure_type_locked;
if (devlink_port->type == DEVLINK_PORT_TYPE_ETH) {
+ struct net *net = devlink_net(devlink_port->devlink);
struct net_device *netdev = devlink_port->type_dev;
- if (netdev &&
+ if (netdev && net_eq(net, dev_net(netdev)) &&
(nla_put_u32(msg, DEVLINK_ATTR_PORT_NETDEV_IFINDEX,
netdev->ifindex) ||
nla_put_string(msg, DEVLINK_ATTR_PORT_NETDEV_NAME,
goto nla_put_failure_type_locked;
}
spin_unlock_bh(&devlink_port->type_lock);
+ rtnl_unlock();
if (devlink_nl_port_attrs_put(msg, devlink_port))
goto nla_put_failure;
if (devlink_nl_port_function_attrs_put(msg, devlink_port, extack))
nla_put_failure_type_locked:
spin_unlock_bh(&devlink_port->type_lock);
+ rtnl_unlock();
nla_put_failure:
genlmsg_cancel(msg, hdr);
return -EMSGSIZE;
err = ops->sb_occ_port_pool_get(devlink_port, devlink_sb->index,
pool_index, &cur, &max);
if (err && err != -EOPNOTSUPP)
- return err;
+ goto sb_occ_get_failure;
if (!err) {
if (nla_put_u32(msg, DEVLINK_ATTR_SB_OCC_CUR, cur))
goto nla_put_failure;
return 0;
nla_put_failure:
+ err = -EMSGSIZE;
+sb_occ_get_failure:
genlmsg_cancel(msg, hdr);
- return -EMSGSIZE;
+ return err;
}
static int devlink_nl_cmd_sb_port_pool_get_doit(struct sk_buff *skb,
nlmsg_free(msg);
}
-void devlink_flash_update_begin_notify(struct devlink *devlink)
+static void devlink_flash_update_begin_notify(struct devlink *devlink)
{
struct devlink_flash_notify params = { 0 };
DEVLINK_CMD_FLASH_UPDATE,
¶ms);
}
-EXPORT_SYMBOL_GPL(devlink_flash_update_begin_notify);
-void devlink_flash_update_end_notify(struct devlink *devlink)
+static void devlink_flash_update_end_notify(struct devlink *devlink)
{
struct devlink_flash_notify params = { 0 };
DEVLINK_CMD_FLASH_UPDATE_END,
¶ms);
}
-EXPORT_SYMBOL_GPL(devlink_flash_update_end_notify);
void devlink_flash_update_status_notify(struct devlink *devlink,
const char *status_msg,
static int devlink_nl_cmd_flash_update(struct sk_buff *skb,
struct genl_info *info)
{
- struct nlattr *nla_component, *nla_overwrite_mask;
+ struct nlattr *nla_component, *nla_overwrite_mask, *nla_file_name;
struct devlink_flash_update_params params = {};
struct devlink *devlink = info->user_ptr[0];
+ const char *file_name;
u32 supported_params;
+ int ret;
if (!devlink->ops->flash_update)
return -EOPNOTSUPP;
supported_params = devlink->ops->supported_flash_update_params;
- params.file_name = nla_data(info->attrs[DEVLINK_ATTR_FLASH_UPDATE_FILE_NAME]);
-
nla_component = info->attrs[DEVLINK_ATTR_FLASH_UPDATE_COMPONENT];
if (nla_component) {
if (!(supported_params & DEVLINK_SUPPORT_FLASH_UPDATE_COMPONENT)) {
params.overwrite_mask = sections.value & sections.selector;
}
- return devlink->ops->flash_update(devlink, ¶ms, info->extack);
+ nla_file_name = info->attrs[DEVLINK_ATTR_FLASH_UPDATE_FILE_NAME];
+ file_name = nla_data(nla_file_name);
+ ret = request_firmware(¶ms.fw, file_name, devlink->dev);
+ if (ret) {
+ NL_SET_ERR_MSG_ATTR(info->extack, nla_file_name, "failed to locate the requested firmware file");
+ return ret;
+ }
+
+ devlink_flash_update_begin_notify(devlink);
+ ret = devlink->ops->flash_update(devlink, ¶ms, info->extack);
+ devlink_flash_update_end_notify(devlink);
+
+ release_firmware(params.fw);
+
+ return ret;
}
static const struct devlink_param devlink_param_generic[] = {
DEVLINK_TRAP(DCCP_PARSING, DROP),
DEVLINK_TRAP(GTP_PARSING, DROP),
DEVLINK_TRAP(ESP_PARSING, DROP),
+ DEVLINK_TRAP(BLACKHOLE_NEXTHOP, DROP),
};
#define DEVLINK_TRAP_GROUP(_id) \
goto out;
}
- params.file_name = file_name;
+ ret = request_firmware(¶ms.fw, file_name, devlink->dev);
+ if (ret)
+ goto out;
mutex_lock(&devlink->lock);
+ devlink_flash_update_begin_notify(devlink);
ret = devlink->ops->flash_update(devlink, ¶ms, NULL);
+ devlink_flash_update_end_notify(devlink);
mutex_unlock(&devlink->lock);
+ release_firmware(params.fw);
+
out:
rtnl_lock();
dev_put(dev);
struct net_device *dev;
nlrule->iifindex = -1;
- nla_strlcpy(nlrule->iifname, tb[FRA_IIFNAME], IFNAMSIZ);
+ nla_strscpy(nlrule->iifname, tb[FRA_IIFNAME], IFNAMSIZ);
dev = __dev_get_by_name(net, nlrule->iifname);
if (dev)
nlrule->iifindex = dev->ifindex;
struct net_device *dev;
nlrule->oifindex = -1;
- nla_strlcpy(nlrule->oifname, tb[FRA_OIFNAME], IFNAMSIZ);
+ nla_strscpy(nlrule->oifname, tb[FRA_OIFNAME], IFNAMSIZ);
dev = __dev_get_by_name(net, nlrule->oifname);
if (dev)
nlrule->oifindex = dev->ifindex;
struct gro_cell *cell = per_cpu_ptr(gcells->cells, i);
napi_disable(&cell->napi);
- netif_napi_del(&cell->napi);
+ __netif_napi_del(&cell->napi);
__skb_queue_purge(&cell->napi_skbs);
}
+ /* This barrier is needed because netpoll could access dev->napi_list
+ * under rcu protection.
+ */
+ synchronize_net();
+
free_percpu(gcells->cells);
gcells->cells = NULL;
}
write_lock(&n->lock);
if ((n->nud_state == NUD_FAILED) ||
+ (tbl->is_multicast &&
+ tbl->is_multicast(n->primary_key)) ||
time_after(tref, n->updated))
remove = true;
write_unlock(&n->lock);
#include <linux/slab.h>
#include <linux/export.h>
#include <linux/if_vlan.h>
+#include <net/dsa.h>
#include <net/tcp.h>
#include <net/udp.h>
#include <net/addrconf.h>
int netpoll_setup(struct netpoll *np)
{
- struct net_device *ndev = NULL;
+ struct net_device *ndev = NULL, *dev = NULL;
+ struct net *net = current->nsproxy->net_ns;
struct in_device *in_dev;
int err;
rtnl_lock();
- if (np->dev_name[0]) {
- struct net *net = current->nsproxy->net_ns;
+ if (np->dev_name[0])
ndev = __dev_get_by_name(net, np->dev_name);
- }
+
if (!ndev) {
np_err(np, "%s doesn't exist, aborting\n", np->dev_name);
err = -ENODEV;
}
dev_hold(ndev);
+ /* bring up DSA management network devices up first */
+ for_each_netdev(net, dev) {
+ if (!netdev_uses_dsa(dev))
+ continue;
+
+ err = dev_change_flags(dev, dev->flags | IFF_UP, NULL);
+ if (err < 0) {
+ np_err(np, "%s failed to open %s\n",
+ np->dev_name, dev->name);
+ goto put;
+ }
+ }
+
if (netdev_master_upper_dev_get(ndev)) {
np_err(np, "%s is a slave device, aborting\n", np->dev_name);
err = -EBUSY;
if (linfo[IFLA_INFO_KIND]) {
char kind[MODULE_NAME_LEN];
- nla_strlcpy(kind, linfo[IFLA_INFO_KIND], sizeof(kind));
+ nla_strscpy(kind, linfo[IFLA_INFO_KIND], sizeof(kind));
ops = rtnl_link_ops_get(kind);
}
if (!ifname) {
ifname = buffer;
if (ifname_attr)
- nla_strlcpy(ifname, ifname_attr, IFNAMSIZ);
+ nla_strscpy(ifname, ifname_attr, IFNAMSIZ);
else if (altifname_attr)
- nla_strlcpy(ifname, altifname_attr, ALTIFNAMSIZ);
+ nla_strscpy(ifname, altifname_attr, ALTIFNAMSIZ);
else
return NULL;
}
goto errout;
if (tb[IFLA_IFNAME])
- nla_strlcpy(ifname, tb[IFLA_IFNAME], IFNAMSIZ);
+ nla_strscpy(ifname, tb[IFLA_IFNAME], IFNAMSIZ);
else
ifname[0] = '\0';
return err;
if (tb[IFLA_IFNAME])
- nla_strlcpy(ifname, tb[IFLA_IFNAME], IFNAMSIZ);
+ nla_strscpy(ifname, tb[IFLA_IFNAME], IFNAMSIZ);
else
ifname[0] = '\0';
memset(linkinfo, 0, sizeof(linkinfo));
if (linkinfo[IFLA_INFO_KIND]) {
- nla_strlcpy(kind, linkinfo[IFLA_INFO_KIND], sizeof(kind));
+ nla_strscpy(kind, linkinfo[IFLA_INFO_KIND], sizeof(kind));
ops = rtnl_link_ops_get(kind);
} else {
kind[0] = '\0';
#endif
/**
- * consume_stateless_skb - free an skbuff, assuming it is stateless
+ * __consume_stateless_skb - free an skbuff, assuming it is stateless
* @skb: buffer to free
*
* Alike consume_skb(), but this variant assumes that this is the last
return;
}
+ lockdep_assert_in_softirq();
+
if (!skb_unref(skb))
return;
#if IS_ENABLED(CONFIG_MPTCP)
[SKB_EXT_MPTCP] = SKB_EXT_CHUNKSIZEOF(struct mptcp_ext),
#endif
-#if IS_ENABLED(CONFIG_KCOV)
- [SKB_EXT_KCOV_HANDLE] = SKB_EXT_CHUNKSIZEOF(u64),
-#endif
};
static __always_inline unsigned int skb_ext_total_length(void)
#endif
#if IS_ENABLED(CONFIG_MPTCP)
skb_ext_type_len[SKB_EXT_MPTCP] +
-#endif
-#if IS_ENABLED(CONFIG_KCOV)
- skb_ext_type_len[SKB_EXT_KCOV_HANDLE] +
#endif
0;
}
if (skb && (skb_next = skb_peek(q))) {
icmp_next = is_icmp_err_skb(skb_next);
if (icmp_next)
- sk->sk_err = SKB_EXT_ERR(skb_next)->ee.ee_origin;
+ sk->sk_err = SKB_EXT_ERR(skb_next)->ee.ee_errno;
}
spin_unlock_irqrestore(&q->lock, flags);
if (unlikely(!eth_p_mpls(skb->protocol)))
return -EINVAL;
+ if (!pskb_may_pull(skb, skb_network_offset(skb) + MPLS_HLEN))
+ return -ENOMEM;
+
lse = be32_to_cpu(mpls_hdr(skb)->label_stack_entry);
ttl = (lse & MPLS_LS_TTL_MASK) >> MPLS_LS_TTL_SHIFT;
if (!--ttl)
struct scatterlist *sge = sk_msg_elem(msg, i);
u32 len = sge->length;
- if (charge)
- sk_mem_uncharge(sk, len);
- if (!msg->skb)
+ /* When the skb owns the memory we free it from consume_skb path. */
+ if (!msg->skb) {
+ if (charge)
+ sk_mem_uncharge(sk, len);
put_page(sg_page(sge));
+ }
memset(sge, 0, sizeof(*sge));
return len;
}
}
EXPORT_SYMBOL_GPL(sk_msg_memcopy_from_iter);
-static int sk_psock_skb_ingress(struct sk_psock *psock, struct sk_buff *skb)
+static struct sk_msg *sk_psock_create_ingress_msg(struct sock *sk,
+ struct sk_buff *skb)
{
- struct sock *sk = psock->sk;
- int copied = 0, num_sge;
struct sk_msg *msg;
+ if (atomic_read(&sk->sk_rmem_alloc) > sk->sk_rcvbuf)
+ return NULL;
+
+ if (!sk_rmem_schedule(sk, skb, skb->truesize))
+ return NULL;
+
msg = kzalloc(sizeof(*msg), __GFP_NOWARN | GFP_ATOMIC);
if (unlikely(!msg))
- return -EAGAIN;
- if (!sk_rmem_schedule(sk, skb, skb->len)) {
- kfree(msg);
- return -EAGAIN;
- }
+ return NULL;
sk_msg_init(msg);
+ return msg;
+}
+
+static int sk_psock_skb_ingress_enqueue(struct sk_buff *skb,
+ struct sk_psock *psock,
+ struct sock *sk,
+ struct sk_msg *msg)
+{
+ int num_sge, copied;
+
+ /* skb linearize may fail with ENOMEM, but lets simply try again
+ * later if this happens. Under memory pressure we don't want to
+ * drop the skb. We need to linearize the skb so that the mapping
+ * in skb_to_sgvec can not error.
+ */
+ if (skb_linearize(skb))
+ return -EAGAIN;
num_sge = skb_to_sgvec(skb, msg->sg.data, 0, skb->len);
if (unlikely(num_sge < 0)) {
kfree(msg);
return num_sge;
}
- sk_mem_charge(sk, skb->len);
copied = skb->len;
msg->sg.start = 0;
msg->sg.size = copied;
return copied;
}
+static int sk_psock_skb_ingress_self(struct sk_psock *psock, struct sk_buff *skb);
+
+static int sk_psock_skb_ingress(struct sk_psock *psock, struct sk_buff *skb)
+{
+ struct sock *sk = psock->sk;
+ struct sk_msg *msg;
+
+ /* If we are receiving on the same sock skb->sk is already assigned,
+ * skip memory accounting and owner transition seeing it already set
+ * correctly.
+ */
+ if (unlikely(skb->sk == sk))
+ return sk_psock_skb_ingress_self(psock, skb);
+ msg = sk_psock_create_ingress_msg(sk, skb);
+ if (!msg)
+ return -EAGAIN;
+
+ /* This will transition ownership of the data from the socket where
+ * the BPF program was run initiating the redirect to the socket
+ * we will eventually receive this data on. The data will be released
+ * from skb_consume found in __tcp_bpf_recvmsg() after its been copied
+ * into user buffers.
+ */
+ skb_set_owner_r(skb, sk);
+ return sk_psock_skb_ingress_enqueue(skb, psock, sk, msg);
+}
+
+/* Puts an skb on the ingress queue of the socket already assigned to the
+ * skb. In this case we do not need to check memory limits or skb_set_owner_r
+ * because the skb is already accounted for here.
+ */
+static int sk_psock_skb_ingress_self(struct sk_psock *psock, struct sk_buff *skb)
+{
+ struct sk_msg *msg = kzalloc(sizeof(*msg), __GFP_NOWARN | GFP_ATOMIC);
+ struct sock *sk = psock->sk;
+
+ if (unlikely(!msg))
+ return -EAGAIN;
+ sk_msg_init(msg);
+ return sk_psock_skb_ingress_enqueue(skb, psock, sk, msg);
+}
+
static int sk_psock_handle_skb(struct sk_psock *psock, struct sk_buff *skb,
u32 off, u32 len, bool ingress)
{
* retrying later from workqueue.
*/
if (skb_queue_empty(&psock->ingress_skb)) {
- err = sk_psock_skb_ingress(psock, skb);
+ err = sk_psock_skb_ingress_self(psock, skb);
}
if (err < 0) {
skb_queue_tail(&psock->ingress_skb, skb);
}
EXPORT_SYMBOL(sk_page_frag_refill);
-static void __lock_sock(struct sock *sk)
+void __lock_sock(struct sock *sk)
__releases(&sk->sk_lock.slock)
__acquires(&sk->sk_lock.slock)
{
*
* sk_lock.slock unlocked, owned = 1, BH enabled
*/
-bool lock_sock_fast(struct sock *sk)
+bool lock_sock_fast(struct sock *sk) __acquires(&sk->sk_lock.slock)
{
might_sleep();
spin_lock_bh(&sk->sk_lock.slock);
* The sk_lock has mutex_lock() semantics here:
*/
mutex_acquire(&sk->sk_lock.dep_map, 0, 0, _RET_IP_);
+ __acquire(&sk->sk_lock.slock);
local_bh_enable();
return true;
}
if (__inet_inherit_port(sk, newsk) < 0)
goto put_and_exit;
- *own_req = inet_ehash_nolisten(newsk, req_to_sk(req_unhash));
+ *own_req = inet_ehash_nolisten(newsk, req_to_sk(req_unhash), NULL);
if (*own_req)
ireq->ireq_opt = NULL;
else
dccp_done(newsk);
goto out;
}
- *own_req = inet_ehash_nolisten(newsk, req_to_sk(req_unhash));
+ *own_req = inet_ehash_nolisten(newsk, req_to_sk(req_unhash), NULL);
/* Clone pktoptions received with SYN, if we own the req */
if (*own_req && ireq->pktopts) {
newnp->pktoptions = skb_clone(ireq->pktopts, GFP_ATOMIC);
ifa->ifa_dev = dn_db;
if (tb[IFA_LABEL])
- nla_strlcpy(ifa->ifa_label, tb[IFA_LABEL], IFNAMSIZ);
+ nla_strscpy(ifa->ifa_label, tb[IFA_LABEL], IFNAMSIZ);
else
memcpy(ifa->ifa_label, dev->name, IFNAMSIZ);
Say Y or M if you want to enable support for tagging frames for the
Lantiq / Intel GSWIP switches.
+config NET_DSA_TAG_DSA_COMMON
+ tristate
+
config NET_DSA_TAG_DSA
tristate "Tag driver for Marvell switches using DSA headers"
+ select NET_DSA_TAG_DSA_COMMON
help
Say Y or M if you want to enable support for tagging frames for the
Marvell switches which use DSA headers.
config NET_DSA_TAG_EDSA
tristate "Tag driver for Marvell switches using EtherType DSA headers"
+ select NET_DSA_TAG_DSA_COMMON
help
Say Y or M if you want to enable support for tagging frames for the
Marvell switches which use EtherType DSA headers.
obj-$(CONFIG_NET_DSA_TAG_8021Q) += tag_8021q.o
obj-$(CONFIG_NET_DSA_TAG_AR9331) += tag_ar9331.o
obj-$(CONFIG_NET_DSA_TAG_BRCM_COMMON) += tag_brcm.o
-obj-$(CONFIG_NET_DSA_TAG_DSA) += tag_dsa.o
-obj-$(CONFIG_NET_DSA_TAG_EDSA) += tag_edsa.o
+obj-$(CONFIG_NET_DSA_TAG_DSA_COMMON) += tag_dsa.o
obj-$(CONFIG_NET_DSA_TAG_GSWIP) += tag_gswip.o
obj-$(CONFIG_NET_DSA_TAG_HELLCREEK) += tag_hellcreek.o
obj-$(CONFIG_NET_DSA_TAG_KSZ) += tag_ksz.o
if (!clone)
return;
- DSA_SKB_CB(skb)->clone = clone;
-
- if (ds->ops->port_txtstamp(ds, p->dp->index, clone, type))
+ if (ds->ops->port_txtstamp(ds, p->dp->index, clone, type)) {
+ DSA_SKB_CB(skb)->clone = clone;
return;
+ }
kfree_skb(clone);
}
// SPDX-License-Identifier: GPL-2.0+
/*
- * net/dsa/tag_dsa.c - (Non-ethertype) DSA tagging
+ * Regular and Ethertype DSA tagging
* Copyright (c) 2008-2009 Marvell Semiconductor
+ *
+ * Regular DSA
+ * -----------
+
+ * For untagged (in 802.1Q terms) packets, the switch will splice in
+ * the tag between the SA and the ethertype of the original
+ * packet. Tagged frames will instead have their outermost .1Q tag
+ * converted to a DSA tag. It expects the same layout when receiving
+ * packets from the CPU.
+ *
+ * Example:
+ *
+ * .----.----.----.---------
+ * Pu: | DA | SA | ET | Payload ...
+ * '----'----'----'---------
+ * 6 6 2 N
+ * .----.----.--------.-----.----.---------
+ * Pt: | DA | SA | 0x8100 | TCI | ET | Payload ...
+ * '----'----'--------'-----'----'---------
+ * 6 6 2 2 2 N
+ * .----.----.-----.----.---------
+ * Pd: | DA | SA | DSA | ET | Payload ...
+ * '----'----'-----'----'---------
+ * 6 6 4 2 N
+ *
+ * No matter if a packet is received untagged (Pu) or tagged (Pt),
+ * they will both have the same layout (Pd) when they are sent to the
+ * CPU. This is done by ignoring 802.3, replacing the ethertype field
+ * with more metadata, among which is a bit to signal if the original
+ * packet was tagged or not.
+ *
+ * Ethertype DSA
+ * -------------
+ * Uses the exact same tag format as regular DSA, but also includes a
+ * proper ethertype field (which the mv88e6xxx driver sets to
+ * ETH_P_EDSA/0xdada) followed by two zero bytes:
+ *
+ * .----.----.--------.--------.-----.----.---------
+ * | DA | SA | 0xdada | 0x0000 | DSA | ET | Payload ...
+ * '----'----'--------'--------'-----'----'---------
+ * 6 6 2 2 4 2 N
*/
#include <linux/etherdevice.h>
#define DSA_HLEN 4
-static struct sk_buff *dsa_xmit(struct sk_buff *skb, struct net_device *dev)
+/**
+ * enum dsa_cmd - DSA Command
+ * @DSA_CMD_TO_CPU: Set on packets that were trapped or mirrored to
+ * the CPU port. This is needed to implement control protocols,
+ * e.g. STP and LLDP, that must not allow those control packets to
+ * be switched according to the normal rules.
+ * @DSA_CMD_FROM_CPU: Used by the CPU to send a packet to a specific
+ * port, ignoring all the barriers that the switch normally
+ * enforces (VLANs, STP port states etc.). No source address
+ * learning takes place. "sudo send packet"
+ * @DSA_CMD_TO_SNIFFER: Set on the copies of packets that matched some
+ * user configured ingress or egress monitor criteria. These are
+ * forwarded by the switch tree to the user configured ingress or
+ * egress monitor port, which can be set to the CPU port or a
+ * regular port. If the destination is a regular port, the tag
+ * will be removed before egressing the port. If the destination
+ * is the CPU port, the tag will not be removed.
+ * @DSA_CMD_FORWARD: This tag is used on all bulk traffic passing
+ * through the switch tree, including the flows that are directed
+ * towards the CPU. Its device/port tuple encodes the original
+ * source port on which the packet ingressed. It can also be used
+ * on transmit by the CPU to defer the forwarding decision to the
+ * hardware, based on the current config of PVT/VTU/ATU
+ * etc. Source address learning takes places if enabled on the
+ * receiving DSA/CPU port.
+ */
+enum dsa_cmd {
+ DSA_CMD_TO_CPU = 0,
+ DSA_CMD_FROM_CPU = 1,
+ DSA_CMD_TO_SNIFFER = 2,
+ DSA_CMD_FORWARD = 3
+};
+
+/**
+ * enum dsa_code - TO_CPU Code
+ *
+ * @DSA_CODE_MGMT_TRAP: DA was classified as a management
+ * address. Typical examples include STP BPDUs and LLDP.
+ * @DSA_CODE_FRAME2REG: Response to a "remote management" request.
+ * @DSA_CODE_IGMP_MLD_TRAP: IGMP/MLD signaling.
+ * @DSA_CODE_POLICY_TRAP: Frame matched some policy configuration on
+ * the device. Typical examples are matching on DA/SA/VID and DHCP
+ * snooping.
+ * @DSA_CODE_ARP_MIRROR: The name says it all really.
+ * @DSA_CODE_POLICY_MIRROR: Same as @DSA_CODE_POLICY_TRAP, but the
+ * particular policy was set to trigger a mirror instead of a
+ * trap.
+ * @DSA_CODE_RESERVED_6: Unused on all devices up to at least 6393X.
+ * @DSA_CODE_RESERVED_7: Unused on all devices up to at least 6393X.
+ *
+ * A 3-bit code is used to relay why a particular frame was sent to
+ * the CPU. We only use this to determine if the packet was mirrored
+ * or trapped, i.e. whether the packet has been forwarded by hardware
+ * or not.
+ *
+ * This is the superset of all possible codes. Any particular device
+ * may only implement a subset.
+ */
+enum dsa_code {
+ DSA_CODE_MGMT_TRAP = 0,
+ DSA_CODE_FRAME2REG = 1,
+ DSA_CODE_IGMP_MLD_TRAP = 2,
+ DSA_CODE_POLICY_TRAP = 3,
+ DSA_CODE_ARP_MIRROR = 4,
+ DSA_CODE_POLICY_MIRROR = 5,
+ DSA_CODE_RESERVED_6 = 6,
+ DSA_CODE_RESERVED_7 = 7
+};
+
+static struct sk_buff *dsa_xmit_ll(struct sk_buff *skb, struct net_device *dev,
+ u8 extra)
{
struct dsa_port *dp = dsa_slave_to_port(dev);
u8 *dsa_header;
- /*
- * Convert the outermost 802.1q tag to a DSA tag for tagged
- * packets, or insert a DSA tag between the addresses and
- * the ethertype field for untagged packets.
- */
if (skb->protocol == htons(ETH_P_8021Q)) {
- /*
- * Construct tagged FROM_CPU DSA tag from 802.1q tag.
- */
- dsa_header = skb->data + 2 * ETH_ALEN;
- dsa_header[0] = 0x60 | dp->ds->index;
+ if (extra) {
+ skb_push(skb, extra);
+ memmove(skb->data, skb->data + extra, 2 * ETH_ALEN);
+ }
+
+ /* Construct tagged FROM_CPU DSA tag from 802.1Q tag. */
+ dsa_header = skb->data + 2 * ETH_ALEN + extra;
+ dsa_header[0] = (DSA_CMD_FROM_CPU << 6) | 0x20 | dp->ds->index;
dsa_header[1] = dp->index << 3;
- /*
- * Move CFI field from byte 2 to byte 1.
- */
+ /* Move CFI field from byte 2 to byte 1. */
if (dsa_header[2] & 0x10) {
dsa_header[1] |= 0x01;
dsa_header[2] &= ~0x10;
}
} else {
- skb_push(skb, DSA_HLEN);
-
- memmove(skb->data, skb->data + DSA_HLEN, 2 * ETH_ALEN);
+ skb_push(skb, DSA_HLEN + extra);
+ memmove(skb->data, skb->data + DSA_HLEN + extra, 2 * ETH_ALEN);
- /*
- * Construct untagged FROM_CPU DSA tag.
- */
- dsa_header = skb->data + 2 * ETH_ALEN;
- dsa_header[0] = 0x40 | dp->ds->index;
+ /* Construct untagged FROM_CPU DSA tag. */
+ dsa_header = skb->data + 2 * ETH_ALEN + extra;
+ dsa_header[0] = (DSA_CMD_FROM_CPU << 6) | dp->ds->index;
dsa_header[1] = dp->index << 3;
dsa_header[2] = 0x00;
dsa_header[3] = 0x00;
return skb;
}
-static struct sk_buff *dsa_rcv(struct sk_buff *skb, struct net_device *dev,
- struct packet_type *pt)
+static struct sk_buff *dsa_rcv_ll(struct sk_buff *skb, struct net_device *dev,
+ u8 extra)
{
+ int source_device, source_port;
+ enum dsa_code code;
+ enum dsa_cmd cmd;
u8 *dsa_header;
- int source_device;
- int source_port;
- if (unlikely(!pskb_may_pull(skb, DSA_HLEN)))
- return NULL;
-
- /*
- * The ethertype field is part of the DSA header.
- */
+ /* The ethertype field is part of the DSA header. */
dsa_header = skb->data - 2;
- /*
- * Check that frame type is either TO_CPU or FORWARD.
- */
- if ((dsa_header[0] & 0xc0) != 0x00 && (dsa_header[0] & 0xc0) != 0xc0)
+ cmd = dsa_header[0] >> 6;
+ switch (cmd) {
+ case DSA_CMD_FORWARD:
+ skb->offload_fwd_mark = 1;
+ break;
+
+ case DSA_CMD_TO_CPU:
+ code = (dsa_header[1] & 0x6) | ((dsa_header[2] >> 4) & 1);
+
+ switch (code) {
+ case DSA_CODE_FRAME2REG:
+ /* Remote management is not implemented yet,
+ * drop.
+ */
+ return NULL;
+ case DSA_CODE_ARP_MIRROR:
+ case DSA_CODE_POLICY_MIRROR:
+ /* Mark mirrored packets to notify any upper
+ * device (like a bridge) that forwarding has
+ * already been done by hardware.
+ */
+ skb->offload_fwd_mark = 1;
+ break;
+ case DSA_CODE_MGMT_TRAP:
+ case DSA_CODE_IGMP_MLD_TRAP:
+ case DSA_CODE_POLICY_TRAP:
+ /* Traps have, by definition, not been
+ * forwarded by hardware, so don't mark them.
+ */
+ break;
+ default:
+ /* Reserved code, this could be anything. Drop
+ * seems like the safest option.
+ */
+ return NULL;
+ }
+
+ break;
+
+ default:
return NULL;
+ }
- /*
- * Determine source device and port.
- */
source_device = dsa_header[0] & 0x1f;
source_port = (dsa_header[1] >> 3) & 0x1f;
if (!skb->dev)
return NULL;
- /*
- * Convert the DSA header to an 802.1q header if the 'tagged'
- * bit in the DSA header is set. If the 'tagged' bit is clear,
- * delete the DSA header entirely.
+ /* If the 'tagged' bit is set; convert the DSA tag to a 802.1Q
+ * tag, and delete the ethertype (extra) if applicable. If the
+ * 'tagged' bit is cleared; delete the DSA tag, and ethertype
+ * if applicable.
*/
if (dsa_header[0] & 0x20) {
u8 new_header[4];
- /*
- * Insert 802.1q ethertype and copy the VLAN-related
+ /* Insert 802.1Q ethertype and copy the VLAN-related
* fields, but clear the bit that will hold CFI (since
* DSA uses that bit location for another purpose).
*/
new_header[2] = dsa_header[2] & ~0x10;
new_header[3] = dsa_header[3];
- /*
- * Move CFI bit from its place in the DSA header to
- * its 802.1q-designated place.
+ /* Move CFI bit from its place in the DSA header to
+ * its 802.1Q-designated place.
*/
if (dsa_header[1] & 0x01)
new_header[2] |= 0x10;
- /*
- * Update packet checksum if skb is CHECKSUM_COMPLETE.
- */
+ /* Update packet checksum if skb is CHECKSUM_COMPLETE. */
if (skb->ip_summed == CHECKSUM_COMPLETE) {
__wsum c = skb->csum;
c = csum_add(c, csum_partial(new_header + 2, 2, 0));
}
memcpy(dsa_header, new_header, DSA_HLEN);
+
+ if (extra)
+ memmove(skb->data - ETH_HLEN,
+ skb->data - ETH_HLEN - extra,
+ 2 * ETH_ALEN);
} else {
- /*
- * Remove DSA tag and update checksum.
- */
skb_pull_rcsum(skb, DSA_HLEN);
memmove(skb->data - ETH_HLEN,
- skb->data - ETH_HLEN - DSA_HLEN,
+ skb->data - ETH_HLEN - DSA_HLEN - extra,
2 * ETH_ALEN);
}
- skb->offload_fwd_mark = 1;
-
return skb;
}
+#if IS_ENABLED(CONFIG_NET_DSA_TAG_DSA)
+
+static struct sk_buff *dsa_xmit(struct sk_buff *skb, struct net_device *dev)
+{
+ return dsa_xmit_ll(skb, dev, 0);
+}
+
+static struct sk_buff *dsa_rcv(struct sk_buff *skb, struct net_device *dev,
+ struct packet_type *pt)
+{
+ if (unlikely(!pskb_may_pull(skb, DSA_HLEN)))
+ return NULL;
+
+ return dsa_rcv_ll(skb, dev, 0);
+}
+
static const struct dsa_device_ops dsa_netdev_ops = {
- .name = "dsa",
- .proto = DSA_TAG_PROTO_DSA,
- .xmit = dsa_xmit,
- .rcv = dsa_rcv,
+ .name = "dsa",
+ .proto = DSA_TAG_PROTO_DSA,
+ .xmit = dsa_xmit,
+ .rcv = dsa_rcv,
.overhead = DSA_HLEN,
};
-MODULE_LICENSE("GPL");
+DSA_TAG_DRIVER(dsa_netdev_ops);
MODULE_ALIAS_DSA_TAG_DRIVER(DSA_TAG_PROTO_DSA);
+#endif /* CONFIG_NET_DSA_TAG_DSA */
+
+#if IS_ENABLED(CONFIG_NET_DSA_TAG_EDSA)
+
+#define EDSA_HLEN 8
+
+static struct sk_buff *edsa_xmit(struct sk_buff *skb, struct net_device *dev)
+{
+ u8 *edsa_header;
+
+ skb = dsa_xmit_ll(skb, dev, EDSA_HLEN - DSA_HLEN);
+ if (!skb)
+ return NULL;
-module_dsa_tag_driver(dsa_netdev_ops);
+ edsa_header = skb->data + 2 * ETH_ALEN;
+ edsa_header[0] = (ETH_P_EDSA >> 8) & 0xff;
+ edsa_header[1] = ETH_P_EDSA & 0xff;
+ edsa_header[2] = 0x00;
+ edsa_header[3] = 0x00;
+ return skb;
+}
+
+static struct sk_buff *edsa_rcv(struct sk_buff *skb, struct net_device *dev,
+ struct packet_type *pt)
+{
+ if (unlikely(!pskb_may_pull(skb, EDSA_HLEN)))
+ return NULL;
+
+ skb_pull_rcsum(skb, EDSA_HLEN - DSA_HLEN);
+
+ return dsa_rcv_ll(skb, dev, EDSA_HLEN - DSA_HLEN);
+}
+
+static const struct dsa_device_ops edsa_netdev_ops = {
+ .name = "edsa",
+ .proto = DSA_TAG_PROTO_EDSA,
+ .xmit = edsa_xmit,
+ .rcv = edsa_rcv,
+ .overhead = EDSA_HLEN,
+};
+
+DSA_TAG_DRIVER(edsa_netdev_ops);
+MODULE_ALIAS_DSA_TAG_DRIVER(DSA_TAG_PROTO_EDSA);
+#endif /* CONFIG_NET_DSA_TAG_EDSA */
+
+static struct dsa_tag_driver *dsa_tag_drivers[] = {
+#if IS_ENABLED(CONFIG_NET_DSA_TAG_DSA)
+ &DSA_TAG_DRIVER_NAME(dsa_netdev_ops),
+#endif
+#if IS_ENABLED(CONFIG_NET_DSA_TAG_EDSA)
+ &DSA_TAG_DRIVER_NAME(edsa_netdev_ops),
+#endif
+};
+
+module_dsa_tag_drivers(dsa_tag_drivers);
+
+MODULE_LICENSE("GPL");
+++ /dev/null
-// SPDX-License-Identifier: GPL-2.0+
-/*
- * net/dsa/tag_edsa.c - Ethertype DSA tagging
- * Copyright (c) 2008-2009 Marvell Semiconductor
- */
-
-#include <linux/etherdevice.h>
-#include <linux/list.h>
-#include <linux/slab.h>
-
-#include "dsa_priv.h"
-
-#define DSA_HLEN 4
-#define EDSA_HLEN 8
-
-#define FRAME_TYPE_TO_CPU 0x00
-#define FRAME_TYPE_FORWARD 0x03
-
-#define TO_CPU_CODE_MGMT_TRAP 0x00
-#define TO_CPU_CODE_FRAME2REG 0x01
-#define TO_CPU_CODE_IGMP_MLD_TRAP 0x02
-#define TO_CPU_CODE_POLICY_TRAP 0x03
-#define TO_CPU_CODE_ARP_MIRROR 0x04
-#define TO_CPU_CODE_POLICY_MIRROR 0x05
-
-static struct sk_buff *edsa_xmit(struct sk_buff *skb, struct net_device *dev)
-{
- struct dsa_port *dp = dsa_slave_to_port(dev);
- u8 *edsa_header;
-
- /*
- * Convert the outermost 802.1q tag to a DSA tag and prepend
- * a DSA ethertype field is the packet is tagged, or insert
- * a DSA ethertype plus DSA tag between the addresses and the
- * current ethertype field if the packet is untagged.
- */
- if (skb->protocol == htons(ETH_P_8021Q)) {
- skb_push(skb, DSA_HLEN);
-
- memmove(skb->data, skb->data + DSA_HLEN, 2 * ETH_ALEN);
-
- /*
- * Construct tagged FROM_CPU DSA tag from 802.1q tag.
- */
- edsa_header = skb->data + 2 * ETH_ALEN;
- edsa_header[0] = (ETH_P_EDSA >> 8) & 0xff;
- edsa_header[1] = ETH_P_EDSA & 0xff;
- edsa_header[2] = 0x00;
- edsa_header[3] = 0x00;
- edsa_header[4] = 0x60 | dp->ds->index;
- edsa_header[5] = dp->index << 3;
-
- /*
- * Move CFI field from byte 6 to byte 5.
- */
- if (edsa_header[6] & 0x10) {
- edsa_header[5] |= 0x01;
- edsa_header[6] &= ~0x10;
- }
- } else {
- skb_push(skb, EDSA_HLEN);
-
- memmove(skb->data, skb->data + EDSA_HLEN, 2 * ETH_ALEN);
-
- /*
- * Construct untagged FROM_CPU DSA tag.
- */
- edsa_header = skb->data + 2 * ETH_ALEN;
- edsa_header[0] = (ETH_P_EDSA >> 8) & 0xff;
- edsa_header[1] = ETH_P_EDSA & 0xff;
- edsa_header[2] = 0x00;
- edsa_header[3] = 0x00;
- edsa_header[4] = 0x40 | dp->ds->index;
- edsa_header[5] = dp->index << 3;
- edsa_header[6] = 0x00;
- edsa_header[7] = 0x00;
- }
-
- return skb;
-}
-
-static struct sk_buff *edsa_rcv(struct sk_buff *skb, struct net_device *dev,
- struct packet_type *pt)
-{
- u8 *edsa_header;
- int frame_type;
- int code;
- int source_device;
- int source_port;
-
- if (unlikely(!pskb_may_pull(skb, EDSA_HLEN)))
- return NULL;
-
- /*
- * Skip the two null bytes after the ethertype.
- */
- edsa_header = skb->data + 2;
-
- /*
- * Check that frame type is either TO_CPU or FORWARD.
- */
- frame_type = edsa_header[0] >> 6;
-
- switch (frame_type) {
- case FRAME_TYPE_TO_CPU:
- code = (edsa_header[1] & 0x6) | ((edsa_header[2] >> 4) & 1);
-
- /*
- * Mark the frame to never egress on any port of the same switch
- * unless it's a trapped IGMP/MLD packet, in which case the
- * bridge might want to forward it.
- */
- if (code != TO_CPU_CODE_IGMP_MLD_TRAP)
- skb->offload_fwd_mark = 1;
-
- break;
-
- case FRAME_TYPE_FORWARD:
- skb->offload_fwd_mark = 1;
- break;
-
- default:
- return NULL;
- }
-
- /*
- * Determine source device and port.
- */
- source_device = edsa_header[0] & 0x1f;
- source_port = (edsa_header[1] >> 3) & 0x1f;
-
- skb->dev = dsa_master_find_slave(dev, source_device, source_port);
- if (!skb->dev)
- return NULL;
-
- /*
- * If the 'tagged' bit is set, convert the DSA tag to a 802.1q
- * tag and delete the ethertype part. If the 'tagged' bit is
- * clear, delete the ethertype and the DSA tag parts.
- */
- if (edsa_header[0] & 0x20) {
- u8 new_header[4];
-
- /*
- * Insert 802.1q ethertype and copy the VLAN-related
- * fields, but clear the bit that will hold CFI (since
- * DSA uses that bit location for another purpose).
- */
- new_header[0] = (ETH_P_8021Q >> 8) & 0xff;
- new_header[1] = ETH_P_8021Q & 0xff;
- new_header[2] = edsa_header[2] & ~0x10;
- new_header[3] = edsa_header[3];
-
- /*
- * Move CFI bit from its place in the DSA header to
- * its 802.1q-designated place.
- */
- if (edsa_header[1] & 0x01)
- new_header[2] |= 0x10;
-
- skb_pull_rcsum(skb, DSA_HLEN);
-
- /*
- * Update packet checksum if skb is CHECKSUM_COMPLETE.
- */
- if (skb->ip_summed == CHECKSUM_COMPLETE) {
- __wsum c = skb->csum;
- c = csum_add(c, csum_partial(new_header + 2, 2, 0));
- c = csum_sub(c, csum_partial(edsa_header + 2, 2, 0));
- skb->csum = c;
- }
-
- memcpy(edsa_header, new_header, DSA_HLEN);
-
- memmove(skb->data - ETH_HLEN,
- skb->data - ETH_HLEN - DSA_HLEN,
- 2 * ETH_ALEN);
- } else {
- /*
- * Remove DSA tag and update checksum.
- */
- skb_pull_rcsum(skb, EDSA_HLEN);
- memmove(skb->data - ETH_HLEN,
- skb->data - ETH_HLEN - EDSA_HLEN,
- 2 * ETH_ALEN);
- }
-
- return skb;
-}
-
-static const struct dsa_device_ops edsa_netdev_ops = {
- .name = "edsa",
- .proto = DSA_TAG_PROTO_EDSA,
- .xmit = edsa_xmit,
- .rcv = edsa_rcv,
- .overhead = EDSA_HLEN,
-};
-
-MODULE_LICENSE("GPL");
-MODULE_ALIAS_DSA_TAG_DRIVER(DSA_TAG_PROTO_EDSA);
-
-module_dsa_tag_driver(edsa_netdev_ops);
* Based on tag_ksz.c.
*/
-#include <linux/etherdevice.h>
-#include <linux/list.h>
-#include <linux/slab.h>
+#include <linux/skbuff.h>
#include <net/dsa.h>
#include "dsa_priv.h"
EXPORT_SYMBOL(eth_header_cache_update);
/**
- * eth_header_parser_protocol - extract protocol from L2 header
+ * eth_header_parse_protocol - extract protocol from L2 header
* @skb: packet to extract protocol from
*/
__be16 eth_header_parse_protocol(const struct sk_buff *skb)
EXPORT_SYMBOL(eth_platform_get_mac_address);
/**
- * Obtain the MAC address from an nvmem cell named 'mac-address' associated
- * with given device.
+ * nvmem_get_mac_address - Obtain the MAC address from an nvmem cell named
+ * 'mac-address' associated with given device.
*
* @dev: Device with which the mac-address cell is associated.
* @addrbuf: Buffer to which the MAC address will be copied on success.
if (info->attrs[IEEE802154_ATTR_DEV_NAME]) {
char name[IFNAMSIZ + 1];
- nla_strlcpy(name, info->attrs[IEEE802154_ATTR_DEV_NAME],
+ nla_strscpy(name, info->attrs[IEEE802154_ATTR_DEV_NAME],
sizeof(name));
dev = dev_get_by_name(&init_net, name);
} else if (info->attrs[IEEE802154_ATTR_DEV_INDEX]) {
static void arp_solicit(struct neighbour *neigh, struct sk_buff *skb);
static void arp_error_report(struct neighbour *neigh, struct sk_buff *skb);
static void parp_redo(struct sk_buff *skb);
+static int arp_is_multicast(const void *pkey);
static const struct neigh_ops arp_generic_ops = {
.family = AF_INET,
.key_eq = arp_key_eq,
.constructor = arp_constructor,
.proxy_redo = parp_redo,
+ .is_multicast = arp_is_multicast,
.id = "arp_cache",
.parms = {
.tbl = &arp_tbl,
arp_process(dev_net(skb->dev), NULL, skb);
}
+static int arp_is_multicast(const void *pkey)
+{
+ return ipv4_is_multicast(*((__be32 *)pkey));
+}
/*
* Receive an arp request from the device layer.
ifa->ifa_broadcast = nla_get_in_addr(tb[IFA_BROADCAST]);
if (tb[IFA_LABEL])
- nla_strlcpy(ifa->ifa_label, tb[IFA_LABEL], IFNAMSIZ);
+ nla_strscpy(ifa->ifa_label, tb[IFA_LABEL], IFNAMSIZ);
else
memcpy(ifa->ifa_label, dev->name, IFNAMSIZ);
cfg->fc_gw4 = *((__be32 *)via->rtvia_addr);
break;
case AF_INET6:
-#ifdef CONFIG_IPV6
+#if IS_ENABLED(CONFIG_IPV6)
if (alen != sizeof(struct in6_addr)) {
NL_SET_ERR_MSG(extack, "Invalid IPv6 address in RTA_VIA");
return -EINVAL;
char tmp[TCP_CA_NAME_MAX];
bool ecn_ca = false;
- nla_strlcpy(tmp, nla, sizeof(tmp));
+ nla_strscpy(tmp, nla, sizeof(tmp));
val = tcp_ca_get_key_by_name(fi->fib_net, tmp, &ecn_ca);
} else {
if (nla_len(nla) != sizeof(u32))
timer_setup(&req->rsk_timer, reqsk_timer_handler, TIMER_PINNED);
mod_timer(&req->rsk_timer, jiffies + timeout);
- inet_ehash_insert(req_to_sk(req), NULL);
+ inet_ehash_insert(req_to_sk(req), NULL, NULL);
/* before letting lookups find us, make sure all req fields
* are committed to memory and refcnt initialized.
*/
r->idiag_inode = 0;
if (net_admin && nla_put_u32(skb, INET_DIAG_MARK,
- inet_rsk(reqsk)->ir_mark))
+ inet_rsk(reqsk)->ir_mark)) {
+ nlmsg_cancel(skb, nlh);
return -EMSGSIZE;
+ }
nlmsg_end(skb, nlh);
return 0;
#include <net/addrconf.h>
#include <net/inet_connection_sock.h>
#include <net/inet_hashtables.h>
+#if IS_ENABLED(CONFIG_IPV6)
+#include <net/inet6_hashtables.h>
+#endif
#include <net/secure_seq.h>
#include <net/ip.h>
#include <net/tcp.h>
inet->inet_dport);
}
-/* insert a socket into ehash, and eventually remove another one
- * (The another one can be a SYN_RECV or TIMEWAIT
+/* Searches for an exsiting socket in the ehash bucket list.
+ * Returns true if found, false otherwise.
*/
-bool inet_ehash_insert(struct sock *sk, struct sock *osk)
+static bool inet_ehash_lookup_by_sk(struct sock *sk,
+ struct hlist_nulls_head *list)
+{
+ const __portpair ports = INET_COMBINED_PORTS(sk->sk_dport, sk->sk_num);
+ const int sdif = sk->sk_bound_dev_if;
+ const int dif = sk->sk_bound_dev_if;
+ const struct hlist_nulls_node *node;
+ struct net *net = sock_net(sk);
+ struct sock *esk;
+
+ INET_ADDR_COOKIE(acookie, sk->sk_daddr, sk->sk_rcv_saddr);
+
+ sk_nulls_for_each_rcu(esk, node, list) {
+ if (esk->sk_hash != sk->sk_hash)
+ continue;
+ if (sk->sk_family == AF_INET) {
+ if (unlikely(INET_MATCH(esk, net, acookie,
+ sk->sk_daddr,
+ sk->sk_rcv_saddr,
+ ports, dif, sdif))) {
+ return true;
+ }
+ }
+#if IS_ENABLED(CONFIG_IPV6)
+ else if (sk->sk_family == AF_INET6) {
+ if (unlikely(INET6_MATCH(esk, net,
+ &sk->sk_v6_daddr,
+ &sk->sk_v6_rcv_saddr,
+ ports, dif, sdif))) {
+ return true;
+ }
+ }
+#endif
+ }
+ return false;
+}
+
+/* Insert a socket into ehash, and eventually remove another one
+ * (The another one can be a SYN_RECV or TIMEWAIT)
+ * If an existing socket already exists, socket sk is not inserted,
+ * and sets found_dup_sk parameter to true.
+ */
+bool inet_ehash_insert(struct sock *sk, struct sock *osk, bool *found_dup_sk)
{
struct inet_hashinfo *hashinfo = sk->sk_prot->h.hashinfo;
struct hlist_nulls_head *list;
if (osk) {
WARN_ON_ONCE(sk->sk_hash != osk->sk_hash);
ret = sk_nulls_del_node_init_rcu(osk);
+ } else if (found_dup_sk) {
+ *found_dup_sk = inet_ehash_lookup_by_sk(sk, list);
+ if (*found_dup_sk)
+ ret = false;
}
+
if (ret)
__sk_nulls_add_node_rcu(sk, list);
+
spin_unlock(lock);
+
return ret;
}
-bool inet_ehash_nolisten(struct sock *sk, struct sock *osk)
+bool inet_ehash_nolisten(struct sock *sk, struct sock *osk, bool *found_dup_sk)
{
- bool ok = inet_ehash_insert(sk, osk);
+ bool ok = inet_ehash_insert(sk, osk, found_dup_sk);
if (ok) {
sock_prot_inuse_add(sock_net(sk), sk->sk_prot, 1);
int err = 0;
if (sk->sk_state != TCP_LISTEN) {
- inet_ehash_nolisten(sk, osk);
+ inet_ehash_nolisten(sk, osk, NULL);
return 0;
}
WARN_ON(!sk_unhashed(sk));
tb = inet_csk(sk)->icsk_bind_hash;
spin_lock_bh(&head->lock);
if (sk_head(&tb->owners) == sk && !sk->sk_bind_node.next) {
- inet_ehash_nolisten(sk, NULL);
+ inet_ehash_nolisten(sk, NULL, NULL);
spin_unlock_bh(&head->lock);
return 0;
}
inet_bind_hash(sk, tb, port);
if (sk_unhashed(sk)) {
inet_sk(sk)->inet_sport = htons(port);
- inet_ehash_nolisten(sk, (struct sock *)tw);
+ inet_ehash_nolisten(sk, (struct sock *)tw, NULL);
}
if (tw)
inet_twsk_bind_unhash(tw, hinfo);
if (type == RTAX_CC_ALGO) {
char tmp[TCP_CA_NAME_MAX];
- nla_strlcpy(tmp, nla, sizeof(tmp));
+ nla_strscpy(tmp, nla, sizeof(tmp));
val = tcp_ca_get_key_by_name(net, tmp, &ecn_ca);
if (val == TCP_CA_UNSPEC) {
NL_SET_ERR_MSG(extack, "Unknown tcp congestion algorithm");
if (rt->dst.dev &&
nla_put_u32(skb, RTA_OIF, rt->dst.dev->ifindex))
goto nla_put_failure;
+ if (rt->dst.lwtstate &&
+ lwtunnel_fill_encap(skb, rt->dst.lwtstate, RTA_ENCAP, RTA_ENCAP_TYPE) < 0)
+ goto nla_put_failure;
#ifdef CONFIG_IP_ROUTE_CLASSID
if (rt->dst.tclassid &&
nla_put_u32(skb, RTA_FLOW, rt->dst.tclassid))
fl4.daddr = dst;
fl4.saddr = src;
- fl4.flowi4_tos = rtm->rtm_tos;
+ fl4.flowi4_tos = rtm->rtm_tos & IPTOS_RT_MASK;
fl4.flowi4_oif = tb[RTA_OIF] ? nla_get_u32(tb[RTA_OIF]) : 0;
fl4.flowi4_mark = mark;
fl4.flowi4_uid = uid;
fl4.flowi4_iif = iif; /* for rt_fill_info */
skb->dev = dev;
skb->mark = mark;
- err = ip_route_input_rcu(skb, dst, src, rtm->rtm_tos,
- dev, &res);
+ err = ip_route_input_rcu(skb, dst, src,
+ rtm->rtm_tos & IPTOS_RT_MASK, dev,
+ &res);
rt = skb_rtable(skb);
if (err == 0 && rt->dst.error)
* importantly be able to generate EPOLLOUT for Edge Trigger epoll()
* users.
*/
-static void tcp_remove_empty_skb(struct sock *sk, struct sk_buff *skb)
+void tcp_remove_empty_skb(struct sock *sk, struct sk_buff *skb)
{
if (skb && !skb->len) {
tcp_unlink_write_queue(skb, sk);
}
}
+struct sk_buff *tcp_build_frag(struct sock *sk, int size_goal, int flags,
+ struct page *page, int offset, size_t *size)
+{
+ struct sk_buff *skb = tcp_write_queue_tail(sk);
+ struct tcp_sock *tp = tcp_sk(sk);
+ bool can_coalesce;
+ int copy, i;
+
+ if (!skb || (copy = size_goal - skb->len) <= 0 ||
+ !tcp_skb_can_collapse_to(skb)) {
+new_segment:
+ if (!sk_stream_memory_free(sk))
+ return NULL;
+
+ skb = sk_stream_alloc_skb(sk, 0, sk->sk_allocation,
+ tcp_rtx_and_write_queues_empty(sk));
+ if (!skb)
+ return NULL;
+
+#ifdef CONFIG_TLS_DEVICE
+ skb->decrypted = !!(flags & MSG_SENDPAGE_DECRYPTED);
+#endif
+ skb_entail(sk, skb);
+ copy = size_goal;
+ }
+
+ if (copy > *size)
+ copy = *size;
+
+ i = skb_shinfo(skb)->nr_frags;
+ can_coalesce = skb_can_coalesce(skb, i, page, offset);
+ if (!can_coalesce && i >= sysctl_max_skb_frags) {
+ tcp_mark_push(tp, skb);
+ goto new_segment;
+ }
+ if (!sk_wmem_schedule(sk, copy))
+ return NULL;
+
+ if (can_coalesce) {
+ skb_frag_size_add(&skb_shinfo(skb)->frags[i - 1], copy);
+ } else {
+ get_page(page);
+ skb_fill_page_desc(skb, i, page, offset, copy);
+ }
+
+ if (!(flags & MSG_NO_SHARED_FRAGS))
+ skb_shinfo(skb)->tx_flags |= SKBTX_SHARED_FRAG;
+
+ skb->len += copy;
+ skb->data_len += copy;
+ skb->truesize += copy;
+ sk_wmem_queued_add(sk, copy);
+ sk_mem_charge(sk, copy);
+ skb->ip_summed = CHECKSUM_PARTIAL;
+ WRITE_ONCE(tp->write_seq, tp->write_seq + copy);
+ TCP_SKB_CB(skb)->end_seq += copy;
+ tcp_skb_pcount_set(skb, 0);
+
+ *size = copy;
+ return skb;
+}
+
ssize_t do_tcp_sendpages(struct sock *sk, struct page *page, int offset,
size_t size, int flags)
{
goto out_err;
while (size > 0) {
- struct sk_buff *skb = tcp_write_queue_tail(sk);
- int copy, i;
- bool can_coalesce;
+ struct sk_buff *skb;
+ size_t copy = size;
- if (!skb || (copy = size_goal - skb->len) <= 0 ||
- !tcp_skb_can_collapse_to(skb)) {
-new_segment:
- if (!sk_stream_memory_free(sk))
- goto wait_for_space;
-
- skb = sk_stream_alloc_skb(sk, 0, sk->sk_allocation,
- tcp_rtx_and_write_queues_empty(sk));
- if (!skb)
- goto wait_for_space;
-
-#ifdef CONFIG_TLS_DEVICE
- skb->decrypted = !!(flags & MSG_SENDPAGE_DECRYPTED);
-#endif
- skb_entail(sk, skb);
- copy = size_goal;
- }
-
- if (copy > size)
- copy = size;
-
- i = skb_shinfo(skb)->nr_frags;
- can_coalesce = skb_can_coalesce(skb, i, page, offset);
- if (!can_coalesce && i >= sysctl_max_skb_frags) {
- tcp_mark_push(tp, skb);
- goto new_segment;
- }
- if (!sk_wmem_schedule(sk, copy))
+ skb = tcp_build_frag(sk, size_goal, flags, page, offset, ©);
+ if (!skb)
goto wait_for_space;
- if (can_coalesce) {
- skb_frag_size_add(&skb_shinfo(skb)->frags[i - 1], copy);
- } else {
- get_page(page);
- skb_fill_page_desc(skb, i, page, offset, copy);
- }
-
- if (!(flags & MSG_NO_SHARED_FRAGS))
- skb_shinfo(skb)->tx_flags |= SKBTX_SHARED_FRAG;
-
- skb->len += copy;
- skb->data_len += copy;
- skb->truesize += copy;
- sk_wmem_queued_add(sk, copy);
- sk_mem_charge(sk, copy);
- skb->ip_summed = CHECKSUM_PARTIAL;
- WRITE_ONCE(tp->write_seq, tp->write_seq + copy);
- TCP_SKB_CB(skb)->end_seq += copy;
- tcp_skb_pcount_set(skb, 0);
-
if (!copied)
TCP_SKB_CB(skb)->tcp_flags &= ~TCPHDR_PSH;
return too_many_orphans || out_of_socket_memory;
}
-void tcp_close(struct sock *sk, long timeout)
+void __tcp_close(struct sock *sk, long timeout)
{
struct sk_buff *skb;
int data_was_unread = 0;
int state;
- lock_sock(sk);
sk->sk_shutdown = SHUTDOWN_MASK;
if (sk->sk_state == TCP_LISTEN) {
out:
bh_unlock_sock(sk);
local_bh_enable();
+}
+
+void tcp_close(struct sock *sk, long timeout)
+{
+ lock_sock(sk);
+ __tcp_close(sk, timeout);
release_sock(sk);
sock_put(sk);
}
filter_expired = after(tcp_jiffies32,
bbr->min_rtt_stamp + bbr_min_rtt_win_sec * HZ);
if (rs->rtt_us >= 0 &&
- (rs->rtt_us <= bbr->min_rtt_us ||
+ (rs->rtt_us < bbr->min_rtt_us ||
(filter_expired && !rs->is_ack_delayed))) {
bbr->min_rtt_us = rs->rtt_us;
bbr->min_rtt_stamp = tcp_jiffies32;
{
struct iov_iter *iter = &msg->msg_iter;
int peek = flags & MSG_PEEK;
- int i, ret, copied = 0;
struct sk_msg *msg_rx;
+ int i, copied = 0;
msg_rx = list_first_entry_or_null(&psock->ingress_msg,
struct sk_msg, list);
page = sg_page(sge);
if (copied + copy > len)
copy = len - copied;
- ret = copy_page_to_iter(page, sge->offset, copy, iter);
- if (ret != copy) {
- msg_rx->sg.start = i;
- return -EFAULT;
- }
+ copy = copy_page_to_iter(page, sge->offset, copy, iter);
+ if (!copy)
+ return copied ? copied : -EFAULT;
copied += copy;
if (likely(!peek)) {
sge->offset += copy;
sge->length -= copy;
- sk_mem_uncharge(sk, copy);
+ if (!msg_rx->skb)
+ sk_mem_uncharge(sk, copy);
msg_rx->sg.size -= copy;
if (!sge->length) {
put_page(page);
}
} else {
+ /* Lets not optimize peek case if copy_page_to_iter
+ * didn't copy the entire length lets just break.
+ */
+ if (copy != sge->length)
+ return copied;
sk_msg_iter_var_next(i);
}
icsk->icsk_ca_setsockopt = 1;
memset(icsk->icsk_ca_priv, 0, sizeof(icsk->icsk_ca_priv));
+ if (ca->flags & TCP_CONG_NEEDS_ECN)
+ INET_ECN_xmit(sk);
+ else
+ INET_ECN_dontxmit(sk);
+
if (!((1 << sk->sk_state) & (TCPF_CLOSE | TCPF_LISTEN)))
tcp_init_congestion_control(sk);
}
/* Note: tcp_v6_init_req() might override ir_iif for link locals */
inet_rsk(req)->ir_iif = inet_request_bound_dev_if(sk, skb);
- af_ops->init_req(req, sk, skb);
-
- if (security_inet_conn_request(sk, skb, req))
+ dst = af_ops->route_req(sk, skb, &fl, req);
+ if (!dst)
goto drop_and_free;
if (tmp_opt.tstamp_ok)
tcp_rsk(req)->ts_off = af_ops->init_ts_off(net, skb);
- dst = af_ops->route_req(sk, &fl, req);
- if (!dst)
- goto drop_and_free;
-
if (!want_cookie && !isn) {
/* Kill the following clause, if you dislike this way. */
if (!net->ipv4.sysctl_tcp_syncookies &&
skb = tcp_make_synack(sk, dst, req, foc, synack_type, syn_skb);
- tos = sock_net(sk)->ipv4.sysctl_tcp_reflect_tos ?
- tcp_rsk(req)->syn_tos : inet_sk(sk)->tos;
-
if (skb) {
__tcp_v4_send_check(skb, ireq->ir_loc_addr, ireq->ir_rmt_addr);
+ tos = sock_net(sk)->ipv4.sysctl_tcp_reflect_tos ?
+ tcp_rsk(req)->syn_tos & ~INET_ECN_MASK :
+ inet_sk(sk)->tos;
+
+ if (!INET_ECN_is_capable(tos) &&
+ tcp_bpf_ca_needs_ecn((struct sock *)req))
+ tos |= INET_ECN_ECT_0;
+
rcu_read_lock();
err = ip_build_and_send_pkt(skb, sk, ireq->ir_loc_addr,
ireq->ir_rmt_addr,
rcu_dereference(ireq->ireq_opt),
- tos & ~INET_ECN_MASK);
+ tos);
rcu_read_unlock();
err = net_xmit_eval(err);
}
}
static struct dst_entry *tcp_v4_route_req(const struct sock *sk,
+ struct sk_buff *skb,
struct flowi *fl,
- const struct request_sock *req)
+ struct request_sock *req)
{
+ tcp_v4_init_req(req, sk, skb);
+
+ if (security_inet_conn_request(sk, skb, req))
+ return NULL;
+
return inet_csk_route_req(sk, &fl->u.ip4, req);
}
.req_md5_lookup = tcp_v4_md5_lookup,
.calc_md5_hash = tcp_v4_md5_hash_skb,
#endif
- .init_req = tcp_v4_init_req,
#ifdef CONFIG_SYN_COOKIES
.cookie_init_seq = cookie_v4_init_sequence,
#endif
bool *own_req)
{
struct inet_request_sock *ireq;
+ bool found_dup_sk = false;
struct inet_sock *newinet;
struct tcp_sock *newtp;
struct sock *newsk;
if (__inet_inherit_port(sk, newsk) < 0)
goto put_and_exit;
- *own_req = inet_ehash_nolisten(newsk, req_to_sk(req_unhash));
+ *own_req = inet_ehash_nolisten(newsk, req_to_sk(req_unhash),
+ &found_dup_sk);
if (likely(*own_req)) {
tcp_move_syn(newtp, req);
ireq->ireq_opt = NULL;
} else {
- newinet->inet_opt = NULL;
+ if (!req_unhash && found_dup_sk) {
+ /* This code path should only be executed in the
+ * syncookie case only
+ */
+ bh_unlock_sock(newsk);
+ sock_put(newsk);
+ newsk = NULL;
+ } else {
+ newinet->inet_opt = NULL;
+ }
}
return newsk;
}
#endif /* CONFIG_PROC_FS */
+/* @wake is one when sk_stream_write_space() calls us.
+ * This sends EPOLLOUT only if notsent_bytes is half the limit.
+ * This mimics the strategy used in sock_def_write_space().
+ */
+bool tcp_stream_memory_free(const struct sock *sk, int wake)
+{
+ const struct tcp_sock *tp = tcp_sk(sk);
+ u32 notsent_bytes = READ_ONCE(tp->write_seq) -
+ READ_ONCE(tp->snd_nxt);
+
+ return (notsent_bytes << wake) < tcp_notsent_lowat(tp);
+}
+EXPORT_SYMBOL(tcp_stream_memory_free);
+
struct proto tcp_prot = {
.name = "TCP",
.owner = THIS_MODULE,
struct mptcp_out_options mptcp;
};
-static void mptcp_options_write(__be32 *ptr, struct tcp_out_options *opts)
+static void mptcp_options_write(__be32 *ptr, const struct tcp_sock *tp,
+ struct tcp_out_options *opts)
{
#if IS_ENABLED(CONFIG_MPTCP)
if (unlikely(OPTION_MPTCP & opts->options))
- mptcp_write_options(ptr, &opts->mptcp);
+ mptcp_write_options(ptr, tp, &opts->mptcp);
#endif
}
smc_options_write(ptr, &options);
- mptcp_options_write(ptr, opts);
+ mptcp_options_write(ptr, tp, opts);
}
static void smc_set_option(const struct tcp_sock *tp,
}
}
- tcp_options_write((__be32 *)(th + 1), tp, &opts);
skb_shinfo(skb)->gso_type = sk->sk_gso_type;
if (likely(!(tcb->tcp_flags & TCPHDR_SYN))) {
th->window = htons(tcp_select_window(sk));
*/
th->window = htons(min(tp->rcv_wnd, 65535U));
}
+
+ tcp_options_write((__be32 *)(th + 1), tp, &opts);
+
#ifdef CONFIG_TCP_MD5SIG
/* Calculate the MD5 hash, as we have all we need now */
if (md5) {
iph->daddr, dport, inet_iif(skb),
inet_sdif(skb), &udp_table, NULL);
}
-EXPORT_SYMBOL_GPL(udp4_lib_lookup_skb);
/* Must be called under rcu_read_lock().
* Does increment socket refcount.
return -EMSGSIZE;
if (args->netnsid >= 0 &&
- nla_put_s32(skb, IFA_TARGET_NETNSID, args->netnsid))
+ nla_put_s32(skb, IFA_TARGET_NETNSID, args->netnsid)) {
+ nlmsg_cancel(skb, nlh);
return -EMSGSIZE;
+ }
put_ifaddrmsg(nlh, 128, IFA_F_PERMANENT, scope, ifindex);
if (nla_put_in6_addr(skb, IFA_MULTICAST, &ifmca->mca_addr) < 0 ||
return -EMSGSIZE;
if (args->netnsid >= 0 &&
- nla_put_s32(skb, IFA_TARGET_NETNSID, args->netnsid))
+ nla_put_s32(skb, IFA_TARGET_NETNSID, args->netnsid)) {
+ nlmsg_cancel(skb, nlh);
return -EMSGSIZE;
+ }
put_ifaddrmsg(nlh, 128, IFA_F_PERMANENT, scope, ifindex);
if (nla_put_in6_addr(skb, IFA_ANYCAST, &ifaca->aca_addr) < 0 ||
/* add default label */
static int __net_init ip6addrlbl_net_init(struct net *net)
{
- int err = 0;
+ struct ip6addrlbl_entry *p = NULL;
+ struct hlist_node *n;
+ int err;
int i;
ADDRLABEL(KERN_DEBUG "%s\n", __func__);
INIT_HLIST_HEAD(&net->ipv6.ip6addrlbl_table.head);
for (i = 0; i < ARRAY_SIZE(ip6addrlbl_init_table); i++) {
- int ret = ip6addrlbl_add(net,
- ip6addrlbl_init_table[i].prefix,
- ip6addrlbl_init_table[i].prefixlen,
- 0,
- ip6addrlbl_init_table[i].label, 0);
- /* XXX: should we free all rules when we catch an error? */
- if (ret && (!err || err != -ENOMEM))
- err = ret;
+ err = ip6addrlbl_add(net,
+ ip6addrlbl_init_table[i].prefix,
+ ip6addrlbl_init_table[i].prefixlen,
+ 0,
+ ip6addrlbl_init_table[i].label, 0);
+ if (err)
+ goto err_ip6addrlbl_add;
+ }
+ return 0;
+
+err_ip6addrlbl_add:
+ hlist_for_each_entry_safe(p, n, &net->ipv6.ip6addrlbl_table.head, list) {
+ hlist_del_rcu(&p->list);
+ kfree_rcu(p, rcu);
}
return err;
}
memcpy(auth_data, ah->auth_data, ahp->icv_trunc_len);
memset(ah->auth_data, 0, ahp->icv_trunc_len);
- if (ipv6_clear_mutable_options(ip6h, hdr_len, XFRM_POLICY_IN))
+ err = ipv6_clear_mutable_options(ip6h, hdr_len, XFRM_POLICY_IN);
+ if (err)
goto out_free;
ip6h->priority = 0;
return;
if (rt->dst.dev) {
- dev->needed_headroom = rt->dst.dev->hard_header_len +
- t_hlen;
+ unsigned short dst_len = rt->dst.dev->hard_header_len +
+ t_hlen;
+
+ if (t->dev->header_ops)
+ dev->hard_header_len = dst_len;
+ else
+ dev->needed_headroom = dst_len;
if (set_mtu) {
dev->mtu = rt->dst.dev->mtu - t_hlen;
tunnel->hlen = tunnel->tun_hlen + tunnel->encap_hlen;
t_hlen = tunnel->hlen + sizeof(struct ipv6hdr);
- tunnel->dev->needed_headroom = LL_MAX_HEADER + t_hlen;
+
+ if (tunnel->dev->header_ops)
+ tunnel->dev->hard_header_len = LL_MAX_HEADER + t_hlen;
+ else
+ tunnel->dev->needed_headroom = LL_MAX_HEADER + t_hlen;
+
return t_hlen;
}
if (sk->sk_type != SOCK_STREAM)
return -ENOPROTOOPT;
- msg.msg_control = optval;
+ msg.msg_control_user = optval;
msg.msg_controllen = len;
msg.msg_flags = flags;
msg.msg_control_is_user = true;
static int pndisc_constructor(struct pneigh_entry *n);
static void pndisc_destructor(struct pneigh_entry *n);
static void pndisc_redo(struct sk_buff *skb);
+static int ndisc_is_multicast(const void *pkey);
static const struct neigh_ops ndisc_generic_ops = {
.family = AF_INET6,
.pconstructor = pndisc_constructor,
.pdestructor = pndisc_destructor,
.proxy_redo = pndisc_redo,
+ .is_multicast = ndisc_is_multicast,
.allow_add = ndisc_allow_add,
.id = "ndisc_cache",
.parms = {
kfree_skb(skb);
}
+static int ndisc_is_multicast(const void *pkey)
+{
+ return ipv6_addr_is_multicast((struct in6_addr *)pkey);
+}
+
static bool ndisc_suppress_frag_ndisc(struct sk_buff *skb)
{
struct inet6_dev *idev = __in6_dev_get(skb->dev);
int nf_ct_frag6_gather(struct net *net, struct sk_buff *skb, u32 user)
{
u16 savethdr = skb->transport_header;
+ u8 nexthdr = NEXTHDR_FRAGMENT;
int fhoff, nhoff, ret;
struct frag_hdr *fhdr;
struct frag_queue *fq;
if (find_prev_fhdr(skb, &prevhdr, &nhoff, &fhoff) < 0)
return 0;
+ /* Discard the first fragment if it does not include all headers
+ * RFC 8200, Section 4.5
+ */
+ if (ipv6frag_thdr_truncated(skb, fhoff, &nexthdr)) {
+ pr_debug("Drop incomplete fragment\n");
+ return 0;
+ }
+
if (!pskb_may_pull(skb, fhoff + sizeof(*fhdr)))
return -ENOMEM;
struct frag_queue *fq;
const struct ipv6hdr *hdr = ipv6_hdr(skb);
struct net *net = dev_net(skb_dst(skb)->dev);
- __be16 frag_off;
- int iif, offset;
u8 nexthdr;
+ int iif;
if (IP6CB(skb)->flags & IP6SKB_FRAGMENTED)
goto fail_hdr;
* the source of the fragment, with the Pointer field set to zero.
*/
nexthdr = hdr->nexthdr;
- offset = ipv6_skip_exthdr(skb, skb_transport_offset(skb), &nexthdr, &frag_off);
- if (offset >= 0) {
- /* Check some common protocols' header */
- if (nexthdr == IPPROTO_TCP)
- offset += sizeof(struct tcphdr);
- else if (nexthdr == IPPROTO_UDP)
- offset += sizeof(struct udphdr);
- else if (nexthdr == IPPROTO_ICMPV6)
- offset += sizeof(struct icmp6hdr);
- else
- offset += 1;
-
- if (!(frag_off & htons(IP6_OFFSET)) && offset > skb->len) {
- __IP6_INC_STATS(net, __in6_dev_get_safely(skb->dev),
- IPSTATS_MIB_INHDRERRORS);
- icmpv6_param_prob(skb, ICMPV6_HDR_INCOMP, 0);
- return -1;
- }
+ if (ipv6frag_thdr_truncated(skb, skb_transport_offset(skb), &nexthdr)) {
+ __IP6_INC_STATS(net, __in6_dev_get_safely(skb->dev),
+ IPSTATS_MIB_INHDRERRORS);
+ icmpv6_param_prob(skb, ICMPV6_HDR_INCOMP, 0);
+ return -1;
}
iif = skb->dev ? skb->dev->ifindex : 0;
if (dst->dev && nla_put_u32(skb, RTA_OIF, dst->dev->ifindex))
goto nla_put_failure;
+
+ if (dst->lwtstate &&
+ lwtunnel_fill_encap(skb, dst->lwtstate, RTA_ENCAP, RTA_ENCAP_TYPE) < 0)
+ goto nla_put_failure;
} else if (rt->fib6_nsiblings) {
struct fib6_info *sibling, *next_sibling;
struct nlattr *mp;
if (np->repflow && ireq->pktopts)
fl6->flowlabel = ip6_flowlabel(ipv6_hdr(ireq->pktopts));
+ tclass = sock_net(sk)->ipv4.sysctl_tcp_reflect_tos ?
+ tcp_rsk(req)->syn_tos & ~INET_ECN_MASK :
+ np->tclass;
+
+ if (!INET_ECN_is_capable(tclass) &&
+ tcp_bpf_ca_needs_ecn((struct sock *)req))
+ tclass |= INET_ECN_ECT_0;
+
rcu_read_lock();
opt = ireq->ipv6_opt;
- tclass = sock_net(sk)->ipv4.sysctl_tcp_reflect_tos ?
- tcp_rsk(req)->syn_tos : np->tclass;
if (!opt)
opt = rcu_dereference(np->opt);
err = ip6_xmit(sk, skb, fl6, sk->sk_mark, opt,
- tclass & ~INET_ECN_MASK,
- sk->sk_priority);
+ tclass, sk->sk_priority);
rcu_read_unlock();
err = net_xmit_eval(err);
}
}
static struct dst_entry *tcp_v6_route_req(const struct sock *sk,
+ struct sk_buff *skb,
struct flowi *fl,
- const struct request_sock *req)
+ struct request_sock *req)
{
+ tcp_v6_init_req(req, sk, skb);
+
+ if (security_inet_conn_request(sk, skb, req))
+ return NULL;
+
return inet6_csk_route_req(sk, &fl->u.ip6, req, IPPROTO_TCP);
}
.req_md5_lookup = tcp_v6_md5_lookup,
.calc_md5_hash = tcp_v6_md5_hash_skb,
#endif
- .init_req = tcp_v6_init_req,
#ifdef CONFIG_SYN_COOKIES
.cookie_init_seq = cookie_v6_init_sequence,
#endif
const struct ipv6_pinfo *np = tcp_inet6_sk(sk);
struct ipv6_txoptions *opt;
struct inet_sock *newinet;
+ bool found_dup_sk = false;
struct tcp_sock *newtp;
struct sock *newsk;
#ifdef CONFIG_TCP_MD5SIG
tcp_done(newsk);
goto out;
}
- *own_req = inet_ehash_nolisten(newsk, req_to_sk(req_unhash));
+ *own_req = inet_ehash_nolisten(newsk, req_to_sk(req_unhash),
+ &found_dup_sk);
if (*own_req) {
tcp_move_syn(newtp, req);
skb_set_owner_r(newnp->pktoptions, newsk);
}
}
+ } else {
+ if (!req_unhash && found_dup_sk) {
+ /* This code path should only be executed in the
+ * syncookie case only
+ */
+ bh_unlock_sock(newsk);
+ sock_put(newsk);
+ newsk = NULL;
+ }
}
return newsk;
&iph->daddr, dport, inet6_iif(skb),
inet6_sdif(skb), &udp_table, NULL);
}
-EXPORT_SYMBOL_GPL(udp6_lib_lookup_skb);
/* Must be called under rcu_read_lock().
* Does increment socket refcount.
}
/* Create the new socket */
- nsk = iucv_sock_alloc(NULL, sk->sk_type, GFP_ATOMIC, 0);
+ nsk = iucv_sock_alloc(NULL, sk->sk_protocol, GFP_ATOMIC, 0);
if (!nsk) {
err = pr_iucv->path_sever(path, user_data);
iucv_path_free(path);
goto out;
}
- nsk = iucv_sock_alloc(NULL, sk->sk_type, GFP_ATOMIC, 0);
+ nsk = iucv_sock_alloc(NULL, sk->sk_protocol, GFP_ATOMIC, 0);
bh_lock_sock(sk);
if ((sk->sk_state != IUCV_LISTEN) ||
sk_acceptq_is_full(sk) ||
return used;
}
+/* Handle device status changes. */
+static int lapb_device_event(struct notifier_block *this, unsigned long event,
+ void *ptr)
+{
+ struct net_device *dev = netdev_notifier_info_to_dev(ptr);
+ struct lapb_cb *lapb;
+
+ if (!net_eq(dev_net(dev), &init_net))
+ return NOTIFY_DONE;
+
+ if (dev->type != ARPHRD_X25)
+ return NOTIFY_DONE;
+
+ lapb = lapb_devtostruct(dev);
+ if (!lapb)
+ return NOTIFY_DONE;
+
+ switch (event) {
+ case NETDEV_UP:
+ lapb_dbg(0, "(%p) Interface up: %s\n", dev, dev->name);
+
+ if (netif_carrier_ok(dev)) {
+ lapb_dbg(0, "(%p): Carrier is already up: %s\n", dev,
+ dev->name);
+ if (lapb->mode & LAPB_DCE) {
+ lapb_start_t1timer(lapb);
+ } else {
+ if (lapb->state == LAPB_STATE_0) {
+ lapb->state = LAPB_STATE_1;
+ lapb_establish_data_link(lapb);
+ }
+ }
+ }
+ break;
+ case NETDEV_GOING_DOWN:
+ if (netif_carrier_ok(dev))
+ lapb_disconnect_request(dev);
+ break;
+ case NETDEV_DOWN:
+ lapb_dbg(0, "(%p) Interface down: %s\n", dev, dev->name);
+ lapb_dbg(0, "(%p) S%d -> S0\n", dev, lapb->state);
+ lapb_clear_queues(lapb);
+ lapb->state = LAPB_STATE_0;
+ lapb->n2count = 0;
+ lapb_stop_t1timer(lapb);
+ lapb_stop_t2timer(lapb);
+ break;
+ case NETDEV_CHANGE:
+ if (netif_carrier_ok(dev)) {
+ lapb_dbg(0, "(%p): Carrier detected: %s\n", dev,
+ dev->name);
+ if (lapb->mode & LAPB_DCE) {
+ lapb_start_t1timer(lapb);
+ } else {
+ if (lapb->state == LAPB_STATE_0) {
+ lapb->state = LAPB_STATE_1;
+ lapb_establish_data_link(lapb);
+ }
+ }
+ } else {
+ lapb_dbg(0, "(%p) Carrier lost: %s\n", dev, dev->name);
+ lapb_dbg(0, "(%p) S%d -> S0\n", dev, lapb->state);
+ lapb_clear_queues(lapb);
+ lapb->state = LAPB_STATE_0;
+ lapb->n2count = 0;
+ lapb_stop_t1timer(lapb);
+ lapb_stop_t2timer(lapb);
+ }
+ break;
+ }
+
+ return NOTIFY_DONE;
+}
+
+static struct notifier_block lapb_dev_notifier = {
+ .notifier_call = lapb_device_event,
+};
+
static int __init lapb_init(void)
{
- return 0;
+ return register_netdevice_notifier(&lapb_dev_notifier);
}
static void __exit lapb_exit(void)
{
WARN_ON(!list_empty(&lapb_list));
+
+ unregister_netdevice_notifier(&lapb_dev_notifier);
}
MODULE_AUTHOR("Jonathan Naylor <g4klx@g4klx.demon.co.uk>");
switch (lapb->state) {
/*
- * If we are a DCE, keep going DM .. DM .. DM
+ * If we are a DCE, send DM up to N2 times, then switch to
+ * STATE_1 and send SABM(E).
*/
case LAPB_STATE_0:
- if (lapb->mode & LAPB_DCE)
+ if (lapb->mode & LAPB_DCE &&
+ lapb->n2count != lapb->n2) {
+ lapb->n2count++;
lapb_send_control(lapb, LAPB_DM, LAPB_POLLOFF, LAPB_RESPONSE);
+ } else {
+ lapb->state = LAPB_STATE_1;
+ lapb_establish_data_link(lapb);
+ }
break;
/*
success = !!(info->flags & IEEE80211_TX_STAT_ACK);
for (i = 0; i < IEEE80211_TX_MAX_RATES; i++) {
- if (ar[i].idx < 0)
+ if (ar[i].idx < 0 || !ar[i].count)
break;
ndx = rix_to_ndx(mi, ar[i].idx);
mi->r[ndx].stats.success += success;
}
- if ((info->flags & IEEE80211_TX_CTL_RATE_CTRL_PROBE) && (i >= 0))
- mi->sample_packets++;
-
- if (mi->sample_deferred > 0)
- mi->sample_deferred--;
-
if (time_after(jiffies, mi->last_stats_update +
mp->update_interval / (mp->new_avg ? 2 : 1)))
minstrel_update_stats(mp, mi);
return;
delta = (mi->total_packets * sampling_ratio / 100) -
- (mi->sample_packets + mi->sample_deferred / 2);
+ mi->sample_packets;
/* delta < 0: no sampling required */
prev_sample = mi->prev_sample;
return;
if (mi->total_packets >= 10000) {
- mi->sample_deferred = 0;
mi->sample_packets = 0;
mi->total_packets = 0;
} else if (delta > mi->n_rates * 2) {
* rate sampling method should be used.
* Respect such rates that are not sampled for 20 interations.
*/
- if (mrr_capable &&
- msr->perfect_tx_time > mr->perfect_tx_time &&
- msr->stats.sample_skipped < 20) {
- /* Only use IEEE80211_TX_CTL_RATE_CTRL_PROBE to mark
- * packets that have the sampling rate deferred to the
- * second MRR stage. Increase the sample counter only
- * if the deferred sample rate was actually used.
- * Use the sample_deferred counter to make sure that
- * the sampling is not done in large bursts */
- info->flags |= IEEE80211_TX_CTL_RATE_CTRL_PROBE;
- rate++;
- mi->sample_deferred++;
- } else {
+ if (msr->perfect_tx_time < mr->perfect_tx_time ||
+ msr->stats.sample_skipped >= 20) {
if (!msr->sample_limit)
return;
rate->idx = mi->r[ndx].rix;
rate->count = minstrel_get_retry_count(&mi->r[ndx], info);
+ info->flags |= IEEE80211_TX_CTL_RATE_CTRL_PROBE;
}
u8 max_prob_rate;
unsigned int total_packets;
unsigned int sample_packets;
- int sample_deferred;
unsigned int sample_row;
unsigned int sample_column;
out_drop_sta:
local->num_sta--;
synchronize_net();
- __cleanup_single_sta(sta);
+ cleanup_single_sta(sta);
out_err:
mutex_unlock(&local->sta_mtx);
kfree(sinfo);
err = sta_info_insert_check(sta);
if (err) {
+ sta_info_free(local, sta);
mutex_unlock(&local->sta_mtx);
rcu_read_lock();
- goto out_free;
+ return err;
}
- err = sta_info_insert_finish(sta);
- if (err)
- goto out_free;
-
- return 0;
- out_free:
- sta_info_free(local, sta);
- return err;
+ return sta_info_insert_finish(sta);
}
int sta_info_insert(struct sta_info *sta)
int ac;
if (info->flags & (IEEE80211_TX_CTL_NO_PS_BUFFER |
- IEEE80211_TX_CTL_AMPDU)) {
+ IEEE80211_TX_CTL_AMPDU |
+ IEEE80211_TX_CTL_HW_80211_ENCAP)) {
ieee80211_free_txskb(&local->hw, skb);
return;
}
ieee80211_mpsp_trigger_process(
ieee80211_get_qos_ctl(hdr), sta, true, acked);
- if (!acked && test_sta_flag(sta, WLAN_STA_PS_STA)) {
- /*
- * The STA is in power save mode, so assume
- * that this TX packet failed because of that.
- */
- ieee80211_handle_filtered_frame(local, sta, skb);
- return;
- }
-
if (ieee80211_hw_check(&local->hw, HAS_RATE_CONTROL) &&
(ieee80211_is_data(hdr->frame_control)) &&
(rates_idx != -1))
-info->status.ack_signal);
}
} else if (test_sta_flag(sta, WLAN_STA_PS_STA)) {
+ /*
+ * The STA is in power save mode, so assume
+ * that this TX packet failed because of that.
+ */
+ if (skb)
+ ieee80211_handle_filtered_frame(local, sta, skb);
return;
} else if (noack_success) {
/* nothing to do here, do not account as lost */
info->mptcpi_flags = flags;
info->mptcpi_token = READ_ONCE(msk->token);
info->mptcpi_write_seq = READ_ONCE(msk->write_seq);
- info->mptcpi_snd_una = atomic64_read(&msk->snd_una);
+ info->mptcpi_snd_una = READ_ONCE(msk->snd_una);
info->mptcpi_rcv_nxt = READ_ONCE(msk->ack_seq);
unlock_sock_fast(sk, slow);
}
mp_opt->add_addr = 1;
mp_opt->addr_id = *ptr++;
- pr_debug("ADD_ADDR: id=%d, echo=%d", mp_opt->addr_id, mp_opt->echo);
+ pr_debug("ADD_ADDR%s: id=%d, echo=%d",
+ (mp_opt->family == MPTCP_ADDR_IPVERSION_6) ? "6" : "",
+ mp_opt->addr_id, mp_opt->echo);
if (mp_opt->family == MPTCP_ADDR_IPVERSION_4) {
memcpy((u8 *)&mp_opt->addr.s_addr, (u8 *)ptr, 4);
ptr += 4;
bool ret = false;
mpext = skb ? mptcp_get_ext(skb) : NULL;
- snd_data_fin_enable = READ_ONCE(msk->snd_data_fin_enable);
+ snd_data_fin_enable = mptcp_data_fin_enabled(msk);
if (!skb || (mpext && mpext->use_map) || snd_data_fin_enable) {
unsigned int map_size;
opts->ext_copy.ack64 = 0;
}
opts->ext_copy.use_ack = 1;
+ WRITE_ONCE(msk->old_wspace, __mptcp_space((struct sock *)msk));
/* Add kind/length/subtype/flag overhead if mapping is not populated */
if (dss_size == 0)
}
#endif
-static bool mptcp_established_options_add_addr(struct sock *sk,
+static bool mptcp_established_options_add_addr(struct sock *sk, struct sk_buff *skb,
unsigned int *size,
unsigned int remaining,
struct mptcp_out_options *opts)
{
struct mptcp_subflow_context *subflow = mptcp_subflow_ctx(sk);
struct mptcp_sock *msk = mptcp_sk(subflow->conn);
+ bool drop_other_suboptions = false;
+ unsigned int opt_size = *size;
struct mptcp_addr_info saddr;
bool echo;
int len;
+ if (mptcp_pm_should_add_signal_ipv6(msk) &&
+ skb && skb_is_tcp_pure_ack(skb)) {
+ pr_debug("drop other suboptions");
+ opts->suboptions = 0;
+ remaining += opt_size;
+ drop_other_suboptions = true;
+ }
+
if (!mptcp_pm_should_add_signal(msk) ||
!(mptcp_pm_add_addr_signal(msk, remaining, &saddr, &echo)))
return false;
return false;
*size = len;
+ if (drop_other_suboptions)
+ *size -= opt_size;
opts->addr_id = saddr.id;
if (saddr.family == AF_INET) {
opts->suboptions |= OPTION_MPTCP_ADD_ADDR;
*size += opt_size;
remaining -= opt_size;
- if (mptcp_established_options_add_addr(sk, &opt_size, remaining, opts)) {
+ if (mptcp_established_options_add_addr(sk, skb, &opt_size, remaining, opts)) {
*size += opt_size;
remaining -= opt_size;
ret = true;
goto fully_established;
}
+ if (mp_opt->add_addr) {
+ WRITE_ONCE(msk->fully_established, true);
+ return true;
+ }
+
/* If the first established packet does not contain MP_CAPABLE + data
* then fallback to TCP. Fallback scenarios requires a reset for
* MP_JOIN subflows.
return cur_ack;
}
-static void update_una(struct mptcp_sock *msk,
- struct mptcp_options_received *mp_opt)
+static void ack_update_msk(struct mptcp_sock *msk,
+ struct sock *ssk,
+ struct mptcp_options_received *mp_opt)
{
- u64 new_snd_una, snd_una, old_snd_una = atomic64_read(&msk->snd_una);
- u64 write_seq = READ_ONCE(msk->write_seq);
+ u64 new_wnd_end, new_snd_una, snd_nxt = READ_ONCE(msk->snd_nxt);
+ struct sock *sk = (struct sock *)msk;
+ u64 old_snd_una;
+
+ mptcp_data_lock(sk);
/* avoid ack expansion on update conflict, to reduce the risk of
* wrongly expanding to a future ack sequence number, which is way
* more dangerous than missing an ack
*/
+ old_snd_una = msk->snd_una;
new_snd_una = expand_ack(old_snd_una, mp_opt->data_ack, mp_opt->ack64);
/* ACK for data not even sent yet? Ignore. */
- if (after64(new_snd_una, write_seq))
+ if (after64(new_snd_una, snd_nxt))
new_snd_una = old_snd_una;
- while (after64(new_snd_una, old_snd_una)) {
- snd_una = old_snd_una;
- old_snd_una = atomic64_cmpxchg(&msk->snd_una, snd_una,
- new_snd_una);
- if (old_snd_una == snd_una) {
- mptcp_data_acked((struct sock *)msk);
- break;
- }
+ new_wnd_end = new_snd_una + tcp_sk(ssk)->snd_wnd;
+
+ if (after64(new_wnd_end, msk->wnd_end)) {
+ msk->wnd_end = new_wnd_end;
+ __mptcp_wnd_updated(sk, ssk);
}
+
+ if (after64(new_snd_una, old_snd_una)) {
+ msk->snd_una = new_snd_una;
+ __mptcp_data_acked(sk);
+ }
+ mptcp_data_unlock(sk);
}
bool mptcp_update_rcv_data_fin(struct mptcp_sock *msk, u64 data_fin_seq, bool use_64bit)
struct mptcp_options_received mp_opt;
struct mptcp_ext *mpext;
- if (__mptcp_check_fallback(msk))
+ if (__mptcp_check_fallback(msk)) {
+ /* Keep it simple and unconditionally trigger send data cleanup and
+ * pending queue spooling. We will need to acquire the data lock
+ * for more accurate checks, and once the lock is acquired, such
+ * helpers are cheap.
+ */
+ mptcp_data_lock(subflow->conn);
+ if (mptcp_send_head(subflow->conn))
+ __mptcp_wnd_updated(subflow->conn, sk);
+ __mptcp_data_acked(subflow->conn);
+ mptcp_data_unlock(subflow->conn);
return;
+ }
mptcp_get_options(skb, &mp_opt);
if (!check_fully_established(msk, sk, subflow, skb, &mp_opt))
* monodirectional flows will stuck
*/
if (mp_opt.use_ack)
- update_una(msk, &mp_opt);
+ ack_update_msk(msk, sk, &mp_opt);
/* Zero-data-length packets are dropped by the caller and not
* propagated to the MPTCP layer, so the skb extension does not
}
}
-void mptcp_write_options(__be32 *ptr, struct mptcp_out_options *opts)
+static void mptcp_set_rwin(const struct tcp_sock *tp)
+{
+ const struct sock *ssk = (const struct sock *)tp;
+ const struct mptcp_subflow_context *subflow;
+ struct mptcp_sock *msk;
+ u64 ack_seq;
+
+ subflow = mptcp_subflow_ctx(ssk);
+ msk = mptcp_sk(subflow->conn);
+
+ ack_seq = READ_ONCE(msk->ack_seq) + tp->rcv_wnd;
+
+ if (after64(ack_seq, READ_ONCE(msk->rcv_wnd_sent)))
+ WRITE_ONCE(msk->rcv_wnd_sent, ack_seq);
+}
+
+void mptcp_write_options(__be32 *ptr, const struct tcp_sock *tp,
+ struct mptcp_out_options *opts)
{
if ((OPTION_MPTCP_MPC_SYN | OPTION_MPTCP_MPC_SYNACK |
OPTION_MPTCP_MPC_ACK) & opts->suboptions) {
TCPOPT_NOP << 8 | TCPOPT_NOP, ptr);
}
}
+
+ if (tp)
+ mptcp_set_rwin(tp);
}
const struct mptcp_addr_info *addr,
bool echo)
{
+ u8 add_addr = READ_ONCE(msk->pm.add_addr_signal);
+
pr_debug("msk=%p, local_id=%d", msk, addr->id);
msk->pm.local = *addr;
- WRITE_ONCE(msk->pm.add_addr_echo, echo);
- WRITE_ONCE(msk->pm.add_addr_signal, true);
+ add_addr |= BIT(MPTCP_ADD_ADDR_SIGNAL);
+ if (echo)
+ add_addr |= BIT(MPTCP_ADD_ADDR_ECHO);
+ if (addr->family == AF_INET6)
+ add_addr |= BIT(MPTCP_ADD_ADDR_IPV6);
+ WRITE_ONCE(msk->pm.add_addr_signal, add_addr);
return 0;
}
return false;
msk->pm.status |= BIT(new_status);
- if (schedule_work(&msk->work))
- sock_hold((struct sock *)msk);
+ mptcp_schedule_work((struct sock *)msk);
return true;
}
spin_lock_bh(&pm->lock);
- if (!READ_ONCE(pm->accept_addr))
+ if (!READ_ONCE(pm->accept_addr)) {
mptcp_pm_announce_addr(msk, addr, true);
- else if (mptcp_pm_schedule_work(msk, MPTCP_PM_ADD_ADDR_RECEIVED))
+ mptcp_pm_add_addr_send_ack(msk);
+ } else if (mptcp_pm_schedule_work(msk, MPTCP_PM_ADD_ADDR_RECEIVED)) {
pm->remote = *addr;
+ }
spin_unlock_bh(&pm->lock);
}
+void mptcp_pm_add_addr_send_ack(struct mptcp_sock *msk)
+{
+ if (!mptcp_pm_should_add_signal_ipv6(msk))
+ return;
+
+ mptcp_pm_schedule_work(msk, MPTCP_PM_ADD_ADDR_SEND_ACK);
+}
+
void mptcp_pm_rm_addr_received(struct mptcp_sock *msk, u8 rm_id)
{
struct mptcp_pm_data *pm = &msk->pm;
if (!mptcp_pm_should_add_signal(msk))
goto out_unlock;
- *echo = READ_ONCE(msk->pm.add_addr_echo);
+ *echo = mptcp_pm_should_add_signal_echo(msk);
if (remaining < mptcp_add_addr_len(msk->pm.local.family, *echo))
goto out_unlock;
*saddr = msk->pm.local;
- WRITE_ONCE(msk->pm.add_addr_signal, false);
+ WRITE_ONCE(msk->pm.add_addr_signal, 0);
ret = true;
out_unlock:
msk->pm.subflows = 0;
msk->pm.rm_id = 0;
WRITE_ONCE(msk->pm.work_pending, false);
- WRITE_ONCE(msk->pm.add_addr_signal, false);
+ WRITE_ONCE(msk->pm.add_addr_signal, 0);
WRITE_ONCE(msk->pm.rm_addr_signal, false);
WRITE_ONCE(msk->pm.accept_addr, false);
WRITE_ONCE(msk->pm.accept_subflow, false);
- WRITE_ONCE(msk->pm.add_addr_echo, false);
msk->pm.status = 0;
spin_lock_init(&msk->pm.lock);
if (!mptcp_pm_should_add_signal(msk)) {
pr_debug("retransmit ADD_ADDR id=%d", entry->addr.id);
mptcp_pm_announce_addr(msk, &entry->addr, false);
+ mptcp_pm_add_addr_send_ack(msk);
entry->retrans_times++;
}
if (mptcp_pm_alloc_anno_list(msk, local)) {
msk->pm.add_addr_signaled++;
mptcp_pm_announce_addr(msk, &local->addr, false);
+ mptcp_pm_nl_add_addr_send_ack(msk);
}
} else {
/* pick failed, avoid fourther attempts later */
spin_lock_bh(&msk->pm.lock);
mptcp_pm_announce_addr(msk, &remote, true);
+ mptcp_pm_nl_add_addr_send_ack(msk);
+}
+
+void mptcp_pm_nl_add_addr_send_ack(struct mptcp_sock *msk)
+{
+ struct mptcp_subflow_context *subflow;
+
+ if (!mptcp_pm_should_add_signal_ipv6(msk))
+ return;
+
+ __mptcp_flush_join_list(msk);
+ subflow = list_first_entry_or_null(&msk->conn_list, typeof(*subflow), node);
+ if (subflow) {
+ struct sock *ssk = mptcp_subflow_tcp_sock(subflow);
+ u8 add_addr;
+
+ spin_unlock_bh(&msk->pm.lock);
+ pr_debug("send ack for add_addr6");
+ lock_sock(ssk);
+ tcp_send_ack(ssk);
+ release_sock(ssk);
+ spin_lock_bh(&msk->pm.lock);
+
+ add_addr = READ_ONCE(msk->pm.add_addr_signal);
+ add_addr &= ~BIT(MPTCP_ADD_ADDR_IPV6);
+ WRITE_ONCE(msk->pm.add_addr_signal, add_addr);
+ }
}
void mptcp_pm_nl_rm_addr_received(struct mptcp_sock *msk)
list_for_each_entry_safe(subflow, tmp, &msk->conn_list, node) {
struct sock *ssk = mptcp_subflow_tcp_sock(subflow);
int how = RCV_SHUTDOWN | SEND_SHUTDOWN;
- long timeout = 0;
if (msk->pm.rm_id != subflow->remote_id)
continue;
spin_unlock_bh(&msk->pm.lock);
mptcp_subflow_shutdown(sk, ssk, how);
- __mptcp_close_ssk(sk, ssk, subflow, timeout);
+ __mptcp_close_ssk(sk, ssk, subflow);
spin_lock_bh(&msk->pm.lock);
msk->pm.add_addr_accepted--;
list_for_each_entry_safe(subflow, tmp, &msk->conn_list, node) {
struct sock *ssk = mptcp_subflow_tcp_sock(subflow);
int how = RCV_SHUTDOWN | SEND_SHUTDOWN;
- long timeout = 0;
if (rm_id != subflow->local_id)
continue;
spin_unlock_bh(&msk->pm.lock);
mptcp_subflow_shutdown(sk, ssk, how);
- __mptcp_close_ssk(sk, ssk, subflow, timeout);
+ __mptcp_close_ssk(sk, ssk, subflow);
spin_lock_bh(&msk->pm.lock);
msk->pm.local_addr_used--;
#include <net/transp_v6.h>
#endif
#include <net/mptcp.h>
+#include <net/xfrm.h>
#include "protocol.h"
#include "mib.h"
static struct percpu_counter mptcp_sockets_allocated;
+static void __mptcp_destroy_sock(struct sock *sk);
+static void __mptcp_check_send_data_fin(struct sock *sk);
+
/* If msk has an initial subflow socket, and the MP_CAPABLE handshake has not
* completed yet or has failed, return the subflow socket.
* Otherwise return NULL.
return msk->subflow;
}
+/* Returns end sequence number of the receiver's advertised window */
+static u64 mptcp_wnd_end(const struct mptcp_sock *msk)
+{
+ return READ_ONCE(msk->wnd_end);
+}
+
static bool mptcp_is_tcpsk(struct sock *sk)
{
struct socket *sock = sk->sk_socket;
msk->subflow = ssock;
subflow = mptcp_subflow_ctx(ssock->sk);
list_add(&subflow->node, &msk->conn_list);
+ sock_hold(ssock->sk);
subflow->request_mptcp = 1;
/* accept() will wait on first subflow sk_wq, and we always wakes up
struct rb_node **p, *parent;
u64 seq, end_seq, max_seq;
struct sk_buff *skb1;
- int space;
seq = MPTCP_SKB_CB(skb)->map_seq;
end_seq = MPTCP_SKB_CB(skb)->end_seq;
- space = tcp_space(sk);
- max_seq = space > 0 ? space + msk->ack_seq : msk->ack_seq;
+ max_seq = READ_ONCE(msk->rcv_wnd_sent);
pr_debug("msk=%p seq=%llx limit=%llx empty=%d", msk, seq, max_seq,
RB_EMPTY_ROOT(&msk->out_of_order_queue));
- if (after64(seq, max_seq)) {
+ if (after64(end_seq, max_seq)) {
/* out of window */
mptcp_drop(sk, skb);
+ pr_debug("oow by %lld, rcv_wnd_sent %llu\n",
+ (unsigned long long)end_seq - (unsigned long)max_seq,
+ (unsigned long long)msk->rcv_wnd_sent);
MPTCP_INC_STATS(sock_net(sk), MPTCP_MIB_NODSSWINDOW);
return;
}
mptcp_sk(sk)->timer_ival = 0;
}
-static void mptcp_check_data_fin_ack(struct sock *sk)
+static void mptcp_close_wake_up(struct sock *sk)
+{
+ if (sock_flag(sk, SOCK_DEAD))
+ return;
+
+ sk->sk_state_change(sk);
+ if (sk->sk_shutdown == SHUTDOWN_MASK ||
+ sk->sk_state == TCP_CLOSE)
+ sk_wake_async(sk, SOCK_WAKE_WAITD, POLL_HUP);
+ else
+ sk_wake_async(sk, SOCK_WAKE_WAITD, POLL_IN);
+}
+
+static bool mptcp_pending_data_fin_ack(struct sock *sk)
{
struct mptcp_sock *msk = mptcp_sk(sk);
- if (__mptcp_check_fallback(msk))
- return;
+ return !__mptcp_check_fallback(msk) &&
+ ((1 << sk->sk_state) &
+ (TCPF_FIN_WAIT1 | TCPF_CLOSING | TCPF_LAST_ACK)) &&
+ msk->write_seq == READ_ONCE(msk->snd_una);
+}
+
+static void mptcp_check_data_fin_ack(struct sock *sk)
+{
+ struct mptcp_sock *msk = mptcp_sk(sk);
/* Look for an acknowledged DATA_FIN */
- if (((1 << sk->sk_state) &
- (TCPF_FIN_WAIT1 | TCPF_CLOSING | TCPF_LAST_ACK)) &&
- msk->write_seq == atomic64_read(&msk->snd_una)) {
+ if (mptcp_pending_data_fin_ack(sk)) {
mptcp_stop_timer(sk);
WRITE_ONCE(msk->snd_data_fin_enable, 0);
switch (sk->sk_state) {
case TCP_FIN_WAIT1:
inet_sk_state_store(sk, TCP_FIN_WAIT2);
- sk->sk_state_change(sk);
break;
case TCP_CLOSING:
case TCP_LAST_ACK:
inet_sk_state_store(sk, TCP_CLOSE);
- sk->sk_state_change(sk);
break;
}
- if (sk->sk_shutdown == SHUTDOWN_MASK ||
- sk->sk_state == TCP_CLOSE)
- sk_wake_async(sk, SOCK_WAKE_WAITD, POLL_HUP);
- else
- sk_wake_async(sk, SOCK_WAKE_WAITD, POLL_IN);
+ mptcp_close_wake_up(sk);
}
}
mptcp_sk(sk)->timer_ival = tout > 0 ? tout : TCP_RTO_MIN;
}
-static void mptcp_check_data_fin(struct sock *sk)
+static bool mptcp_subflow_active(struct mptcp_subflow_context *subflow)
+{
+ struct sock *ssk = mptcp_subflow_tcp_sock(subflow);
+
+ /* can't send if JOIN hasn't completed yet (i.e. is usable for mptcp) */
+ if (subflow->request_join && !subflow->fully_established)
+ return false;
+
+ /* only send if our side has not closed yet */
+ return ((1 << ssk->sk_state) & (TCPF_ESTABLISHED | TCPF_CLOSE_WAIT));
+}
+
+static bool tcp_can_send_ack(const struct sock *ssk)
+{
+ return !((1 << inet_sk_state_load(ssk)) &
+ (TCPF_SYN_SENT | TCPF_SYN_RECV | TCPF_TIME_WAIT | TCPF_CLOSE));
+}
+
+static void mptcp_send_ack(struct mptcp_sock *msk)
+{
+ struct mptcp_subflow_context *subflow;
+
+ mptcp_for_each_subflow(msk, subflow) {
+ struct sock *ssk = mptcp_subflow_tcp_sock(subflow);
+
+ lock_sock(ssk);
+ if (tcp_can_send_ack(ssk))
+ tcp_send_ack(ssk);
+ release_sock(ssk);
+ }
+}
+
+static bool mptcp_subflow_cleanup_rbuf(struct sock *ssk)
+{
+ int ret;
+
+ lock_sock(ssk);
+ ret = tcp_can_send_ack(ssk);
+ if (ret)
+ tcp_cleanup_rbuf(ssk, 1);
+ release_sock(ssk);
+ return ret;
+}
+
+static void mptcp_cleanup_rbuf(struct mptcp_sock *msk)
+{
+ struct sock *ack_hint = READ_ONCE(msk->ack_hint);
+ struct mptcp_subflow_context *subflow;
+
+ /* if the hinted ssk is still active, try to use it */
+ if (likely(ack_hint)) {
+ mptcp_for_each_subflow(msk, subflow) {
+ struct sock *ssk = mptcp_subflow_tcp_sock(subflow);
+
+ if (ack_hint == ssk && mptcp_subflow_cleanup_rbuf(ssk))
+ return;
+ }
+ }
+
+ /* otherwise pick the first active subflow */
+ mptcp_for_each_subflow(msk, subflow)
+ if (mptcp_subflow_cleanup_rbuf(mptcp_subflow_tcp_sock(subflow)))
+ return;
+}
+
+static bool mptcp_check_data_fin(struct sock *sk)
{
struct mptcp_sock *msk = mptcp_sk(sk);
u64 rcv_data_fin_seq;
+ bool ret = false;
if (__mptcp_check_fallback(msk) || !msk->first)
- return;
+ return ret;
/* Need to ack a DATA_FIN received from a peer while this side
* of the connection is in ESTABLISHED, FIN_WAIT1, or FIN_WAIT2.
*/
if (mptcp_pending_data_fin(sk, &rcv_data_fin_seq)) {
- struct mptcp_subflow_context *subflow;
-
WRITE_ONCE(msk->ack_seq, msk->ack_seq + 1);
WRITE_ONCE(msk->rcv_data_fin, 0);
break;
case TCP_FIN_WAIT2:
inet_sk_state_store(sk, TCP_CLOSE);
- // @@ Close subflows now?
break;
default:
/* Other states not expected */
break;
}
+ ret = true;
mptcp_set_timeout(sk, NULL);
- mptcp_for_each_subflow(msk, subflow) {
- struct sock *ssk = mptcp_subflow_tcp_sock(subflow);
-
- lock_sock(ssk);
- tcp_send_ack(ssk);
- release_sock(ssk);
- }
-
- sk->sk_state_change(sk);
-
- if (sk->sk_shutdown == SHUTDOWN_MASK ||
- sk->sk_state == TCP_CLOSE)
- sk_wake_async(sk, SOCK_WAKE_WAITD, POLL_HUP);
- else
- sk_wake_async(sk, SOCK_WAKE_WAITD, POLL_IN);
+ mptcp_send_ack(msk);
+ mptcp_close_wake_up(sk);
}
+ return ret;
}
static bool __mptcp_move_skbs_from_subflow(struct mptcp_sock *msk,
unsigned int moved = 0;
bool more_data_avail;
struct tcp_sock *tp;
- u32 old_copied_seq;
bool done = false;
int sk_rbuf;
pr_debug("msk=%p ssk=%p", msk, ssk);
tp = tcp_sk(ssk);
- old_copied_seq = tp->copied_seq;
do {
u32 map_remaining, offset;
u32 seq = tp->copied_seq;
break;
}
} while (more_data_avail);
+ WRITE_ONCE(msk->ack_hint, ssk);
*bytes += moved;
- if (tp->copied_seq != old_copied_seq)
- tcp_cleanup_rbuf(ssk, 1);
-
return done;
}
-static bool mptcp_ofo_queue(struct mptcp_sock *msk)
+static bool __mptcp_ofo_queue(struct mptcp_sock *msk)
{
struct sock *sk = (struct sock *)msk;
struct sk_buff *skb, *tail;
/* In most cases we will be able to lock the mptcp socket. If its already
* owned, we need to defer to the work queue to avoid ABBA deadlock.
*/
-static bool move_skbs_to_msk(struct mptcp_sock *msk, struct sock *ssk)
+static void move_skbs_to_msk(struct mptcp_sock *msk, struct sock *ssk)
{
struct sock *sk = (struct sock *)msk;
unsigned int moved = 0;
- if (READ_ONCE(sk->sk_lock.owned))
- return false;
-
- if (unlikely(!spin_trylock_bh(&sk->sk_lock.slock)))
- return false;
-
- /* must re-check after taking the lock */
- if (!READ_ONCE(sk->sk_lock.owned)) {
- __mptcp_move_skbs_from_subflow(msk, ssk, &moved);
- mptcp_ofo_queue(msk);
+ if (inet_sk_state_load(sk) == TCP_CLOSE)
+ return;
- /* If the moves have caught up with the DATA_FIN sequence number
- * it's time to ack the DATA_FIN and change socket state, but
- * this is not a good place to change state. Let the workqueue
- * do it.
- */
- if (mptcp_pending_data_fin(sk, NULL) &&
- schedule_work(&msk->work))
- sock_hold(sk);
- }
+ mptcp_data_lock(sk);
- spin_unlock_bh(&sk->sk_lock.slock);
+ __mptcp_move_skbs_from_subflow(msk, ssk, &moved);
+ __mptcp_ofo_queue(msk);
- return moved > 0;
+ /* If the moves have caught up with the DATA_FIN sequence number
+ * it's time to ack the DATA_FIN and change socket state, but
+ * this is not a good place to change state. Let the workqueue
+ * do it.
+ */
+ if (mptcp_pending_data_fin(sk, NULL))
+ mptcp_schedule_work(sk);
+ mptcp_data_unlock(sk);
}
void mptcp_data_ready(struct sock *sk, struct sock *ssk)
if (atomic_read(&sk->sk_rmem_alloc) > sk_rbuf)
goto wake;
- if (move_skbs_to_msk(msk, ssk))
- goto wake;
-
- /* mptcp socket is owned, release_cb should retry */
- if (!test_and_set_bit(TCP_DELACK_TIMER_DEFERRED,
- &sk->sk_tsq_flags)) {
- sock_hold(sk);
+ move_skbs_to_msk(msk, ssk);
- /* need to try again, its possible release_cb() has already
- * been called after the test_and_set_bit() above.
- */
- move_skbs_to_msk(msk, ssk);
- }
wake:
if (wake)
sk->sk_data_ready(sk);
}
-static void __mptcp_flush_join_list(struct mptcp_sock *msk)
+void __mptcp_flush_join_list(struct mptcp_sock *msk)
{
if (likely(list_empty(&msk->join_list)))
return;
struct inet_connection_sock *icsk = inet_csk(sk);
unsigned long tout;
+ /* prevent rescheduling on close */
+ if (unlikely(inet_sk_state_load(sk) == TCP_CLOSE))
+ return;
+
/* should never be called with mptcp level timer cleared */
tout = READ_ONCE(mptcp_sk(sk)->timer_ival);
if (WARN_ON_ONCE(!tout))
sk_reset_timer(sk, &icsk->icsk_retransmit_timer, jiffies + tout);
}
-void mptcp_data_acked(struct sock *sk)
+bool mptcp_schedule_work(struct sock *sk)
{
- mptcp_reset_timer(sk);
-
- if ((!test_bit(MPTCP_SEND_SPACE, &mptcp_sk(sk)->flags) ||
- (inet_sk_state_load(sk) != TCP_ESTABLISHED)) &&
- schedule_work(&mptcp_sk(sk)->work))
+ if (inet_sk_state_load(sk) != TCP_CLOSE &&
+ schedule_work(&mptcp_sk(sk)->work)) {
+ /* each subflow already holds a reference to the sk, and the
+ * workqueue is invoked by a subflow, so sk can't go away here.
+ */
sock_hold(sk);
+ return true;
+ }
+ return false;
}
void mptcp_subflow_eof(struct sock *sk)
{
- struct mptcp_sock *msk = mptcp_sk(sk);
-
- if (!test_and_set_bit(MPTCP_WORK_EOF, &msk->flags) &&
- schedule_work(&msk->work))
- sock_hold(sk);
+ if (!test_and_set_bit(MPTCP_WORK_EOF, &mptcp_sk(sk)->flags))
+ mptcp_schedule_work(sk);
}
static void mptcp_check_for_eof(struct mptcp_sock *msk)
mptcp_for_each_subflow(msk, subflow)
receivers += !subflow->rx_eof;
+ if (receivers)
+ return;
- if (!receivers && !(sk->sk_shutdown & RCV_SHUTDOWN)) {
+ if (!(sk->sk_shutdown & RCV_SHUTDOWN)) {
/* hopefully temporary hack: propagate shutdown status
* to msk, when all subflows agree on it
*/
set_bit(MPTCP_DATA_READY, &msk->flags);
sk->sk_data_ready(sk);
}
-}
-
-static bool mptcp_ext_cache_refill(struct mptcp_sock *msk)
-{
- const struct sock *sk = (const struct sock *)msk;
-
- if (!msk->cached_ext)
- msk->cached_ext = __skb_ext_alloc(sk->sk_allocation);
- return !!msk->cached_ext;
+ switch (sk->sk_state) {
+ case TCP_ESTABLISHED:
+ inet_sk_state_store(sk, TCP_CLOSE_WAIT);
+ break;
+ case TCP_FIN_WAIT1:
+ inet_sk_state_store(sk, TCP_CLOSING);
+ break;
+ case TCP_FIN_WAIT2:
+ inet_sk_state_store(sk, TCP_CLOSE);
+ break;
+ default:
+ return;
+ }
+ mptcp_close_wake_up(sk);
}
static struct sock *mptcp_subflow_recv_lookup(const struct mptcp_sock *msk)
const struct mptcp_data_frag *df)
{
return df && pfrag->page == df->page &&
+ pfrag->size - pfrag->offset > 0 &&
df->data_seq + df->data_len == msk->write_seq;
}
+static int mptcp_wmem_with_overhead(struct sock *sk, int size)
+{
+ struct mptcp_sock *msk = mptcp_sk(sk);
+ int ret, skbs;
+
+ ret = size + ((sizeof(struct mptcp_data_frag) * size) >> PAGE_SHIFT);
+ skbs = (msk->tx_pending_data + size) / msk->size_goal_cache;
+ if (skbs < msk->skb_tx_cache.qlen)
+ return ret;
+
+ return ret + (skbs - msk->skb_tx_cache.qlen) * SKB_TRUESIZE(MAX_TCP_HEADER);
+}
+
+static void __mptcp_wmem_reserve(struct sock *sk, int size)
+{
+ int amount = mptcp_wmem_with_overhead(sk, size);
+ struct mptcp_sock *msk = mptcp_sk(sk);
+
+ WARN_ON_ONCE(msk->wmem_reserved);
+ if (amount <= sk->sk_forward_alloc)
+ goto reserve;
+
+ /* under memory pressure try to reserve at most a single page
+ * otherwise try to reserve the full estimate and fallback
+ * to a single page before entering the error path
+ */
+ if ((tcp_under_memory_pressure(sk) && amount > PAGE_SIZE) ||
+ !sk_wmem_schedule(sk, amount)) {
+ if (amount <= PAGE_SIZE)
+ goto nomem;
+
+ amount = PAGE_SIZE;
+ if (!sk_wmem_schedule(sk, amount))
+ goto nomem;
+ }
+
+reserve:
+ msk->wmem_reserved = amount;
+ sk->sk_forward_alloc -= amount;
+ return;
+
+nomem:
+ /* we will wait for memory on next allocation */
+ msk->wmem_reserved = -1;
+}
+
+static void __mptcp_update_wmem(struct sock *sk)
+{
+ struct mptcp_sock *msk = mptcp_sk(sk);
+
+ if (!msk->wmem_reserved)
+ return;
+
+ if (msk->wmem_reserved < 0)
+ msk->wmem_reserved = 0;
+ if (msk->wmem_reserved > 0) {
+ sk->sk_forward_alloc += msk->wmem_reserved;
+ msk->wmem_reserved = 0;
+ }
+}
+
+static bool mptcp_wmem_alloc(struct sock *sk, int size)
+{
+ struct mptcp_sock *msk = mptcp_sk(sk);
+
+ /* check for pre-existing error condition */
+ if (msk->wmem_reserved < 0)
+ return false;
+
+ if (msk->wmem_reserved >= size)
+ goto account;
+
+ mptcp_data_lock(sk);
+ if (!sk_wmem_schedule(sk, size)) {
+ mptcp_data_unlock(sk);
+ return false;
+ }
+
+ sk->sk_forward_alloc -= size;
+ msk->wmem_reserved += size;
+ mptcp_data_unlock(sk);
+
+account:
+ msk->wmem_reserved -= size;
+ return true;
+}
+
+static void mptcp_wmem_uncharge(struct sock *sk, int size)
+{
+ struct mptcp_sock *msk = mptcp_sk(sk);
+
+ if (msk->wmem_reserved < 0)
+ msk->wmem_reserved = 0;
+ msk->wmem_reserved += size;
+}
+
+static void mptcp_mem_reclaim_partial(struct sock *sk)
+{
+ struct mptcp_sock *msk = mptcp_sk(sk);
+
+ /* if we are experiencing a transint allocation error,
+ * the forward allocation memory has been already
+ * released
+ */
+ if (msk->wmem_reserved < 0)
+ return;
+
+ mptcp_data_lock(sk);
+ sk->sk_forward_alloc += msk->wmem_reserved;
+ sk_mem_reclaim_partial(sk);
+ msk->wmem_reserved = sk->sk_forward_alloc;
+ sk->sk_forward_alloc = 0;
+ mptcp_data_unlock(sk);
+}
+
static void dfrag_uncharge(struct sock *sk, int len)
{
sk_mem_uncharge(sk, len);
put_page(dfrag->page);
}
-static bool mptcp_is_writeable(struct mptcp_sock *msk)
-{
- struct mptcp_subflow_context *subflow;
-
- if (!sk_stream_is_writeable((struct sock *)msk))
- return false;
-
- mptcp_for_each_subflow(msk, subflow) {
- if (sk_stream_is_writeable(subflow->tcp_sock))
- return true;
- }
- return false;
-}
-
-static void mptcp_clean_una(struct sock *sk)
+static void __mptcp_clean_una(struct sock *sk)
{
struct mptcp_sock *msk = mptcp_sk(sk);
struct mptcp_data_frag *dtmp, *dfrag;
* plain TCP
*/
if (__mptcp_check_fallback(msk))
- atomic64_set(&msk->snd_una, msk->write_seq);
- snd_una = atomic64_read(&msk->snd_una);
+ msk->snd_una = READ_ONCE(msk->snd_nxt);
+ snd_una = msk->snd_una;
list_for_each_entry_safe(dfrag, dtmp, &msk->rtx_queue, list) {
if (after64(dfrag->data_seq + dfrag->data_len, snd_una))
break;
+ if (WARN_ON_ONCE(dfrag == msk->first_pending))
+ break;
dfrag_clear(sk, dfrag);
cleaned = true;
}
if (dfrag && after64(snd_una, dfrag->data_seq)) {
u64 delta = snd_una - dfrag->data_seq;
- if (WARN_ON_ONCE(delta > dfrag->data_len))
+ if (WARN_ON_ONCE(delta > dfrag->already_sent))
goto out;
dfrag->data_seq += delta;
dfrag->offset += delta;
dfrag->data_len -= delta;
+ dfrag->already_sent -= delta;
dfrag_uncharge(sk, delta);
cleaned = true;
}
out:
- if (cleaned)
- sk_mem_reclaim_partial(sk);
+ if (cleaned) {
+ if (tcp_under_memory_pressure(sk)) {
+ __mptcp_update_wmem(sk);
+ sk_mem_reclaim_partial(sk);
+ }
+
+ if (sk_stream_is_writeable(sk)) {
+ /* pairs with memory barrier in mptcp_poll */
+ smp_mb();
+ if (test_and_clear_bit(MPTCP_NOSPACE, &msk->flags))
+ sk_stream_write_space(sk);
+ }
+ }
+
+ if (snd_una == READ_ONCE(msk->snd_nxt)) {
+ if (msk->timer_ival)
+ mptcp_stop_timer(sk);
+ } else {
+ mptcp_reset_timer(sk);
+ }
}
-static void mptcp_clean_una_wakeup(struct sock *sk)
+static void mptcp_enter_memory_pressure(struct sock *sk)
{
+ struct mptcp_subflow_context *subflow;
struct mptcp_sock *msk = mptcp_sk(sk);
+ bool first = true;
- mptcp_clean_una(sk);
-
- /* Only wake up writers if a subflow is ready */
- if (mptcp_is_writeable(msk)) {
- set_bit(MPTCP_SEND_SPACE, &msk->flags);
- smp_mb__after_atomic();
+ sk_stream_moderate_sndbuf(sk);
+ mptcp_for_each_subflow(msk, subflow) {
+ struct sock *ssk = mptcp_subflow_tcp_sock(subflow);
- /* set SEND_SPACE before sk_stream_write_space clears
- * NOSPACE
- */
- sk_stream_write_space(sk);
+ if (first)
+ tcp_enter_memory_pressure(ssk);
+ sk_stream_moderate_sndbuf(ssk);
+ first = false;
}
}
pfrag, sk->sk_allocation)))
return true;
- sk->sk_prot->enter_memory_pressure(sk);
- sk_stream_moderate_sndbuf(sk);
+ mptcp_enter_memory_pressure(sk);
return false;
}
dfrag->data_seq = msk->write_seq;
dfrag->overhead = offset - orig_offset + sizeof(struct mptcp_data_frag);
dfrag->offset = offset + sizeof(struct mptcp_data_frag);
+ dfrag->already_sent = 0;
dfrag->page = pfrag->page;
return dfrag;
}
+struct mptcp_sendmsg_info {
+ int mss_now;
+ int size_goal;
+ u16 limit;
+ u16 sent;
+ unsigned int flags;
+};
+
+static int mptcp_check_allowed_size(struct mptcp_sock *msk, u64 data_seq,
+ int avail_size)
+{
+ u64 window_end = mptcp_wnd_end(msk);
+
+ if (__mptcp_check_fallback(msk))
+ return avail_size;
+
+ if (!before64(data_seq + avail_size, window_end)) {
+ u64 allowed_size = window_end - data_seq;
+
+ return min_t(unsigned int, allowed_size, avail_size);
+ }
+
+ return avail_size;
+}
+
+static bool __mptcp_add_ext(struct sk_buff *skb, gfp_t gfp)
+{
+ struct skb_ext *mpext = __skb_ext_alloc(gfp);
+
+ if (!mpext)
+ return false;
+ __skb_ext_set(skb, SKB_EXT_MPTCP, mpext);
+ return true;
+}
+
+static struct sk_buff *__mptcp_do_alloc_tx_skb(struct sock *sk, gfp_t gfp)
+{
+ struct sk_buff *skb;
+
+ skb = alloc_skb_fclone(MAX_TCP_HEADER, gfp);
+ if (likely(skb)) {
+ if (likely(__mptcp_add_ext(skb, gfp))) {
+ skb_reserve(skb, MAX_TCP_HEADER);
+ skb->reserved_tailroom = skb->end - skb->tail;
+ return skb;
+ }
+ __kfree_skb(skb);
+ } else {
+ mptcp_enter_memory_pressure(sk);
+ }
+ return NULL;
+}
+
+static bool mptcp_tx_cache_refill(struct sock *sk, int size,
+ struct sk_buff_head *skbs, int *total_ts)
+{
+ struct mptcp_sock *msk = mptcp_sk(sk);
+ struct sk_buff *skb;
+ int space_needed;
+
+ if (unlikely(tcp_under_memory_pressure(sk))) {
+ mptcp_mem_reclaim_partial(sk);
+
+ /* under pressure pre-allocate at most a single skb */
+ if (msk->skb_tx_cache.qlen)
+ return true;
+ space_needed = msk->size_goal_cache;
+ } else {
+ space_needed = msk->tx_pending_data + size -
+ msk->skb_tx_cache.qlen * msk->size_goal_cache;
+ }
+
+ while (space_needed > 0) {
+ skb = __mptcp_do_alloc_tx_skb(sk, sk->sk_allocation);
+ if (unlikely(!skb)) {
+ /* under memory pressure, try to pass the caller a
+ * single skb to allow forward progress
+ */
+ while (skbs->qlen > 1) {
+ skb = __skb_dequeue_tail(skbs);
+ __kfree_skb(skb);
+ }
+ return skbs->qlen > 0;
+ }
+
+ *total_ts += skb->truesize;
+ __skb_queue_tail(skbs, skb);
+ space_needed -= msk->size_goal_cache;
+ }
+ return true;
+}
+
+static bool __mptcp_alloc_tx_skb(struct sock *sk, struct sock *ssk, gfp_t gfp)
+{
+ struct mptcp_sock *msk = mptcp_sk(sk);
+ struct sk_buff *skb;
+
+ if (ssk->sk_tx_skb_cache) {
+ skb = ssk->sk_tx_skb_cache;
+ if (unlikely(!skb_ext_find(skb, SKB_EXT_MPTCP) &&
+ !__mptcp_add_ext(skb, gfp)))
+ return false;
+ return true;
+ }
+
+ skb = skb_peek(&msk->skb_tx_cache);
+ if (skb) {
+ if (likely(sk_wmem_schedule(ssk, skb->truesize))) {
+ skb = __skb_dequeue(&msk->skb_tx_cache);
+ if (WARN_ON_ONCE(!skb))
+ return false;
+
+ mptcp_wmem_uncharge(sk, skb->truesize);
+ ssk->sk_tx_skb_cache = skb;
+ return true;
+ }
+
+ /* over memory limit, no point to try to allocate a new skb */
+ return false;
+ }
+
+ skb = __mptcp_do_alloc_tx_skb(sk, gfp);
+ if (!skb)
+ return false;
+
+ if (likely(sk_wmem_schedule(ssk, skb->truesize))) {
+ ssk->sk_tx_skb_cache = skb;
+ return true;
+ }
+ kfree_skb(skb);
+ return false;
+}
+
+static bool mptcp_must_reclaim_memory(struct sock *sk, struct sock *ssk)
+{
+ return !ssk->sk_tx_skb_cache &&
+ !skb_peek(&mptcp_sk(sk)->skb_tx_cache) &&
+ tcp_under_memory_pressure(sk);
+}
+
+static bool mptcp_alloc_tx_skb(struct sock *sk, struct sock *ssk)
+{
+ if (unlikely(mptcp_must_reclaim_memory(sk, ssk)))
+ mptcp_mem_reclaim_partial(sk);
+ return __mptcp_alloc_tx_skb(sk, ssk, sk->sk_allocation);
+}
+
static int mptcp_sendmsg_frag(struct sock *sk, struct sock *ssk,
- struct msghdr *msg, struct mptcp_data_frag *dfrag,
- long *timeo, int *pmss_now,
- int *ps_goal)
+ struct mptcp_data_frag *dfrag,
+ struct mptcp_sendmsg_info *info)
{
- int mss_now, avail_size, size_goal, offset, ret, frag_truesize = 0;
- bool dfrag_collapsed, can_collapse = false;
+ u64 data_seq = dfrag->data_seq + info->sent;
struct mptcp_sock *msk = mptcp_sk(sk);
+ bool zero_window_probe = false;
struct mptcp_ext *mpext = NULL;
- bool retransmission = !!dfrag;
struct sk_buff *skb, *tail;
- struct page_frag *pfrag;
- struct page *page;
- u64 *write_seq;
- size_t psize;
-
- /* use the mptcp page cache so that we can easily move the data
- * from one substream to another, but do per subflow memory accounting
- * Note: pfrag is used only !retransmission, but the compiler if
- * fooled into a warning if we don't init here
- */
- pfrag = sk_page_frag(sk);
- if (!retransmission) {
- write_seq = &msk->write_seq;
- page = pfrag->page;
- } else {
- write_seq = &dfrag->data_seq;
- page = dfrag->page;
- }
+ bool can_collapse = false;
+ int avail_size;
+ size_t ret = 0;
- /* compute copy limit */
- mss_now = tcp_send_mss(ssk, &size_goal, msg->msg_flags);
- *pmss_now = mss_now;
- *ps_goal = size_goal;
- avail_size = size_goal;
+ pr_debug("msk=%p ssk=%p sending dfrag at seq=%lld len=%d already sent=%d",
+ msk, ssk, dfrag->data_seq, dfrag->data_len, info->sent);
+
+ /* compute send limit */
+ info->mss_now = tcp_send_mss(ssk, &info->size_goal, info->flags);
+ avail_size = info->size_goal;
+ msk->size_goal_cache = info->size_goal;
skb = tcp_write_queue_tail(ssk);
if (skb) {
- mpext = skb_ext_find(skb, SKB_EXT_MPTCP);
-
/* Limit the write to the size available in the
* current skb, if any, so that we create at most a new skb.
* Explicitly tells TCP internals to avoid collapsing on later
* queue management operation, to avoid breaking the ext <->
* SSN association set here
*/
- can_collapse = (size_goal - skb->len > 0) &&
- mptcp_skb_can_collapse_to(*write_seq, skb, mpext);
+ mpext = skb_ext_find(skb, SKB_EXT_MPTCP);
+ can_collapse = (info->size_goal - skb->len > 0) &&
+ mptcp_skb_can_collapse_to(data_seq, skb, mpext);
if (!can_collapse)
TCP_SKB_CB(skb)->eor = 1;
else
- avail_size = size_goal - skb->len;
+ avail_size = info->size_goal - skb->len;
}
- if (!retransmission) {
- /* reuse tail pfrag, if possible, or carve a new one from the
- * page allocator
- */
- dfrag = mptcp_rtx_tail(sk);
- offset = pfrag->offset;
- dfrag_collapsed = mptcp_frag_can_collapse_to(msk, pfrag, dfrag);
- if (!dfrag_collapsed) {
- dfrag = mptcp_carve_data_frag(msk, pfrag, offset);
- offset = dfrag->offset;
- frag_truesize = dfrag->overhead;
- }
- psize = min_t(size_t, pfrag->size - offset, avail_size);
-
- /* Copy to page */
- pr_debug("left=%zu", msg_data_left(msg));
- psize = copy_page_from_iter(pfrag->page, offset,
- min_t(size_t, msg_data_left(msg),
- psize),
- &msg->msg_iter);
- pr_debug("left=%zu", msg_data_left(msg));
- if (!psize)
- return -EINVAL;
-
- if (!sk_wmem_schedule(sk, psize + dfrag->overhead)) {
- iov_iter_revert(&msg->msg_iter, psize);
- return -ENOMEM;
- }
- } else {
- offset = dfrag->offset;
- psize = min_t(size_t, dfrag->data_len, avail_size);
- }
+ /* Zero window and all data acked? Probe. */
+ avail_size = mptcp_check_allowed_size(msk, data_seq, avail_size);
+ if (avail_size == 0) {
+ u64 snd_una = READ_ONCE(msk->snd_una);
- /* tell the TCP stack to delay the push so that we can safely
- * access the skb after the sendpages call
- */
- ret = do_tcp_sendpages(ssk, page, offset, psize,
- msg->msg_flags | MSG_SENDPAGE_NOTLAST | MSG_DONTWAIT);
- if (ret <= 0) {
- if (!retransmission)
- iov_iter_revert(&msg->msg_iter, psize);
- return ret;
+ if (skb || snd_una != msk->snd_nxt)
+ return 0;
+ zero_window_probe = true;
+ data_seq = snd_una - 1;
+ avail_size = 1;
}
- frag_truesize += ret;
- if (!retransmission) {
- if (unlikely(ret < psize))
- iov_iter_revert(&msg->msg_iter, psize - ret);
+ if (WARN_ON_ONCE(info->sent > info->limit ||
+ info->limit > dfrag->data_len))
+ return 0;
- /* send successful, keep track of sent data for mptcp-level
- * retransmission
- */
- dfrag->data_len += ret;
- if (!dfrag_collapsed) {
- get_page(dfrag->page);
- list_add_tail(&dfrag->list, &msk->rtx_queue);
- sk_wmem_queued_add(sk, frag_truesize);
- } else {
- sk_wmem_queued_add(sk, ret);
- }
-
- /* charge data on mptcp rtx queue to the master socket
- * Note: we charge such data both to sk and ssk
- */
- sk->sk_forward_alloc -= frag_truesize;
+ ret = info->limit - info->sent;
+ tail = tcp_build_frag(ssk, avail_size, info->flags, dfrag->page,
+ dfrag->offset + info->sent, &ret);
+ if (!tail) {
+ tcp_remove_empty_skb(sk, tcp_write_queue_tail(ssk));
+ return -ENOMEM;
}
- /* if the tail skb extension is still the cached one, collapsing
- * really happened. Note: we can't check for 'same skb' as the sk_buff
- * hdr on tail can be transmitted, freed and re-allocated by the
- * do_tcp_sendpages() call
+ /* if the tail skb is still the cached one, collapsing really happened.
*/
- tail = tcp_write_queue_tail(ssk);
- if (mpext && tail && mpext == skb_ext_find(tail, SKB_EXT_MPTCP)) {
+ if (skb == tail) {
WARN_ON_ONCE(!can_collapse);
mpext->data_len += ret;
+ WARN_ON_ONCE(zero_window_probe);
goto out;
}
- skb = tcp_write_queue_tail(ssk);
- mpext = __skb_ext_set(skb, SKB_EXT_MPTCP, msk->cached_ext);
- msk->cached_ext = NULL;
+ mpext = skb_ext_find(tail, SKB_EXT_MPTCP);
+ if (WARN_ON_ONCE(!mpext)) {
+ /* should never reach here, stream corrupted */
+ return -EINVAL;
+ }
memset(mpext, 0, sizeof(*mpext));
- mpext->data_seq = *write_seq;
+ mpext->data_seq = data_seq;
mpext->subflow_seq = mptcp_subflow_ctx(ssk)->rel_write_seq;
mpext->data_len = ret;
mpext->use_map = 1;
mpext->data_seq, mpext->subflow_seq, mpext->data_len,
mpext->dsn64);
+ if (zero_window_probe) {
+ mptcp_subflow_ctx(ssk)->rel_write_seq += ret;
+ mpext->frozen = 1;
+ ret = 0;
+ tcp_push_pending_frames(ssk);
+ }
out:
- if (!retransmission)
- pfrag->offset += frag_truesize;
- WRITE_ONCE(*write_seq, *write_seq + ret);
mptcp_subflow_ctx(ssk)->rel_write_seq += ret;
-
return ret;
}
-static void mptcp_nospace(struct mptcp_sock *msk)
-{
- struct mptcp_subflow_context *subflow;
-
- clear_bit(MPTCP_SEND_SPACE, &msk->flags);
- smp_mb__after_atomic(); /* msk->flags is changed by write_space cb */
-
- mptcp_for_each_subflow(msk, subflow) {
- struct sock *ssk = mptcp_subflow_tcp_sock(subflow);
- struct socket *sock = READ_ONCE(ssk->sk_socket);
-
- /* enables ssk->write_space() callbacks */
- if (sock)
- set_bit(SOCK_NOSPACE, &sock->flags);
- }
-}
-
-static bool mptcp_subflow_active(struct mptcp_subflow_context *subflow)
-{
- struct sock *ssk = mptcp_subflow_tcp_sock(subflow);
-
- /* can't send if JOIN hasn't completed yet (i.e. is usable for mptcp) */
- if (subflow->request_join && !subflow->fully_established)
- return false;
-
- /* only send if our side has not closed yet */
- return ((1 << ssk->sk_state) & (TCPF_ESTABLISHED | TCPF_CLOSE_WAIT));
-}
-
#define MPTCP_SEND_BURST_SIZE ((1 << 16) - \
sizeof(struct tcphdr) - \
MAX_TCP_OPTION_SPACE - \
sock_owned_by_me((struct sock *)msk);
*sndbuf = 0;
- if (!mptcp_ext_cache_refill(msk))
- return NULL;
-
if (__mptcp_check_fallback(msk)) {
if (!msk->first)
return NULL;
return NULL;
}
-static void ssk_check_wmem(struct mptcp_sock *msk)
+static void mptcp_push_release(struct sock *sk, struct sock *ssk,
+ struct mptcp_sendmsg_info *info)
{
- if (unlikely(!mptcp_is_writeable(msk)))
- mptcp_nospace(msk);
+ mptcp_set_timeout(sk, ssk);
+ tcp_push(ssk, 0, info->mss_now, tcp_sk(ssk)->nonagle, info->size_goal);
+ release_sock(ssk);
+}
+
+static void mptcp_push_pending(struct sock *sk, unsigned int flags)
+{
+ struct sock *prev_ssk = NULL, *ssk = NULL;
+ struct mptcp_sock *msk = mptcp_sk(sk);
+ struct mptcp_sendmsg_info info = {
+ .flags = flags,
+ };
+ struct mptcp_data_frag *dfrag;
+ int len, copied = 0;
+ u32 sndbuf;
+
+ while ((dfrag = mptcp_send_head(sk))) {
+ info.sent = dfrag->already_sent;
+ info.limit = dfrag->data_len;
+ len = dfrag->data_len - dfrag->already_sent;
+ while (len > 0) {
+ int ret = 0;
+
+ prev_ssk = ssk;
+ __mptcp_flush_join_list(msk);
+ ssk = mptcp_subflow_get_send(msk, &sndbuf);
+
+ /* do auto tuning */
+ if (!(sk->sk_userlocks & SOCK_SNDBUF_LOCK) &&
+ sndbuf > READ_ONCE(sk->sk_sndbuf))
+ WRITE_ONCE(sk->sk_sndbuf, sndbuf);
+
+ /* try to keep the subflow socket lock across
+ * consecutive xmit on the same socket
+ */
+ if (ssk != prev_ssk && prev_ssk)
+ mptcp_push_release(sk, prev_ssk, &info);
+ if (!ssk)
+ goto out;
+
+ if (ssk != prev_ssk || !prev_ssk)
+ lock_sock(ssk);
+
+ /* keep it simple and always provide a new skb for the
+ * subflow, even if we will not use it when collapsing
+ * on the pending one
+ */
+ if (!mptcp_alloc_tx_skb(sk, ssk)) {
+ mptcp_push_release(sk, ssk, &info);
+ goto out;
+ }
+
+ ret = mptcp_sendmsg_frag(sk, ssk, dfrag, &info);
+ if (ret <= 0) {
+ mptcp_push_release(sk, ssk, &info);
+ goto out;
+ }
+
+ info.sent += ret;
+ dfrag->already_sent += ret;
+ msk->snd_nxt += ret;
+ msk->snd_burst -= ret;
+ msk->tx_pending_data -= ret;
+ copied += ret;
+ len -= ret;
+ }
+ WRITE_ONCE(msk->first_pending, mptcp_send_next(sk));
+ }
+
+ /* at this point we held the socket lock for the last subflow we used */
+ if (ssk)
+ mptcp_push_release(sk, ssk, &info);
+
+out:
+ if (copied) {
+ /* start the timer, if it's not pending */
+ if (!mptcp_timer_pending(sk))
+ mptcp_reset_timer(sk);
+ __mptcp_check_send_data_fin(sk);
+ }
+}
+
+static void __mptcp_subflow_push_pending(struct sock *sk, struct sock *ssk)
+{
+ struct mptcp_sock *msk = mptcp_sk(sk);
+ struct mptcp_sendmsg_info info;
+ struct mptcp_data_frag *dfrag;
+ int len, copied = 0;
+
+ info.flags = 0;
+ while ((dfrag = mptcp_send_head(sk))) {
+ info.sent = dfrag->already_sent;
+ info.limit = dfrag->data_len;
+ len = dfrag->data_len - dfrag->already_sent;
+ while (len > 0) {
+ int ret = 0;
+
+ /* do auto tuning */
+ if (!(sk->sk_userlocks & SOCK_SNDBUF_LOCK) &&
+ ssk->sk_sndbuf > READ_ONCE(sk->sk_sndbuf))
+ WRITE_ONCE(sk->sk_sndbuf, ssk->sk_sndbuf);
+
+ if (unlikely(mptcp_must_reclaim_memory(sk, ssk))) {
+ __mptcp_update_wmem(sk);
+ sk_mem_reclaim_partial(sk);
+ }
+ if (!__mptcp_alloc_tx_skb(sk, ssk, GFP_ATOMIC))
+ goto out;
+
+ ret = mptcp_sendmsg_frag(sk, ssk, dfrag, &info);
+ if (ret <= 0)
+ goto out;
+
+ info.sent += ret;
+ dfrag->already_sent += ret;
+ msk->snd_nxt += ret;
+ msk->snd_burst -= ret;
+ msk->tx_pending_data -= ret;
+ copied += ret;
+ len -= ret;
+ }
+ WRITE_ONCE(msk->first_pending, mptcp_send_next(sk));
+ }
+
+out:
+ /* __mptcp_alloc_tx_skb could have released some wmem and we are
+ * not going to flush it via release_sock()
+ */
+ __mptcp_update_wmem(sk);
+ if (copied) {
+ mptcp_set_timeout(sk, ssk);
+ tcp_push(ssk, 0, info.mss_now, tcp_sk(ssk)->nonagle,
+ info.size_goal);
+ if (msk->snd_data_fin_enable &&
+ msk->snd_nxt + 1 == msk->write_seq)
+ mptcp_schedule_work(sk);
+ }
}
static int mptcp_sendmsg(struct sock *sk, struct msghdr *msg, size_t len)
{
- int mss_now = 0, size_goal = 0, ret = 0;
struct mptcp_sock *msk = mptcp_sk(sk);
struct page_frag *pfrag;
size_t copied = 0;
- struct sock *ssk;
- u32 sndbuf;
- bool tx_ok;
+ int ret = 0;
long timeo;
if (msg->msg_flags & ~(MSG_MORE | MSG_DONTWAIT | MSG_NOSIGNAL))
return -EOPNOTSUPP;
- lock_sock(sk);
+ mptcp_lock_sock(sk, __mptcp_wmem_reserve(sk, len));
timeo = sock_sndtimeo(sk, msg->msg_flags & MSG_DONTWAIT);
}
pfrag = sk_page_frag(sk);
-restart:
- mptcp_clean_una(sk);
-
- if (sk->sk_err || (sk->sk_shutdown & SEND_SHUTDOWN)) {
- ret = -EPIPE;
- goto out;
- }
-
- __mptcp_flush_join_list(msk);
- ssk = mptcp_subflow_get_send(msk, &sndbuf);
- while (!sk_stream_memory_free(sk) ||
- !ssk ||
- !mptcp_page_frag_refill(ssk, pfrag)) {
- if (ssk) {
- /* make sure retransmit timer is
- * running before we wait for memory.
- *
- * The retransmit timer might be needed
- * to make the peer send an up-to-date
- * MPTCP Ack.
- */
- mptcp_set_timeout(sk, ssk);
- if (!mptcp_timer_pending(sk))
- mptcp_reset_timer(sk);
- }
- mptcp_nospace(msk);
- ret = sk_stream_wait_memory(sk, &timeo);
- if (ret)
- goto out;
+ while (msg_data_left(msg)) {
+ int total_ts, frag_truesize = 0;
+ struct mptcp_data_frag *dfrag;
+ struct sk_buff_head skbs;
+ bool dfrag_collapsed;
+ size_t psize, offset;
- mptcp_clean_una(sk);
-
- ssk = mptcp_subflow_get_send(msk, &sndbuf);
- if (list_empty(&msk->conn_list)) {
- ret = -ENOTCONN;
+ if (sk->sk_err || (sk->sk_shutdown & SEND_SHUTDOWN)) {
+ ret = -EPIPE;
goto out;
}
- }
- /* do auto tuning */
- if (!(sk->sk_userlocks & SOCK_SNDBUF_LOCK) &&
- sndbuf > READ_ONCE(sk->sk_sndbuf))
- WRITE_ONCE(sk->sk_sndbuf, sndbuf);
+ /* reuse tail pfrag, if possible, or carve a new one from the
+ * page allocator
+ */
+ dfrag = mptcp_pending_tail(sk);
+ dfrag_collapsed = mptcp_frag_can_collapse_to(msk, pfrag, dfrag);
+ if (!dfrag_collapsed) {
+ if (!sk_stream_memory_free(sk))
+ goto wait_for_memory;
- pr_debug("conn_list->subflow=%p", ssk);
+ if (!mptcp_page_frag_refill(sk, pfrag))
+ goto wait_for_memory;
- lock_sock(ssk);
- tx_ok = msg_data_left(msg);
- while (tx_ok) {
- ret = mptcp_sendmsg_frag(sk, ssk, msg, NULL, &timeo, &mss_now,
- &size_goal);
- if (ret < 0) {
- if (ret == -EAGAIN && timeo > 0) {
- mptcp_set_timeout(sk, ssk);
- release_sock(ssk);
- goto restart;
- }
- break;
+ dfrag = mptcp_carve_data_frag(msk, pfrag, pfrag->offset);
+ frag_truesize = dfrag->overhead;
}
- /* burst can be negative, we will try move to the next subflow
- * at selection time, if possible.
+ /* we do not bound vs wspace, to allow a single packet.
+ * memory accounting will prevent execessive memory usage
+ * anyway
*/
- msk->snd_burst -= ret;
- copied += ret;
-
- tx_ok = msg_data_left(msg);
- if (!tx_ok)
- break;
+ offset = dfrag->offset + dfrag->data_len;
+ psize = pfrag->size - offset;
+ psize = min_t(size_t, psize, msg_data_left(msg));
+ total_ts = psize + frag_truesize;
+ __skb_queue_head_init(&skbs);
+ if (!mptcp_tx_cache_refill(sk, psize, &skbs, &total_ts))
+ goto wait_for_memory;
+
+ if (!mptcp_wmem_alloc(sk, total_ts)) {
+ __skb_queue_purge(&skbs);
+ goto wait_for_memory;
+ }
- if (!sk_stream_memory_free(ssk) ||
- !mptcp_page_frag_refill(ssk, pfrag) ||
- !mptcp_ext_cache_refill(msk)) {
- tcp_push(ssk, msg->msg_flags, mss_now,
- tcp_sk(ssk)->nonagle, size_goal);
- mptcp_set_timeout(sk, ssk);
- release_sock(ssk);
- goto restart;
+ skb_queue_splice_tail(&skbs, &msk->skb_tx_cache);
+ if (copy_page_from_iter(dfrag->page, offset, psize,
+ &msg->msg_iter) != psize) {
+ mptcp_wmem_uncharge(sk, psize + frag_truesize);
+ ret = -EFAULT;
+ goto out;
}
- /* memory is charged to mptcp level socket as well, i.e.
- * if msg is very large, mptcp socket may run out of buffer
- * space. mptcp_clean_una() will release data that has
- * been acked at mptcp level in the mean time, so there is
- * a good chance we can continue sending data right away.
- *
- * Normally, when the tcp subflow can accept more data, then
- * so can the MPTCP socket. However, we need to cope with
- * peers that might lag behind in their MPTCP-level
- * acknowledgements, i.e. data might have been acked at
- * tcp level only. So, we must also check the MPTCP socket
- * limits before we send more data.
+ /* data successfully copied into the write queue */
+ copied += psize;
+ dfrag->data_len += psize;
+ frag_truesize += psize;
+ pfrag->offset += frag_truesize;
+ WRITE_ONCE(msk->write_seq, msk->write_seq + psize);
+
+ /* charge data on mptcp pending queue to the msk socket
+ * Note: we charge such data both to sk and ssk
*/
- if (unlikely(!sk_stream_memory_free(sk))) {
- tcp_push(ssk, msg->msg_flags, mss_now,
- tcp_sk(ssk)->nonagle, size_goal);
- mptcp_clean_una(sk);
- if (!sk_stream_memory_free(sk)) {
- /* can't send more for now, need to wait for
- * MPTCP-level ACKs from peer.
- *
- * Wakeup will happen via mptcp_clean_una().
- */
- mptcp_set_timeout(sk, ssk);
- release_sock(ssk);
- goto restart;
- }
+ sk_wmem_queued_add(sk, frag_truesize);
+ if (!dfrag_collapsed) {
+ get_page(dfrag->page);
+ list_add_tail(&dfrag->list, &msk->rtx_queue);
+ if (!msk->first_pending)
+ WRITE_ONCE(msk->first_pending, dfrag);
}
+ pr_debug("msk=%p dfrag at seq=%lld len=%d sent=%d new=%d", msk,
+ dfrag->data_seq, dfrag->data_len, dfrag->already_sent,
+ !dfrag_collapsed);
+
+ continue;
+
+wait_for_memory:
+ set_bit(MPTCP_NOSPACE, &msk->flags);
+ mptcp_push_pending(sk, msg->msg_flags);
+ ret = sk_stream_wait_memory(sk, &timeo);
+ if (ret)
+ goto out;
}
- mptcp_set_timeout(sk, ssk);
if (copied) {
- tcp_push(ssk, msg->msg_flags, mss_now, tcp_sk(ssk)->nonagle,
- size_goal);
-
- /* start the timer, if it's not pending */
- if (!mptcp_timer_pending(sk))
- mptcp_reset_timer(sk);
+ msk->tx_pending_data += copied;
+ mptcp_push_pending(sk, msg->msg_flags);
}
- release_sock(ssk);
out:
- ssk_check_wmem(msk);
release_sock(sk);
return copied ? : ret;
}
struct msghdr *msg,
size_t len)
{
- struct sock *sk = (struct sock *)msk;
struct sk_buff *skb;
int copied = 0;
- while ((skb = skb_peek(&sk->sk_receive_queue)) != NULL) {
+ while ((skb = skb_peek(&msk->receive_queue)) != NULL) {
u32 offset = MPTCP_SKB_CB(skb)->offset;
u32 data_len = skb->len - offset;
u32 count = min_t(size_t, len - copied, data_len);
break;
}
- __skb_unlink(skb, &sk->sk_receive_queue);
+ /* we will bulk release the skb memory later */
+ skb->destructor = NULL;
+ msk->rmem_released += skb->truesize;
+ __skb_unlink(skb, &msk->receive_queue);
__kfree_skb(skb);
if (copied >= len)
msk->rcvq_space.time = mstamp;
}
-static bool __mptcp_move_skbs(struct mptcp_sock *msk)
+static void __mptcp_update_rmem(struct sock *sk)
{
- unsigned int moved = 0;
- bool done;
+ struct mptcp_sock *msk = mptcp_sk(sk);
- /* avoid looping forever below on racing close */
- if (((struct sock *)msk)->sk_state == TCP_CLOSE)
- return false;
+ if (!msk->rmem_released)
+ return;
+
+ atomic_sub(msk->rmem_released, &sk->sk_rmem_alloc);
+ sk_mem_uncharge(sk, msk->rmem_released);
+ msk->rmem_released = 0;
+}
+
+static void __mptcp_splice_receive_queue(struct sock *sk)
+{
+ struct mptcp_sock *msk = mptcp_sk(sk);
+
+ skb_queue_splice_tail_init(&sk->sk_receive_queue, &msk->receive_queue);
+}
+
+static bool __mptcp_move_skbs(struct mptcp_sock *msk, unsigned int rcv)
+{
+ struct sock *sk = (struct sock *)msk;
+ unsigned int moved = 0;
+ bool ret, done;
__mptcp_flush_join_list(msk);
do {
struct sock *ssk = mptcp_subflow_recv_lookup(msk);
bool slowpath;
- if (!ssk)
+ /* we can have data pending in the subflows only if the msk
+ * receive buffer was full at subflow_data_ready() time,
+ * that is an unlikely slow path.
+ */
+ if (likely(!ssk))
break;
slowpath = lock_sock_fast(ssk);
+ mptcp_data_lock(sk);
done = __mptcp_move_skbs_from_subflow(msk, ssk, &moved);
+ mptcp_data_unlock(sk);
+ if (moved && rcv) {
+ WRITE_ONCE(msk->rmem_pending, min(rcv, moved));
+ tcp_cleanup_rbuf(ssk, 1);
+ WRITE_ONCE(msk->rmem_pending, 0);
+ }
unlock_sock_fast(ssk, slowpath);
} while (!done);
- if (mptcp_ofo_queue(msk) || moved > 0) {
- mptcp_check_data_fin((struct sock *)msk);
- return true;
+ /* acquire the data lock only if some input data is pending */
+ ret = moved > 0;
+ if (!RB_EMPTY_ROOT(&msk->out_of_order_queue) ||
+ !skb_queue_empty_lockless(&sk->sk_receive_queue)) {
+ mptcp_data_lock(sk);
+ __mptcp_update_rmem(sk);
+ ret |= __mptcp_ofo_queue(msk);
+ __mptcp_splice_receive_queue(sk);
+ mptcp_data_unlock(sk);
}
- return false;
+ if (ret)
+ mptcp_check_data_fin((struct sock *)msk);
+ return !skb_queue_empty(&msk->receive_queue);
}
static int mptcp_recvmsg(struct sock *sk, struct msghdr *msg, size_t len,
if (msg->msg_flags & ~(MSG_WAITALL | MSG_DONTWAIT))
return -EOPNOTSUPP;
- lock_sock(sk);
+ mptcp_lock_sock(sk, __mptcp_splice_receive_queue(sk));
+ if (unlikely(sk->sk_state == TCP_LISTEN)) {
+ copied = -ENOTCONN;
+ goto out_err;
+ }
+
timeo = sock_rcvtimeo(sk, nonblock);
len = min_t(size_t, len, INT_MAX);
target = sock_rcvlowat(sk, flags & MSG_WAITALL, len);
- __mptcp_flush_join_list(msk);
- while (len > (size_t)copied) {
- int bytes_read;
+ while (copied < len) {
+ int bytes_read, old_space;
bytes_read = __mptcp_recvmsg_mskq(msk, msg, len - copied);
if (unlikely(bytes_read < 0)) {
copied += bytes_read;
- if (skb_queue_empty(&sk->sk_receive_queue) &&
- __mptcp_move_skbs(msk))
+ if (skb_queue_empty(&msk->receive_queue) &&
+ __mptcp_move_skbs(msk, len - copied))
continue;
+ /* be sure to advertise window change */
+ old_space = READ_ONCE(msk->old_wspace);
+ if ((tcp_space(sk) - old_space) >= old_space)
+ mptcp_cleanup_rbuf(msk);
+
/* only the master socket status is relevant here. The exit
* conditions mirror closely tcp_recvmsg()
*/
if (test_and_clear_bit(MPTCP_WORK_EOF, &msk->flags))
mptcp_check_for_eof(msk);
- if (sk->sk_shutdown & RCV_SHUTDOWN)
+ if (sk->sk_shutdown & RCV_SHUTDOWN) {
+ /* race breaker: the shutdown could be after the
+ * previous receive queue check
+ */
+ if (__mptcp_move_skbs(msk, len - copied))
+ continue;
break;
+ }
if (sk->sk_state == TCP_CLOSE) {
copied = -ENOTCONN;
mptcp_wait_data(sk, &timeo);
}
- if (skb_queue_empty(&sk->sk_receive_queue)) {
+ if (skb_queue_empty_lockless(&sk->sk_receive_queue) &&
+ skb_queue_empty(&msk->receive_queue)) {
/* entire backlog drained, clear DATA_READY. */
clear_bit(MPTCP_DATA_READY, &msk->flags);
/* .. race-breaker: ssk might have gotten new data
* after last __mptcp_move_skbs() returned false.
*/
- if (unlikely(__mptcp_move_skbs(msk)))
+ if (unlikely(__mptcp_move_skbs(msk, 0)))
set_bit(MPTCP_DATA_READY, &msk->flags);
} else if (unlikely(!test_bit(MPTCP_DATA_READY, &msk->flags))) {
/* data to read but mptcp_wait_data() cleared DATA_READY */
out_err:
pr_debug("msk=%p data_ready=%d rx queue empty=%d copied=%d",
msk, test_bit(MPTCP_DATA_READY, &msk->flags),
- skb_queue_empty(&sk->sk_receive_queue), copied);
+ skb_queue_empty_lockless(&sk->sk_receive_queue), copied);
mptcp_rcv_space_adjust(msk, copied);
release_sock(sk);
{
struct mptcp_sock *msk = mptcp_sk(sk);
- if (atomic64_read(&msk->snd_una) == READ_ONCE(msk->write_seq)) {
- mptcp_stop_timer(sk);
- } else {
- set_bit(MPTCP_WORK_RTX, &msk->flags);
- if (schedule_work(&msk->work))
- sock_hold(sk);
- }
+ set_bit(MPTCP_WORK_RTX, &msk->flags);
+ mptcp_schedule_work(sk);
}
static void mptcp_retransmit_timer(struct timer_list *t)
sock_put(sk);
}
+static void mptcp_timeout_timer(struct timer_list *t)
+{
+ struct sock *sk = from_timer(sk, t, sk_timer);
+
+ mptcp_schedule_work(sk);
+ sock_put(sk);
+}
+
/* Find an idle subflow. Return NULL if there is unacked data at tcp
* level.
*
sock_owned_by_me((const struct sock *)msk);
if (__mptcp_check_fallback(msk))
- return msk->first;
+ return NULL;
mptcp_for_each_subflow(msk, subflow) {
struct sock *ssk = mptcp_subflow_tcp_sock(subflow);
continue;
/* still data outstanding at TCP level? Don't retransmit. */
- if (!tcp_write_queue_empty(ssk))
+ if (!tcp_write_queue_empty(ssk)) {
+ if (inet_csk(ssk)->icsk_ca_state >= TCP_CA_Loss)
+ continue;
return NULL;
+ }
if (subflow->backup) {
if (!backup)
* parent socket.
*/
void __mptcp_close_ssk(struct sock *sk, struct sock *ssk,
- struct mptcp_subflow_context *subflow,
- long timeout)
+ struct mptcp_subflow_context *subflow)
{
- struct socket *sock = READ_ONCE(ssk->sk_socket);
+ bool dispose_socket = false;
+ struct socket *sock;
list_del(&subflow->node);
- if (sock && sock != sk->sk_socket) {
- /* outgoing subflow */
- sock_release(sock);
+ lock_sock(ssk);
+
+ /* if we are invoked by the msk cleanup code, the subflow is
+ * already orphaned
+ */
+ sock = ssk->sk_socket;
+ if (sock) {
+ dispose_socket = sock != sk->sk_socket;
+ sock_orphan(ssk);
+ }
+
+ /* if ssk hit tcp_done(), tcp_cleanup_ulp() cleared the related ops
+ * the ssk has been already destroyed, we just need to release the
+ * reference owned by msk;
+ */
+ if (!inet_csk(ssk)->icsk_ulp_ops) {
+ kfree_rcu(subflow, rcu);
} else {
- /* incoming subflow */
- tcp_close(ssk, timeout);
+ /* otherwise ask tcp do dispose of ssk and subflow ctx */
+ subflow->disposable = 1;
+ __tcp_close(ssk, 0);
+
+ /* close acquired an extra ref */
+ __sock_put(ssk);
}
+ release_sock(ssk);
+ if (dispose_socket)
+ iput(SOCK_INODE(sock));
+
+ sock_put(ssk);
}
static unsigned int mptcp_sync_mss(struct sock *sk, u32 pmtu)
pm->status &= ~BIT(MPTCP_PM_ADD_ADDR_RECEIVED);
mptcp_pm_nl_add_addr_received(msk);
}
+ if (pm->status & BIT(MPTCP_PM_ADD_ADDR_SEND_ACK)) {
+ pm->status &= ~BIT(MPTCP_PM_ADD_ADDR_SEND_ACK);
+ mptcp_pm_nl_add_addr_send_ack(msk);
+ }
if (pm->status & BIT(MPTCP_PM_RM_ADDR_RECEIVED)) {
pm->status &= ~BIT(MPTCP_PM_RM_ADDR_RECEIVED);
mptcp_pm_nl_rm_addr_received(msk);
if (inet_sk_state_load(ssk) != TCP_CLOSE)
continue;
- __mptcp_close_ssk((struct sock *)msk, ssk, subflow, 0);
+ __mptcp_close_ssk((struct sock *)msk, ssk, subflow);
+ }
+}
+
+static bool mptcp_check_close_timeout(const struct sock *sk)
+{
+ s32 delta = tcp_jiffies32 - inet_csk(sk)->icsk_mtup.probe_timestamp;
+ struct mptcp_subflow_context *subflow;
+
+ if (delta >= TCP_TIMEWAIT_LEN)
+ return true;
+
+ /* if all subflows are in closed status don't bother with additional
+ * timeout
+ */
+ mptcp_for_each_subflow(mptcp_sk(sk), subflow) {
+ if (inet_sk_state_load(mptcp_subflow_tcp_sock(subflow)) !=
+ TCP_CLOSE)
+ return false;
}
+ return true;
}
static void mptcp_worker(struct work_struct *work)
{
struct mptcp_sock *msk = container_of(work, struct mptcp_sock, work);
struct sock *ssk, *sk = &msk->sk.icsk_inet.sk;
- int orig_len, orig_offset, mss_now = 0, size_goal = 0;
+ struct mptcp_sendmsg_info info = {};
struct mptcp_data_frag *dfrag;
- u64 orig_write_seq;
size_t copied = 0;
- struct msghdr msg = {
- .msg_flags = MSG_DONTWAIT,
- };
- long timeo = 0;
+ int state, ret;
lock_sock(sk);
- mptcp_clean_una_wakeup(sk);
+ state = sk->sk_state;
+ if (unlikely(state == TCP_CLOSE))
+ goto unlock;
+
mptcp_check_data_fin_ack(sk);
__mptcp_flush_join_list(msk);
if (test_and_clear_bit(MPTCP_WORK_CLOSE_SUBFLOW, &msk->flags))
__mptcp_close_subflow(msk);
- __mptcp_move_skbs(msk);
-
if (msk->pm.status)
pm_work(msk);
if (test_and_clear_bit(MPTCP_WORK_EOF, &msk->flags))
mptcp_check_for_eof(msk);
+ __mptcp_check_send_data_fin(sk);
mptcp_check_data_fin(sk);
+ /* if the msk data is completely acked, or the socket timedout,
+ * there is no point in keeping around an orphaned sk
+ */
+ if (sock_flag(sk, SOCK_DEAD) &&
+ (mptcp_check_close_timeout(sk) ||
+ (state != sk->sk_state &&
+ ((1 << inet_sk_state_load(sk)) & (TCPF_CLOSE | TCPF_FIN_WAIT2))))) {
+ inet_sk_state_store(sk, TCP_CLOSE);
+ __mptcp_destroy_sock(sk);
+ goto unlock;
+ }
+
if (!test_and_clear_bit(MPTCP_WORK_RTX, &msk->flags))
goto unlock;
if (!dfrag)
goto unlock;
- if (!mptcp_ext_cache_refill(msk))
- goto reset_unlock;
-
ssk = mptcp_subflow_get_retrans(msk);
if (!ssk)
goto reset_unlock;
lock_sock(ssk);
- orig_len = dfrag->data_len;
- orig_offset = dfrag->offset;
- orig_write_seq = dfrag->data_seq;
- while (dfrag->data_len > 0) {
- int ret = mptcp_sendmsg_frag(sk, ssk, &msg, dfrag, &timeo,
- &mss_now, &size_goal);
- if (ret < 0)
+ /* limit retransmission to the bytes already sent on some subflows */
+ info.sent = 0;
+ info.limit = dfrag->already_sent;
+ while (info.sent < dfrag->already_sent) {
+ if (!mptcp_alloc_tx_skb(sk, ssk))
+ break;
+
+ ret = mptcp_sendmsg_frag(sk, ssk, dfrag, &info);
+ if (ret <= 0)
break;
MPTCP_INC_STATS(sock_net(sk), MPTCP_MIB_RETRANSSEGS);
copied += ret;
- dfrag->data_len -= ret;
- dfrag->offset += ret;
-
- if (!mptcp_ext_cache_refill(msk))
- break;
+ info.sent += ret;
}
if (copied)
- tcp_push(ssk, msg.msg_flags, mss_now, tcp_sk(ssk)->nonagle,
- size_goal);
-
- dfrag->data_seq = orig_write_seq;
- dfrag->offset = orig_offset;
- dfrag->data_len = orig_len;
+ tcp_push(ssk, 0, info.mss_now, tcp_sk(ssk)->nonagle,
+ info.size_goal);
mptcp_set_timeout(sk, ssk);
release_sock(ssk);
INIT_LIST_HEAD(&msk->conn_list);
INIT_LIST_HEAD(&msk->join_list);
INIT_LIST_HEAD(&msk->rtx_queue);
- __set_bit(MPTCP_SEND_SPACE, &msk->flags);
INIT_WORK(&msk->work, mptcp_worker);
+ __skb_queue_head_init(&msk->receive_queue);
+ __skb_queue_head_init(&msk->skb_tx_cache);
msk->out_of_order_queue = RB_ROOT;
+ msk->first_pending = NULL;
+ msk->wmem_reserved = 0;
+ msk->rmem_released = 0;
+ msk->tx_pending_data = 0;
+ msk->size_goal_cache = TCP_BASE_MSS;
+ msk->ack_hint = NULL;
msk->first = NULL;
inet_csk(sk)->icsk_sync_mss = mptcp_sync_mss;
/* re-use the csk retrans timer for MPTCP-level retrans */
timer_setup(&msk->sk.icsk_retransmit_timer, mptcp_retransmit_timer, 0);
-
+ timer_setup(&sk->sk_timer, mptcp_timeout_timer, 0);
return 0;
}
{
struct mptcp_sock *msk = mptcp_sk(sk);
struct mptcp_data_frag *dtmp, *dfrag;
+ struct sk_buff *skb;
- sk_stop_timer(sk, &msk->sk.icsk_retransmit_timer);
-
+ WRITE_ONCE(msk->first_pending, NULL);
list_for_each_entry_safe(dfrag, dtmp, &msk->rtx_queue, list)
dfrag_clear(sk, dfrag);
+ while ((skb = __skb_dequeue(&msk->skb_tx_cache)) != NULL) {
+ sk->sk_forward_alloc += skb->truesize;
+ kfree_skb(skb);
+ }
}
static void mptcp_cancel_work(struct sock *sk)
struct mptcp_sock *msk = mptcp_sk(sk);
if (cancel_work_sync(&msk->work))
- sock_put(sk);
+ __sock_put(sk);
}
void mptcp_subflow_shutdown(struct sock *sk, struct sock *ssk, int how)
return next & TCP_ACTION_FIN;
}
-static void mptcp_close(struct sock *sk, long timeout)
+static void __mptcp_check_send_data_fin(struct sock *sk)
{
- struct mptcp_subflow_context *subflow, *tmp;
+ struct mptcp_subflow_context *subflow;
struct mptcp_sock *msk = mptcp_sk(sk);
- LIST_HEAD(conn_list);
- lock_sock(sk);
- sk->sk_shutdown = SHUTDOWN_MASK;
+ pr_debug("msk=%p snd_data_fin_enable=%d pending=%d snd_nxt=%llu write_seq=%llu",
+ msk, msk->snd_data_fin_enable, !!mptcp_send_head(sk),
+ msk->snd_nxt, msk->write_seq);
- if (sk->sk_state == TCP_LISTEN) {
- inet_sk_state_store(sk, TCP_CLOSE);
- goto cleanup;
- } else if (sk->sk_state == TCP_CLOSE) {
- goto cleanup;
- }
+ /* we still need to enqueue subflows or not really shutting down,
+ * skip this
+ */
+ if (!msk->snd_data_fin_enable || msk->snd_nxt + 1 != msk->write_seq ||
+ mptcp_send_head(sk))
+ return;
+
+ WRITE_ONCE(msk->snd_nxt, msk->write_seq);
+ /* fallback socket will not get data_fin/ack, can move to the next
+ * state now
+ */
if (__mptcp_check_fallback(msk)) {
- goto update_state;
- } else if (mptcp_close_state(sk)) {
- pr_debug("Sending DATA_FIN sk=%p", sk);
- WRITE_ONCE(msk->write_seq, msk->write_seq + 1);
- WRITE_ONCE(msk->snd_data_fin_enable, 1);
+ if ((1 << sk->sk_state) & (TCPF_CLOSING | TCPF_LAST_ACK)) {
+ inet_sk_state_store(sk, TCP_CLOSE);
+ mptcp_close_wake_up(sk);
+ } else if (sk->sk_state == TCP_FIN_WAIT1) {
+ inet_sk_state_store(sk, TCP_FIN_WAIT2);
+ }
+ }
- mptcp_for_each_subflow(msk, subflow) {
- struct sock *tcp_sk = mptcp_subflow_tcp_sock(subflow);
+ __mptcp_flush_join_list(msk);
+ mptcp_for_each_subflow(msk, subflow) {
+ struct sock *tcp_sk = mptcp_subflow_tcp_sock(subflow);
- mptcp_subflow_shutdown(sk, tcp_sk, SHUTDOWN_MASK);
- }
+ mptcp_subflow_shutdown(sk, tcp_sk, SEND_SHUTDOWN);
}
+}
- sk_stream_wait_close(sk, timeout);
+static void __mptcp_wr_shutdown(struct sock *sk)
+{
+ struct mptcp_sock *msk = mptcp_sk(sk);
-update_state:
- inet_sk_state_store(sk, TCP_CLOSE);
+ pr_debug("msk=%p snd_data_fin_enable=%d shutdown=%x state=%d pending=%d",
+ msk, msk->snd_data_fin_enable, sk->sk_shutdown, sk->sk_state,
+ !!mptcp_send_head(sk));
+
+ /* will be ignored by fallback sockets */
+ WRITE_ONCE(msk->write_seq, msk->write_seq + 1);
+ WRITE_ONCE(msk->snd_data_fin_enable, 1);
+
+ __mptcp_check_send_data_fin(sk);
+}
+
+static void __mptcp_destroy_sock(struct sock *sk)
+{
+ struct mptcp_subflow_context *subflow, *tmp;
+ struct mptcp_sock *msk = mptcp_sk(sk);
+ LIST_HEAD(conn_list);
+
+ pr_debug("msk=%p", msk);
-cleanup:
/* be sure to always acquire the join list lock, to sync vs
* mptcp_finish_join().
*/
spin_unlock_bh(&msk->join_list_lock);
list_splice_init(&msk->conn_list, &conn_list);
- __mptcp_clear_xmit(sk);
-
- release_sock(sk);
+ sk_stop_timer(sk, &msk->sk.icsk_retransmit_timer);
+ sk_stop_timer(sk, &sk->sk_timer);
+ msk->pm.status = 0;
list_for_each_entry_safe(subflow, tmp, &conn_list, node) {
struct sock *ssk = mptcp_subflow_tcp_sock(subflow);
- __mptcp_close_ssk(sk, ssk, subflow, timeout);
+ __mptcp_close_ssk(sk, ssk, subflow);
+ }
+
+ sk->sk_prot->destroy(sk);
+
+ WARN_ON_ONCE(msk->wmem_reserved);
+ WARN_ON_ONCE(msk->rmem_released);
+ sk_stream_kill_queues(sk);
+ xfrm_sk_free_policy(sk);
+ sk_refcnt_debug_release(sk);
+ sock_put(sk);
+}
+
+static void mptcp_close(struct sock *sk, long timeout)
+{
+ struct mptcp_subflow_context *subflow;
+ bool do_cancel_work = false;
+
+ lock_sock(sk);
+ sk->sk_shutdown = SHUTDOWN_MASK;
+
+ if ((1 << sk->sk_state) & (TCPF_LISTEN | TCPF_CLOSE)) {
+ inet_sk_state_store(sk, TCP_CLOSE);
+ goto cleanup;
}
- mptcp_cancel_work(sk);
+ if (mptcp_close_state(sk))
+ __mptcp_wr_shutdown(sk);
+
+ sk_stream_wait_close(sk, timeout);
+
+cleanup:
+ /* orphan all the subflows */
+ inet_csk(sk)->icsk_mtup.probe_timestamp = tcp_jiffies32;
+ list_for_each_entry(subflow, &mptcp_sk(sk)->conn_list, node) {
+ struct sock *ssk = mptcp_subflow_tcp_sock(subflow);
+ bool slow, dispose_socket;
+ struct socket *sock;
- __skb_queue_purge(&sk->sk_receive_queue);
+ slow = lock_sock_fast(ssk);
+ sock = ssk->sk_socket;
+ dispose_socket = sock && sock != sk->sk_socket;
+ sock_orphan(ssk);
+ unlock_sock_fast(ssk, slow);
+
+ /* for the outgoing subflows we additionally need to free
+ * the associated socket
+ */
+ if (dispose_socket)
+ iput(SOCK_INODE(sock));
+ }
+ sock_orphan(sk);
- sk_common_release(sk);
+ sock_hold(sk);
+ pr_debug("msk=%p state=%d", sk, sk->sk_state);
+ if (sk->sk_state == TCP_CLOSE) {
+ __mptcp_destroy_sock(sk);
+ do_cancel_work = true;
+ } else {
+ sk_reset_timer(sk, &sk->sk_timer, jiffies + TCP_TIMEWAIT_LEN);
+ }
+ release_sock(sk);
+ if (do_cancel_work)
+ mptcp_cancel_work(sk);
+ sock_put(sk);
}
static void mptcp_copy_inaddrs(struct sock *msk, const struct sock *ssk)
WRITE_ONCE(msk->fully_established, false);
msk->write_seq = subflow_req->idsn + 1;
- atomic64_set(&msk->snd_una, msk->write_seq);
+ msk->snd_nxt = msk->write_seq;
+ msk->snd_una = msk->write_seq;
+ msk->wnd_end = msk->snd_nxt + req->rsk_rcv_wnd;
+
if (mp_opt->mp_capable) {
msk->can_ack = true;
msk->remote_key = mp_opt->sndr_key;
mptcp_crypto_key_sha(msk->remote_key, NULL, &ack_seq);
ack_seq++;
WRITE_ONCE(msk->ack_seq, ack_seq);
+ WRITE_ONCE(msk->rcv_wnd_sent, ack_seq);
}
sock_reset_flag(nsk, SOCK_RCU_FREE);
TCP_INIT_CWND * tp->advmss);
if (msk->rcvq_space.space == 0)
msk->rcvq_space.space = TCP_INIT_CWND * TCP_MSS_DEFAULT;
+
+ WRITE_ONCE(msk->wnd_end, msk->snd_nxt + tcp_sk(ssk)->snd_wnd);
}
static struct sock *mptcp_accept(struct sock *sk, int flags, int *err,
if (sk_is_mptcp(newsk)) {
struct mptcp_subflow_context *subflow;
struct sock *new_mptcp_sock;
- struct sock *ssk = newsk;
subflow = mptcp_subflow_ctx(newsk);
new_mptcp_sock = subflow->conn;
/* acquire the 2nd reference for the owning socket */
sock_hold(new_mptcp_sock);
-
- local_bh_disable();
- bh_lock_sock(new_mptcp_sock);
- msk = mptcp_sk(new_mptcp_sock);
- msk->first = newsk;
-
newsk = new_mptcp_sock;
- mptcp_copy_inaddrs(newsk, ssk);
- list_add(&subflow->node, &msk->conn_list);
-
- mptcp_rcv_space_init(msk, ssk);
- bh_unlock_sock(new_mptcp_sock);
-
- __MPTCP_INC_STATS(sock_net(sk), MPTCP_MIB_MPCAPABLEPASSIVEACK);
- local_bh_enable();
+ MPTCP_INC_STATS(sock_net(sk), MPTCP_MIB_MPCAPABLEPASSIVEACK);
} else {
MPTCP_INC_STATS(sock_net(sk),
MPTCP_MIB_MPCAPABLEPASSIVEFALLBACK);
void mptcp_destroy_common(struct mptcp_sock *msk)
{
+ struct sock *sk = (struct sock *)msk;
+
+ __mptcp_clear_xmit(sk);
+
+ /* move to sk_receive_queue, sk_stream_kill_queues will purge it */
+ skb_queue_splice_tail_init(&msk->receive_queue, &sk->sk_receive_queue);
+
skb_rbtree_purge(&msk->out_of_order_queue);
mptcp_token_destroy(msk);
mptcp_pm_free_anno_list(msk);
{
struct mptcp_sock *msk = mptcp_sk(sk);
- if (msk->cached_ext)
- __skb_ext_put(msk->cached_ext);
-
mptcp_destroy_common(msk);
sk_sockets_allocated_dec(sk);
}
return -EOPNOTSUPP;
}
-#define MPTCP_DEFERRED_ALL (TCPF_DELACK_TIMER_DEFERRED | \
- TCPF_WRITE_TIMER_DEFERRED)
+void __mptcp_data_acked(struct sock *sk)
+{
+ if (!sock_owned_by_user(sk))
+ __mptcp_clean_una(sk);
+ else
+ set_bit(MPTCP_CLEAN_UNA, &mptcp_sk(sk)->flags);
-/* this is very alike tcp_release_cb() but we must handle differently a
- * different set of events
- */
+ if (mptcp_pending_data_fin_ack(sk))
+ mptcp_schedule_work(sk);
+}
+
+void __mptcp_wnd_updated(struct sock *sk, struct sock *ssk)
+{
+ if (!mptcp_send_head(sk))
+ return;
+
+ if (!sock_owned_by_user(sk))
+ __mptcp_subflow_push_pending(sk, ssk);
+ else
+ set_bit(MPTCP_PUSH_PENDING, &mptcp_sk(sk)->flags);
+}
+
+#define MPTCP_DEFERRED_ALL (TCPF_WRITE_TIMER_DEFERRED)
+
+/* processes deferred events and flush wmem */
static void mptcp_release_cb(struct sock *sk)
{
unsigned long flags, nflags;
+ /* push_pending may touch wmem_reserved, do it before the later
+ * cleanup
+ */
+ if (test_and_clear_bit(MPTCP_CLEAN_UNA, &mptcp_sk(sk)->flags))
+ __mptcp_clean_una(sk);
+ if (test_and_clear_bit(MPTCP_PUSH_PENDING, &mptcp_sk(sk)->flags)) {
+ /* mptcp_push_pending() acquires the subflow socket lock
+ *
+ * 1) can't be invoked in atomic scope
+ * 2) must avoid ABBA deadlock with msk socket spinlock: the RX
+ * datapath acquires the msk socket spinlock while helding
+ * the subflow socket lock
+ */
+
+ spin_unlock_bh(&sk->sk_lock.slock);
+ mptcp_push_pending(sk, 0);
+ spin_lock_bh(&sk->sk_lock.slock);
+ }
+
+ /* clear any wmem reservation and errors */
+ __mptcp_update_wmem(sk);
+ __mptcp_update_rmem(sk);
+
do {
flags = sk->sk_tsq_flags;
if (!(flags & MPTCP_DEFERRED_ALL))
sock_release_ownership(sk);
- if (flags & TCPF_DELACK_TIMER_DEFERRED) {
- struct mptcp_sock *msk = mptcp_sk(sk);
- struct sock *ssk;
-
- ssk = mptcp_subflow_recv_lookup(msk);
- if (!ssk || !schedule_work(&msk->work))
- __sock_put(sk);
- }
-
if (flags & TCPF_WRITE_TIMER_DEFERRED) {
mptcp_retransmit_handler(sk);
__sock_put(sk);
WRITE_ONCE(msk->remote_key, subflow->remote_key);
WRITE_ONCE(msk->local_key, subflow->local_key);
WRITE_ONCE(msk->write_seq, subflow->idsn + 1);
+ WRITE_ONCE(msk->snd_nxt, msk->write_seq);
WRITE_ONCE(msk->ack_seq, ack_seq);
+ WRITE_ONCE(msk->rcv_wnd_sent, ack_seq);
WRITE_ONCE(msk->can_ack, 1);
- atomic64_set(&msk->snd_una, msk->write_seq);
+ WRITE_ONCE(msk->snd_una, msk->write_seq);
mptcp_pm_new_connection(msk, 0);
write_unlock_bh(&sk->sk_callback_lock);
}
-bool mptcp_finish_join(struct sock *sk)
+bool mptcp_finish_join(struct sock *ssk)
{
- struct mptcp_subflow_context *subflow = mptcp_subflow_ctx(sk);
+ struct mptcp_subflow_context *subflow = mptcp_subflow_ctx(ssk);
struct mptcp_sock *msk = mptcp_sk(subflow->conn);
struct sock *parent = (void *)msk;
struct socket *parent_sock;
/* active connections are already on conn_list, and we can't acquire
* msk lock here.
* use the join list lock as synchronization point and double-check
- * msk status to avoid racing with mptcp_close()
+ * msk status to avoid racing with __mptcp_destroy_sock()
*/
spin_lock_bh(&msk->join_list_lock);
ret = inet_sk_state_load(parent) == TCP_ESTABLISHED;
- if (ret && !WARN_ON_ONCE(!list_empty(&subflow->node)))
+ if (ret && !WARN_ON_ONCE(!list_empty(&subflow->node))) {
list_add_tail(&subflow->node, &msk->join_list);
+ sock_hold(ssk);
+ }
spin_unlock_bh(&msk->join_list_lock);
if (!ret)
return false;
* at close time
*/
parent_sock = READ_ONCE(parent->sk_socket);
- if (parent_sock && !sk->sk_socket)
- mptcp_sock_graft(sk, parent_sock);
+ if (parent_sock && !ssk->sk_socket)
+ mptcp_sock_graft(ssk, parent_sock);
subflow->map_seq = READ_ONCE(msk->ack_seq);
return true;
}
-static bool mptcp_memory_free(const struct sock *sk, int wake)
-{
- struct mptcp_sock *msk = mptcp_sk(sk);
-
- return wake ? test_bit(MPTCP_SEND_SPACE, &msk->flags) : true;
-}
-
static struct proto mptcp_prot = {
.name = "MPTCP",
.owner = THIS_MODULE,
.sockets_allocated = &mptcp_sockets_allocated,
.memory_allocated = &tcp_memory_allocated,
.memory_pressure = &tcp_memory_pressure,
- .stream_memory_free = mptcp_memory_free,
.sysctl_wmem_offset = offsetof(struct net, ipv4.sysctl_tcp_wmem),
.sysctl_rmem_offset = offsetof(struct net, ipv4.sysctl_tcp_rmem),
.sysctl_mem = sysctl_tcp_mem,
if (err == 0 && !mptcp_is_tcpsk(newsock->sk)) {
struct mptcp_sock *msk = mptcp_sk(newsock->sk);
struct mptcp_subflow_context *subflow;
+ struct sock *newsk = newsock->sk;
+ bool slowpath;
+
+ slowpath = lock_sock_fast(newsk);
+ mptcp_copy_inaddrs(newsk, msk->first);
+ mptcp_rcv_space_init(msk, msk->first);
/* set ssk->sk_socket of accept()ed flows to mptcp socket.
* This is needed so NOSPACE flag can be set from tcp stack.
if (!ssk->sk_socket)
mptcp_sock_graft(ssk, newsock);
}
+ unlock_sock_fast(newsk, slowpath);
}
if (inet_csk_listen_poll(ssock->sk))
0;
}
+static __poll_t mptcp_check_writeable(struct mptcp_sock *msk)
+{
+ struct sock *sk = (struct sock *)msk;
+
+ if (unlikely(sk->sk_shutdown & SEND_SHUTDOWN))
+ return 0;
+
+ if (sk_stream_is_writeable(sk))
+ return EPOLLOUT | EPOLLWRNORM;
+
+ set_bit(MPTCP_NOSPACE, &msk->flags);
+ smp_mb__after_atomic(); /* msk->flags is changed by write_space cb */
+ if (sk_stream_is_writeable(sk))
+ return EPOLLOUT | EPOLLWRNORM;
+
+ return 0;
+}
+
static __poll_t mptcp_poll(struct file *file, struct socket *sock,
struct poll_table_struct *wait)
{
if (state != TCP_SYN_SENT && state != TCP_SYN_RECV) {
mask |= mptcp_check_readable(msk);
- if (test_bit(MPTCP_SEND_SPACE, &msk->flags))
- mask |= EPOLLOUT | EPOLLWRNORM;
+ mask |= mptcp_check_writeable(msk);
}
if (sk->sk_shutdown & RCV_SHUTDOWN)
mask |= EPOLLIN | EPOLLRDNORM | EPOLLRDHUP;
static int mptcp_shutdown(struct socket *sock, int how)
{
struct mptcp_sock *msk = mptcp_sk(sock->sk);
- struct mptcp_subflow_context *subflow;
+ struct sock *sk = sock->sk;
int ret = 0;
pr_debug("sk=%p, how=%d", msk, how);
- lock_sock(sock->sk);
+ lock_sock(sk);
how++;
if ((how & ~SHUTDOWN_MASK) || !how) {
}
if (sock->state == SS_CONNECTING) {
- if ((1 << sock->sk->sk_state) &
+ if ((1 << sk->sk_state) &
(TCPF_SYN_SENT | TCPF_SYN_RECV | TCPF_CLOSE))
sock->state = SS_DISCONNECTING;
else
sock->state = SS_CONNECTED;
}
- /* If we've already sent a FIN, or it's a closed state, skip this. */
- if (__mptcp_check_fallback(msk)) {
- if (how == SHUT_WR || how == SHUT_RDWR)
- inet_sk_state_store(sock->sk, TCP_FIN_WAIT1);
-
- mptcp_for_each_subflow(msk, subflow) {
- struct sock *tcp_sk = mptcp_subflow_tcp_sock(subflow);
-
- mptcp_subflow_shutdown(sock->sk, tcp_sk, how);
- }
- } else if ((how & SEND_SHUTDOWN) &&
- ((1 << sock->sk->sk_state) &
- (TCPF_ESTABLISHED | TCPF_SYN_SENT |
- TCPF_SYN_RECV | TCPF_CLOSE_WAIT)) &&
- mptcp_close_state(sock->sk)) {
- __mptcp_flush_join_list(msk);
-
- WRITE_ONCE(msk->write_seq, msk->write_seq + 1);
- WRITE_ONCE(msk->snd_data_fin_enable, 1);
-
- mptcp_for_each_subflow(msk, subflow) {
- struct sock *tcp_sk = mptcp_subflow_tcp_sock(subflow);
-
- mptcp_subflow_shutdown(sock->sk, tcp_sk, how);
- }
- }
+ sk->sk_shutdown |= how;
+ if ((how & SEND_SHUTDOWN) && mptcp_close_state(sk))
+ __mptcp_wr_shutdown(sk);
/* Wake up anyone sleeping in poll. */
- sock->sk->sk_state_change(sock->sk);
+ sk->sk_state_change(sk);
out_unlock:
- release_sock(sock->sk);
+ release_sock(sk);
return ret;
}
/* MPTCP socket flags */
#define MPTCP_DATA_READY 0
-#define MPTCP_SEND_SPACE 1
+#define MPTCP_NOSPACE 1
#define MPTCP_WORK_RTX 2
#define MPTCP_WORK_EOF 3
#define MPTCP_FALLBACK_DONE 4
#define MPTCP_WORK_CLOSE_SUBFLOW 5
+#define MPTCP_PUSH_PENDING 6
+#define MPTCP_CLEAN_UNA 7
+
+static inline bool before64(__u64 seq1, __u64 seq2)
+{
+ return (__s64)(seq1 - seq2) < 0;
+}
+
+#define after64(seq2, seq1) before64(seq1, seq2)
struct mptcp_options_received {
u64 sndr_key;
enum mptcp_pm_status {
MPTCP_PM_ADD_ADDR_RECEIVED,
+ MPTCP_PM_ADD_ADDR_SEND_ACK,
MPTCP_PM_RM_ADDR_RECEIVED,
MPTCP_PM_ESTABLISHED,
MPTCP_PM_SUBFLOW_ESTABLISHED,
};
+enum mptcp_add_addr_status {
+ MPTCP_ADD_ADDR_SIGNAL,
+ MPTCP_ADD_ADDR_ECHO,
+ MPTCP_ADD_ADDR_IPV6,
+};
+
struct mptcp_pm_data {
struct mptcp_addr_info local;
struct mptcp_addr_info remote;
spinlock_t lock; /*protects the whole PM data */
- bool add_addr_signal;
+ u8 add_addr_signal;
bool rm_addr_signal;
bool server_side;
bool work_pending;
bool accept_addr;
bool accept_subflow;
- bool add_addr_echo;
u8 add_addr_signaled;
u8 add_addr_accepted;
u8 local_addr_used;
struct mptcp_data_frag {
struct list_head list;
u64 data_seq;
- int data_len;
- int offset;
- int overhead;
+ u16 data_len;
+ u16 offset;
+ u16 overhead;
+ u16 already_sent;
struct page *page;
};
u64 local_key;
u64 remote_key;
u64 write_seq;
+ u64 snd_nxt;
u64 ack_seq;
+ u64 rcv_wnd_sent;
u64 rcv_data_fin_seq;
+ int wmem_reserved;
struct sock *last_snd;
int snd_burst;
- atomic64_t snd_una;
+ int old_wspace;
+ u64 snd_una;
+ u64 wnd_end;
unsigned long timer_ival;
u32 token;
+ int rmem_pending;
+ int rmem_released;
unsigned long flags;
bool can_ack;
bool fully_established;
bool snd_data_fin_enable;
bool use_64bit_ack; /* Set when we received a 64-bit DSN */
spinlock_t join_list_lock;
+ struct sock *ack_hint;
struct work_struct work;
struct sk_buff *ooo_last_skb;
struct rb_root out_of_order_queue;
+ struct sk_buff_head receive_queue;
+ struct sk_buff_head skb_tx_cache; /* this is wmem accounted */
+ int tx_pending_data;
+ int size_goal_cache;
struct list_head conn_list;
struct list_head rtx_queue;
+ struct mptcp_data_frag *first_pending;
struct list_head join_list;
- struct skb_ext *cached_ext; /* for the next sendmsg */
struct socket *subflow; /* outgoing connect/listener/!mp_capable */
struct sock *first;
struct mptcp_pm_data pm;
} rcvq_space;
};
+#define mptcp_lock_sock(___sk, cb) do { \
+ struct sock *__sk = (___sk); /* silence macro reuse warning */ \
+ might_sleep(); \
+ spin_lock_bh(&__sk->sk_lock.slock); \
+ if (__sk->sk_lock.owned) \
+ __lock_sock(__sk); \
+ cb; \
+ __sk->sk_lock.owned = 1; \
+ spin_unlock(&__sk->sk_lock.slock); \
+ mutex_acquire(&__sk->sk_lock.dep_map, 0, 0, _RET_IP_); \
+ local_bh_enable(); \
+} while (0)
+
+#define mptcp_data_lock(sk) spin_lock_bh(&(sk)->sk_lock.slock)
+#define mptcp_data_unlock(sk) spin_unlock_bh(&(sk)->sk_lock.slock)
+
#define mptcp_for_each_subflow(__msk, __subflow) \
list_for_each_entry(__subflow, &((__msk)->conn_list), node)
return (struct mptcp_sock *)sk;
}
+static inline int __mptcp_space(const struct sock *sk)
+{
+ return tcp_space(sk) + READ_ONCE(mptcp_sk(sk)->rmem_pending);
+}
+
+static inline struct mptcp_data_frag *mptcp_send_head(const struct sock *sk)
+{
+ const struct mptcp_sock *msk = mptcp_sk(sk);
+
+ return READ_ONCE(msk->first_pending);
+}
+
+static inline struct mptcp_data_frag *mptcp_send_next(struct sock *sk)
+{
+ struct mptcp_sock *msk = mptcp_sk(sk);
+ struct mptcp_data_frag *cur;
+
+ cur = msk->first_pending;
+ return list_is_last(&cur->list, &msk->rtx_queue) ? NULL :
+ list_next_entry(cur, list);
+}
+
+static inline struct mptcp_data_frag *mptcp_pending_tail(const struct sock *sk)
+{
+ struct mptcp_sock *msk = mptcp_sk(sk);
+
+ if (!msk->first_pending)
+ return NULL;
+
+ if (WARN_ON_ONCE(list_empty(&msk->rtx_queue)))
+ return NULL;
+
+ return list_last_entry(&msk->rtx_queue, struct mptcp_data_frag, list);
+}
+
static inline struct mptcp_data_frag *mptcp_rtx_tail(const struct sock *sk)
{
struct mptcp_sock *msk = mptcp_sk(sk);
- if (list_empty(&msk->rtx_queue))
+ if (!before64(msk->snd_nxt, READ_ONCE(msk->snd_una)))
return NULL;
return list_last_entry(&msk->rtx_queue, struct mptcp_data_frag, list);
mpc_map : 1,
backup : 1,
rx_eof : 1,
- can_ack : 1; /* only after processing the remote a key */
+ can_ack : 1, /* only after processing the remote a key */
+ disposable : 1; /* ctx can be free at ulp release time */
enum mptcp_data_avail data_avail;
u32 remote_nonce;
u64 thmac;
return subflow->map_seq + mptcp_subflow_get_map_offset(subflow);
}
+static inline void mptcp_add_pending_subflow(struct mptcp_sock *msk,
+ struct mptcp_subflow_context *subflow)
+{
+ sock_hold(mptcp_subflow_tcp_sock(subflow));
+ spin_lock_bh(&msk->join_list_lock);
+ list_add_tail(&subflow->node, &msk->join_list);
+ spin_unlock_bh(&msk->join_list_lock);
+}
+
int mptcp_is_enabled(struct net *net);
unsigned int mptcp_get_add_addr_timeout(struct net *net);
void mptcp_subflow_fully_established(struct mptcp_subflow_context *subflow,
void __init mptcp_subflow_init(void);
void mptcp_subflow_shutdown(struct sock *sk, struct sock *ssk, int how);
void __mptcp_close_ssk(struct sock *sk, struct sock *ssk,
- struct mptcp_subflow_context *subflow,
- long timeout);
+ struct mptcp_subflow_context *subflow);
void mptcp_subflow_reset(struct sock *ssk);
/* called with sk socket lock held */
void mptcp_rcv_space_init(struct mptcp_sock *msk, const struct sock *ssk);
void mptcp_data_ready(struct sock *sk, struct sock *ssk);
bool mptcp_finish_join(struct sock *sk);
-void mptcp_data_acked(struct sock *sk);
+bool mptcp_schedule_work(struct sock *sk);
+void __mptcp_wnd_updated(struct sock *sk, struct sock *ssk);
+void __mptcp_data_acked(struct sock *sk);
void mptcp_subflow_eof(struct sock *sk);
bool mptcp_update_rcv_data_fin(struct mptcp_sock *msk, u64 data_fin_seq, bool use_64bit);
+void __mptcp_flush_join_list(struct mptcp_sock *msk);
+static inline bool mptcp_data_fin_enabled(const struct mptcp_sock *msk)
+{
+ return READ_ONCE(msk->snd_data_fin_enable) &&
+ READ_ONCE(msk->write_seq) == READ_ONCE(msk->snd_nxt);
+}
+
void mptcp_destroy_common(struct mptcp_sock *msk);
void __init mptcp_token_init(void);
void mptcp_pm_subflow_closed(struct mptcp_sock *msk, u8 id);
void mptcp_pm_add_addr_received(struct mptcp_sock *msk,
const struct mptcp_addr_info *addr);
+void mptcp_pm_add_addr_send_ack(struct mptcp_sock *msk);
void mptcp_pm_rm_addr_received(struct mptcp_sock *msk, u8 rm_id);
void mptcp_pm_free_anno_list(struct mptcp_sock *msk);
struct mptcp_pm_add_entry *
static inline bool mptcp_pm_should_add_signal(struct mptcp_sock *msk)
{
- return READ_ONCE(msk->pm.add_addr_signal);
+ return READ_ONCE(msk->pm.add_addr_signal) & BIT(MPTCP_ADD_ADDR_SIGNAL);
+}
+
+static inline bool mptcp_pm_should_add_signal_echo(struct mptcp_sock *msk)
+{
+ return READ_ONCE(msk->pm.add_addr_signal) & BIT(MPTCP_ADD_ADDR_ECHO);
+}
+
+static inline bool mptcp_pm_should_add_signal_ipv6(struct mptcp_sock *msk)
+{
+ return READ_ONCE(msk->pm.add_addr_signal) & BIT(MPTCP_ADD_ADDR_IPV6);
}
static inline bool mptcp_pm_should_rm_signal(struct mptcp_sock *msk)
void mptcp_pm_nl_fully_established(struct mptcp_sock *msk);
void mptcp_pm_nl_subflow_established(struct mptcp_sock *msk);
void mptcp_pm_nl_add_addr_received(struct mptcp_sock *msk);
+void mptcp_pm_nl_add_addr_send_ack(struct mptcp_sock *msk);
void mptcp_pm_nl_rm_addr_received(struct mptcp_sock *msk);
void mptcp_pm_nl_rm_subflow_received(struct mptcp_sock *msk, u8 rm_id);
int mptcp_pm_nl_get_local_id(struct mptcp_sock *msk, struct sock_common *skc);
return (struct mptcp_ext *)skb_ext_find(skb, SKB_EXT_MPTCP);
}
-static inline bool before64(__u64 seq1, __u64 seq2)
-{
- return (__s64)(seq1 - seq2) < 0;
-}
-
-#define after64(seq2, seq1) before64(seq1, seq2)
-
void mptcp_diag_subflow_init(struct tcp_ulp_ops *ops);
static inline bool __mptcp_check_fallback(const struct mptcp_sock *msk)
return 0;
}
-static void subflow_init_req(struct request_sock *req,
- const struct sock *sk_listener,
- struct sk_buff *skb)
+/* Init mptcp request socket.
+ *
+ * Returns an error code if a JOIN has failed and a TCP reset
+ * should be sent.
+ */
+static int subflow_init_req(struct request_sock *req,
+ const struct sock *sk_listener,
+ struct sk_buff *skb)
{
struct mptcp_subflow_context *listener = mptcp_subflow_ctx(sk_listener);
struct mptcp_subflow_request_sock *subflow_req = mptcp_subflow_rsk(req);
ret = __subflow_init_req(req, sk_listener);
if (ret)
- return;
+ return 0;
mptcp_get_options(skb, &mp_opt);
SUBFLOW_REQ_INC_STATS(req, MPTCP_MIB_MPCAPABLEPASSIVE);
if (mp_opt.mp_join)
- return;
+ return 0;
} else if (mp_opt.mp_join) {
SUBFLOW_REQ_INC_STATS(req, MPTCP_MIB_JOINSYNRX);
}
} else {
subflow_req->mp_capable = 1;
}
- return;
+ return 0;
}
err = mptcp_token_new_request(req);
subflow_req->remote_nonce = mp_opt.nonce;
subflow_req->msk = subflow_token_join_request(req, skb);
- if (unlikely(req->syncookie) && subflow_req->msk) {
+ /* Can't fall back to TCP in this case. */
+ if (!subflow_req->msk)
+ return -EPERM;
+
+ if (unlikely(req->syncookie)) {
if (mptcp_can_accept_new_subflow(subflow_req->msk))
subflow_init_req_cookie_join_save(subflow_req, skb);
}
pr_debug("token=%u, remote_nonce=%u msk=%p", subflow_req->token,
subflow_req->remote_nonce, subflow_req->msk);
}
+
+ return 0;
}
int mptcp_subflow_init_cookie_req(struct request_sock *req,
}
EXPORT_SYMBOL_GPL(mptcp_subflow_init_cookie_req);
-static void subflow_v4_init_req(struct request_sock *req,
- const struct sock *sk_listener,
- struct sk_buff *skb)
+static struct dst_entry *subflow_v4_route_req(const struct sock *sk,
+ struct sk_buff *skb,
+ struct flowi *fl,
+ struct request_sock *req)
{
+ struct dst_entry *dst;
+ int err;
+
tcp_rsk(req)->is_mptcp = 1;
- tcp_request_sock_ipv4_ops.init_req(req, sk_listener, skb);
+ dst = tcp_request_sock_ipv4_ops.route_req(sk, skb, fl, req);
+ if (!dst)
+ return NULL;
+
+ err = subflow_init_req(req, sk, skb);
+ if (err == 0)
+ return dst;
- subflow_init_req(req, sk_listener, skb);
+ dst_release(dst);
+ if (!req->syncookie)
+ tcp_request_sock_ops.send_reset(sk, skb);
+ return NULL;
}
#if IS_ENABLED(CONFIG_MPTCP_IPV6)
-static void subflow_v6_init_req(struct request_sock *req,
- const struct sock *sk_listener,
- struct sk_buff *skb)
+static struct dst_entry *subflow_v6_route_req(const struct sock *sk,
+ struct sk_buff *skb,
+ struct flowi *fl,
+ struct request_sock *req)
{
+ struct dst_entry *dst;
+ int err;
+
tcp_rsk(req)->is_mptcp = 1;
- tcp_request_sock_ipv6_ops.init_req(req, sk_listener, skb);
+ dst = tcp_request_sock_ipv6_ops.route_req(sk, skb, fl, req);
+ if (!dst)
+ return NULL;
+
+ err = subflow_init_req(req, sk, skb);
+ if (err == 0)
+ return dst;
- subflow_init_req(req, sk_listener, skb);
+ dst_release(dst);
+ if (!req->syncookie)
+ tcp6_request_sock_ops.send_reset(sk, skb);
+ return NULL;
}
#endif
fallback = true;
} else if (subflow_req->mp_join) {
mptcp_get_options(skb, &mp_opt);
- if (!mp_opt.mp_join ||
- !mptcp_can_accept_new_subflow(subflow_req->msk) ||
- !subflow_hmac_valid(req, &mp_opt)) {
+ if (!mp_opt.mp_join || !subflow_hmac_valid(req, &mp_opt) ||
+ !mptcp_can_accept_new_subflow(subflow_req->msk)) {
SUBFLOW_REQ_INC_STATS(req, MPTCP_MIB_JOINACKMAC);
fallback = true;
}
*/
inet_sk_state_store((void *)new_msk, TCP_ESTABLISHED);
+ /* link the newly created socket to the msk */
+ mptcp_add_pending_subflow(mptcp_sk(new_msk), ctx);
+ WRITE_ONCE(mptcp_sk(new_msk)->first, child);
+
/* new mpc subflow takes ownership of the newly
* created mptcp socket
*/
sk_eat_skb(ssk, skb);
if (mptcp_subflow_get_map_offset(subflow) >= subflow->map_data_len)
subflow->map_valid = 0;
- if (incr)
- tcp_cleanup_rbuf(ssk, incr);
}
static bool subflow_check_data_avail(struct sock *ssk)
const struct mptcp_subflow_context *subflow = mptcp_subflow_ctx(ssk);
const struct sock *sk = subflow->conn;
- *space = tcp_space(sk);
+ *space = __mptcp_space(sk);
*full_space = tcp_full_space(sk);
}
mptcp_data_ready(parent, sk);
}
-static void subflow_write_space(struct sock *sk)
+static void subflow_write_space(struct sock *ssk)
{
- struct mptcp_subflow_context *subflow = mptcp_subflow_ctx(sk);
- struct sock *parent = subflow->conn;
-
- if (!sk_stream_is_writeable(sk))
- return;
-
- if (sk_stream_is_writeable(parent)) {
- set_bit(MPTCP_SEND_SPACE, &mptcp_sk(parent)->flags);
- smp_mb__after_atomic();
- /* set SEND_SPACE before sk_stream_write_space clears NOSPACE */
- sk_stream_write_space(parent);
- }
+ /* we take action in __mptcp_clean_una() */
}
static struct inet_connection_sock_af_ops *
if (err && err != -EINPROGRESS)
goto failed;
- spin_lock_bh(&msk->join_list_lock);
- list_add_tail(&subflow->node, &msk->join_list);
- spin_unlock_bh(&msk->join_list_lock);
-
+ mptcp_add_pending_subflow(msk, subflow);
return err;
failed:
+ subflow->disposable = 1;
sock_release(sf);
return err;
}
mptcp_data_ready(parent, sk);
if (__mptcp_check_fallback(mptcp_sk(parent)) &&
- !(parent->sk_shutdown & RCV_SHUTDOWN) &&
!subflow->rx_eof && subflow_is_done(sk)) {
subflow->rx_eof = 1;
mptcp_subflow_eof(parent);
return err;
}
-static void subflow_ulp_release(struct sock *sk)
+static void subflow_ulp_release(struct sock *ssk)
{
- struct mptcp_subflow_context *ctx = mptcp_subflow_ctx(sk);
+ struct mptcp_subflow_context *ctx = mptcp_subflow_ctx(ssk);
+ bool release = true;
+ struct sock *sk;
if (!ctx)
return;
- if (ctx->conn)
- sock_put(ctx->conn);
+ sk = ctx->conn;
+ if (sk) {
+ /* if the msk has been orphaned, keep the ctx
+ * alive, will be freed by mptcp_done()
+ */
+ release = ctx->disposable;
+ sock_put(sk);
+ }
- kfree_rcu(ctx, rcu);
+ if (release)
+ kfree_rcu(ctx, rcu);
}
static void subflow_ulp_clone(const struct request_sock *req,
panic("MPTCP: failed to init subflow request sock ops\n");
subflow_request_sock_ipv4_ops = tcp_request_sock_ipv4_ops;
- subflow_request_sock_ipv4_ops.init_req = subflow_v4_init_req;
+ subflow_request_sock_ipv4_ops.route_req = subflow_v4_route_req;
subflow_specific = ipv4_specific;
subflow_specific.conn_request = subflow_v4_conn_request;
#if IS_ENABLED(CONFIG_MPTCP_IPV6)
subflow_request_sock_ipv6_ops = tcp_request_sock_ipv6_ops;
- subflow_request_sock_ipv6_ops.init_req = subflow_v6_init_req;
+ subflow_request_sock_ipv6_ops.route_req = subflow_v6_route_req;
subflow_v6_specific = ipv6_specific;
subflow_v6_specific.conn_request = subflow_v6_conn_request;
ndp->ptype.dev = dev;
dev_add_pack(&ndp->ptype);
- /* Set up generic netlink interface */
- ncsi_init_netlink(dev);
-
pdev = to_platform_device(dev->dev.parent);
if (pdev) {
np = pdev->dev.of_node;
list_del_rcu(&ndp->node);
spin_unlock_irqrestore(&ncsi_dev_lock, flags);
- ncsi_unregister_netlink(nd->dev);
-
kfree(ndp);
}
EXPORT_SYMBOL_GPL(ncsi_unregister_dev);
.n_small_ops = ARRAY_SIZE(ncsi_ops),
};
-int ncsi_init_netlink(struct net_device *dev)
+static int __init ncsi_init_netlink(void)
{
- int rc;
-
- rc = genl_register_family(&ncsi_genl_family);
- if (rc)
- netdev_err(dev, "ncsi: failed to register netlink family\n");
-
- return rc;
-}
-
-int ncsi_unregister_netlink(struct net_device *dev)
-{
- int rc;
-
- rc = genl_unregister_family(&ncsi_genl_family);
- if (rc)
- netdev_err(dev, "ncsi: failed to unregister netlink family\n");
-
- return rc;
+ return genl_register_family(&ncsi_genl_family);
}
+subsys_initcall(ncsi_init_netlink);
struct nlmsghdr *nlhdr,
int err);
-int ncsi_init_netlink(struct net_device *dev);
-int ncsi_unregister_netlink(struct net_device *dev);
-
#endif /* __NCSI_NETLINK_H__ */
static const struct nla_policy ipaddr_policy[IPSET_ATTR_IPADDR_MAX + 1] = {
[IPSET_ATTR_IPADDR_IPV4] = { .type = NLA_U32 },
- [IPSET_ATTR_IPADDR_IPV6] = { .type = NLA_BINARY,
- .len = sizeof(struct in6_addr) },
+ [IPSET_ATTR_IPADDR_IPV6] = NLA_POLICY_EXACT_LEN(sizeof(struct in6_addr)),
};
int
if (e.cidr > HOST_MASK)
return -IPSET_ERR_INVALID_CIDR;
}
- nla_strlcpy(e.iface, tb[IPSET_ATTR_IFACE], IFNAMSIZ);
+ nla_strscpy(e.iface, tb[IPSET_ATTR_IFACE], IFNAMSIZ);
if (tb[IPSET_ATTR_CADT_FLAGS]) {
u32 cadt_flags = ip_set_get_h32(tb[IPSET_ATTR_CADT_FLAGS]);
ip6_netmask(&e.ip, e.cidr);
- nla_strlcpy(e.iface, tb[IPSET_ATTR_IFACE], IFNAMSIZ);
+ nla_strscpy(e.iface, tb[IPSET_ATTR_IFACE], IFNAMSIZ);
if (tb[IPSET_ATTR_CADT_FLAGS]) {
u32 cadt_flags = ip_set_get_h32(tb[IPSET_ATTR_CADT_FLAGS]);
spin_lock_init(&ipvs->tot_stats.lock);
- proc_create_net("ip_vs", 0, ipvs->net->proc_net, &ip_vs_info_seq_ops,
- sizeof(struct ip_vs_iter));
- proc_create_net_single("ip_vs_stats", 0, ipvs->net->proc_net,
- ip_vs_stats_show, NULL);
- proc_create_net_single("ip_vs_stats_percpu", 0, ipvs->net->proc_net,
- ip_vs_stats_percpu_show, NULL);
+#ifdef CONFIG_PROC_FS
+ if (!proc_create_net("ip_vs", 0, ipvs->net->proc_net,
+ &ip_vs_info_seq_ops, sizeof(struct ip_vs_iter)))
+ goto err_vs;
+ if (!proc_create_net_single("ip_vs_stats", 0, ipvs->net->proc_net,
+ ip_vs_stats_show, NULL))
+ goto err_stats;
+ if (!proc_create_net_single("ip_vs_stats_percpu", 0,
+ ipvs->net->proc_net,
+ ip_vs_stats_percpu_show, NULL))
+ goto err_percpu;
+#endif
if (ip_vs_control_net_init_sysctl(ipvs))
goto err;
return 0;
err:
+#ifdef CONFIG_PROC_FS
+ remove_proc_entry("ip_vs_stats_percpu", ipvs->net->proc_net);
+
+err_percpu:
+ remove_proc_entry("ip_vs_stats", ipvs->net->proc_net);
+
+err_stats:
+ remove_proc_entry("ip_vs", ipvs->net->proc_net);
+
+err_vs:
+#endif
free_percpu(ipvs->tot_stats.cpustats);
return -ENOMEM;
}
{
ip_vs_trash_cleanup(ipvs);
ip_vs_control_net_cleanup_sysctl(ipvs);
+#ifdef CONFIG_PROC_FS
remove_proc_entry("ip_vs_stats_percpu", ipvs->net->proc_net);
remove_proc_entry("ip_vs_stats", ipvs->net->proc_net);
remove_proc_entry("ip_vs", ipvs->net->proc_net);
+#endif
free_percpu(ipvs->tot_stats.cpustats);
}
return true;
}
-static bool nf_conntrack_tcp_established(const struct nf_conn *ct)
-{
- return ct->proto.tcp.state == TCP_CONNTRACK_ESTABLISHED &&
- test_bit(IPS_ASSURED_BIT, &ct->status);
-}
-
/* Returns verdict for packet, or -1 for invalid. */
int nf_conntrack_tcp_packet(struct nf_conn *ct,
struct sk_buff *skb,
static void lockdep_nfnl_nft_mutex_not_held(void)
{
#ifdef CONFIG_PROVE_LOCKING
- WARN_ON_ONCE(lockdep_nfnl_is_held(NFNL_SUBSYS_NFTABLES));
+ if (debug_locks)
+ WARN_ON_ONCE(lockdep_nfnl_is_held(NFNL_SUBSYS_NFTABLES));
#endif
}
if (nla == NULL)
return ERR_PTR(-EINVAL);
- nla_strlcpy(search, nla, sizeof(search));
+ nla_strscpy(search, nla, sizeof(search));
WARN_ON(!rcu_read_lock_held() &&
!lockdep_commit_lock_is_held(net));
goto err_hook_alloc;
}
- nla_strlcpy(ifname, attr, IFNAMSIZ);
+ nla_strscpy(ifname, attr, IFNAMSIZ);
dev = __dev_get_by_name(net, ifname);
if (!dev) {
err = -ENOENT;
struct rhlist_head *tmp, *list;
struct nft_object *obj;
- nla_strlcpy(search, nla, sizeof(search));
+ nla_strscpy(search, nla, sizeof(search));
k.name = search;
WARN_ON_ONCE(!rcu_read_lock_held() &&
return flow;
}
+void nft_flow_rule_set_addr_type(struct nft_flow_rule *flow,
+ enum flow_dissector_key_id addr_type)
+{
+ struct nft_flow_match *match = &flow->match;
+ struct nft_flow_key *mask = &match->mask;
+ struct nft_flow_key *key = &match->key;
+
+ if (match->dissector.used_keys & BIT(FLOW_DISSECTOR_KEY_CONTROL))
+ return;
+
+ key->control.addr_type = addr_type;
+ mask->control.addr_type = 0xffff;
+ match->dissector.used_keys |= BIT(FLOW_DISSECTOR_KEY_CONTROL);
+ match->dissector.offset[FLOW_DISSECTOR_KEY_CONTROL] =
+ offsetof(struct nft_flow_key, control);
+}
+
struct nft_flow_rule *nft_flow_rule_create(struct net *net,
const struct nft_rule *rule)
{
nfacct->flags = flags;
}
- nla_strlcpy(nfacct->name, tb[NFACCT_NAME], NFACCT_NAME_MAX);
+ nla_strscpy(nfacct->name, tb[NFACCT_NAME], NFACCT_NAME_MAX);
if (tb[NFACCT_BYTES]) {
atomic64_set(&nfacct->bytes,
!tb[NFCTH_POLICY_EXPECT_TIMEOUT])
return -EINVAL;
- nla_strlcpy(expect_policy->name,
+ nla_strscpy(expect_policy->name,
tb[NFCTH_POLICY_NAME], NF_CT_HELPER_NAME_LEN);
expect_policy->max_expected =
ntohl(nla_get_be32(tb[NFCTH_POLICY_EXPECT_MAX]));
if (ret < 0)
goto err1;
- nla_strlcpy(helper->name,
+ nla_strscpy(helper->name,
tb[NFCTH_NAME], NF_CT_HELPER_NAME_LEN);
size = ntohl(nla_get_be32(tb[NFCTH_PRIV_DATA_LEN]));
if (size > sizeof_field(struct nf_conn_help, data)) {
u8 *mask = (u8 *)&flow->match.mask;
u8 *key = (u8 *)&flow->match.key;
- if (priv->op != NFT_CMP_EQ || reg->len != priv->len)
+ if (priv->op != NFT_CMP_EQ || priv->len > reg->len)
return -EOPNOTSUPP;
- memcpy(key + reg->offset, &priv->data, priv->len);
- memcpy(mask + reg->offset, ®->mask, priv->len);
+ memcpy(key + reg->offset, &priv->data, reg->len);
+ memcpy(mask + reg->offset, ®->mask, reg->len);
flow->match.dissector.used_keys |= BIT(reg->key);
flow->match.dissector.offset[reg->key] = reg->base_offset;
nft_reg_load16(priv->data.data) != ARPHRD_ETHER)
return -EOPNOTSUPP;
- nft_offload_update_dependency(ctx, &priv->data, priv->len);
+ nft_offload_update_dependency(ctx, &priv->data, reg->len);
return 0;
}
if (!priv->l4proto)
return -ENOENT;
- nla_strlcpy(name, tb[NFTA_CT_HELPER_NAME], sizeof(name));
+ nla_strscpy(name, tb[NFTA_CT_HELPER_NAME], sizeof(name));
if (tb[NFTA_CT_HELPER_L3PROTO])
family = ntohs(nla_get_be16(tb[NFTA_CT_HELPER_L3PROTO]));
priv->prefix = kmalloc(nla_len(nla) + 1, GFP_KERNEL);
if (priv->prefix == NULL)
return -ENOMEM;
- nla_strlcpy(priv->prefix, nla, nla_len(nla) + 1);
+ nla_strscpy(priv->prefix, nla, nla_len(nla) + 1);
} else {
priv->prefix = (char *)nft_log_null_prefix;
}
switch (priv->key) {
case NFT_META_PROTOCOL:
- NFT_OFFLOAD_MATCH(FLOW_DISSECTOR_KEY_BASIC, basic, n_proto,
- sizeof(__u16), reg);
+ NFT_OFFLOAD_MATCH_EXACT(FLOW_DISSECTOR_KEY_BASIC, basic, n_proto,
+ sizeof(__u16), reg);
nft_offload_set_dependency(ctx, NFT_OFFLOAD_DEP_NETWORK);
break;
case NFT_META_L4PROTO:
- NFT_OFFLOAD_MATCH(FLOW_DISSECTOR_KEY_BASIC, basic, ip_proto,
- sizeof(__u8), reg);
+ NFT_OFFLOAD_MATCH_EXACT(FLOW_DISSECTOR_KEY_BASIC, basic, ip_proto,
+ sizeof(__u8), reg);
nft_offload_set_dependency(ctx, NFT_OFFLOAD_DEP_TRANSPORT);
break;
case NFT_META_IIF:
- NFT_OFFLOAD_MATCH(FLOW_DISSECTOR_KEY_META, meta,
- ingress_ifindex, sizeof(__u32), reg);
+ NFT_OFFLOAD_MATCH_EXACT(FLOW_DISSECTOR_KEY_META, meta,
+ ingress_ifindex, sizeof(__u32), reg);
break;
case NFT_META_IIFTYPE:
- NFT_OFFLOAD_MATCH(FLOW_DISSECTOR_KEY_META, meta,
- ingress_iftype, sizeof(__u16), reg);
+ NFT_OFFLOAD_MATCH_EXACT(FLOW_DISSECTOR_KEY_META, meta,
+ ingress_iftype, sizeof(__u16), reg);
break;
default:
return -EOPNOTSUPP;
return -1;
}
+static bool nft_payload_offload_mask(struct nft_offload_reg *reg,
+ u32 priv_len, u32 field_len)
+{
+ unsigned int remainder, delta, k;
+ struct nft_data mask = {};
+ __be32 remainder_mask;
+
+ if (priv_len == field_len) {
+ memset(®->mask, 0xff, priv_len);
+ return true;
+ } else if (priv_len > field_len) {
+ return false;
+ }
+
+ memset(&mask, 0xff, field_len);
+ remainder = priv_len % sizeof(u32);
+ if (remainder) {
+ k = priv_len / sizeof(u32);
+ delta = field_len - priv_len;
+ remainder_mask = htonl(~((1 << (delta * BITS_PER_BYTE)) - 1));
+ mask.data[k] = (__force u32)remainder_mask;
+ }
+
+ memcpy(®->mask, &mask, field_len);
+
+ return true;
+}
+
static int nft_payload_offload_ll(struct nft_offload_ctx *ctx,
struct nft_flow_rule *flow,
const struct nft_payload *priv)
switch (priv->offset) {
case offsetof(struct ethhdr, h_source):
- if (priv->len != ETH_ALEN)
+ if (!nft_payload_offload_mask(reg, priv->len, ETH_ALEN))
return -EOPNOTSUPP;
NFT_OFFLOAD_MATCH(FLOW_DISSECTOR_KEY_ETH_ADDRS, eth_addrs,
src, ETH_ALEN, reg);
break;
case offsetof(struct ethhdr, h_dest):
- if (priv->len != ETH_ALEN)
+ if (!nft_payload_offload_mask(reg, priv->len, ETH_ALEN))
return -EOPNOTSUPP;
NFT_OFFLOAD_MATCH(FLOW_DISSECTOR_KEY_ETH_ADDRS, eth_addrs,
dst, ETH_ALEN, reg);
break;
case offsetof(struct ethhdr, h_proto):
- if (priv->len != sizeof(__be16))
+ if (!nft_payload_offload_mask(reg, priv->len, sizeof(__be16)))
return -EOPNOTSUPP;
NFT_OFFLOAD_MATCH(FLOW_DISSECTOR_KEY_BASIC, basic,
nft_offload_set_dependency(ctx, NFT_OFFLOAD_DEP_NETWORK);
break;
case offsetof(struct vlan_ethhdr, h_vlan_TCI):
- if (priv->len != sizeof(__be16))
+ if (!nft_payload_offload_mask(reg, priv->len, sizeof(__be16)))
return -EOPNOTSUPP;
NFT_OFFLOAD_MATCH(FLOW_DISSECTOR_KEY_VLAN, vlan,
vlan_tci, sizeof(__be16), reg);
break;
case offsetof(struct vlan_ethhdr, h_vlan_encapsulated_proto):
- if (priv->len != sizeof(__be16))
+ if (!nft_payload_offload_mask(reg, priv->len, sizeof(__be16)))
return -EOPNOTSUPP;
NFT_OFFLOAD_MATCH(FLOW_DISSECTOR_KEY_VLAN, vlan,
nft_offload_set_dependency(ctx, NFT_OFFLOAD_DEP_NETWORK);
break;
case offsetof(struct vlan_ethhdr, h_vlan_TCI) + sizeof(struct vlan_hdr):
- if (priv->len != sizeof(__be16))
+ if (!nft_payload_offload_mask(reg, priv->len, sizeof(__be16)))
return -EOPNOTSUPP;
NFT_OFFLOAD_MATCH(FLOW_DISSECTOR_KEY_CVLAN, vlan,
break;
case offsetof(struct vlan_ethhdr, h_vlan_encapsulated_proto) +
sizeof(struct vlan_hdr):
- if (priv->len != sizeof(__be16))
+ if (!nft_payload_offload_mask(reg, priv->len, sizeof(__be16)))
return -EOPNOTSUPP;
NFT_OFFLOAD_MATCH(FLOW_DISSECTOR_KEY_CVLAN, vlan,
switch (priv->offset) {
case offsetof(struct iphdr, saddr):
- if (priv->len != sizeof(struct in_addr))
+ if (!nft_payload_offload_mask(reg, priv->len,
+ sizeof(struct in_addr)))
return -EOPNOTSUPP;
NFT_OFFLOAD_MATCH(FLOW_DISSECTOR_KEY_IPV4_ADDRS, ipv4, src,
sizeof(struct in_addr), reg);
+ nft_flow_rule_set_addr_type(flow, FLOW_DISSECTOR_KEY_IPV4_ADDRS);
break;
case offsetof(struct iphdr, daddr):
- if (priv->len != sizeof(struct in_addr))
+ if (!nft_payload_offload_mask(reg, priv->len,
+ sizeof(struct in_addr)))
return -EOPNOTSUPP;
NFT_OFFLOAD_MATCH(FLOW_DISSECTOR_KEY_IPV4_ADDRS, ipv4, dst,
sizeof(struct in_addr), reg);
+ nft_flow_rule_set_addr_type(flow, FLOW_DISSECTOR_KEY_IPV4_ADDRS);
break;
case offsetof(struct iphdr, protocol):
- if (priv->len != sizeof(__u8))
+ if (!nft_payload_offload_mask(reg, priv->len, sizeof(__u8)))
return -EOPNOTSUPP;
NFT_OFFLOAD_MATCH(FLOW_DISSECTOR_KEY_BASIC, basic, ip_proto,
switch (priv->offset) {
case offsetof(struct ipv6hdr, saddr):
- if (priv->len != sizeof(struct in6_addr))
+ if (!nft_payload_offload_mask(reg, priv->len,
+ sizeof(struct in6_addr)))
return -EOPNOTSUPP;
NFT_OFFLOAD_MATCH(FLOW_DISSECTOR_KEY_IPV6_ADDRS, ipv6, src,
sizeof(struct in6_addr), reg);
+ nft_flow_rule_set_addr_type(flow, FLOW_DISSECTOR_KEY_IPV6_ADDRS);
break;
case offsetof(struct ipv6hdr, daddr):
- if (priv->len != sizeof(struct in6_addr))
+ if (!nft_payload_offload_mask(reg, priv->len,
+ sizeof(struct in6_addr)))
return -EOPNOTSUPP;
NFT_OFFLOAD_MATCH(FLOW_DISSECTOR_KEY_IPV6_ADDRS, ipv6, dst,
sizeof(struct in6_addr), reg);
+ nft_flow_rule_set_addr_type(flow, FLOW_DISSECTOR_KEY_IPV6_ADDRS);
break;
case offsetof(struct ipv6hdr, nexthdr):
- if (priv->len != sizeof(__u8))
+ if (!nft_payload_offload_mask(reg, priv->len, sizeof(__u8)))
return -EOPNOTSUPP;
NFT_OFFLOAD_MATCH(FLOW_DISSECTOR_KEY_BASIC, basic, ip_proto,
switch (priv->offset) {
case offsetof(struct tcphdr, source):
- if (priv->len != sizeof(__be16))
+ if (!nft_payload_offload_mask(reg, priv->len, sizeof(__be16)))
return -EOPNOTSUPP;
NFT_OFFLOAD_MATCH(FLOW_DISSECTOR_KEY_PORTS, tp, src,
sizeof(__be16), reg);
break;
case offsetof(struct tcphdr, dest):
- if (priv->len != sizeof(__be16))
+ if (!nft_payload_offload_mask(reg, priv->len, sizeof(__be16)))
return -EOPNOTSUPP;
NFT_OFFLOAD_MATCH(FLOW_DISSECTOR_KEY_PORTS, tp, dst,
switch (priv->offset) {
case offsetof(struct udphdr, source):
- if (priv->len != sizeof(__be16))
+ if (!nft_payload_offload_mask(reg, priv->len, sizeof(__be16)))
return -EOPNOTSUPP;
NFT_OFFLOAD_MATCH(FLOW_DISSECTOR_KEY_PORTS, tp, src,
sizeof(__be16), reg);
break;
case offsetof(struct udphdr, dest):
- if (priv->len != sizeof(__be16))
+ if (!nft_payload_offload_mask(reg, priv->len, sizeof(__be16)))
return -EOPNOTSUPP;
NFT_OFFLOAD_MATCH(FLOW_DISSECTOR_KEY_PORTS, tp, dst,
ret_val = -ENOMEM;
goto add_free_entry;
}
- nla_strlcpy(entry->domain,
+ nla_strscpy(entry->domain,
info->attrs[NLBL_MGMT_A_DOMAIN], tmp_size);
}
u32 skip_bkt = cb->args[0];
u32 skip_chain = cb->args[1];
u32 skip_addr4 = cb->args[2];
- u32 iter_bkt, iter_chain, iter_addr4 = 0, iter_addr6 = 0;
+ u32 iter_bkt, iter_chain = 0, iter_addr4 = 0, iter_addr6 = 0;
struct netlbl_unlhsh_iface *iface;
struct list_head *iter_list;
struct netlbl_af4list *addr4;
}
static void nci_hci_resp_received(struct nci_dev *ndev, u8 pipe,
- u8 result, struct sk_buff *skb)
+ struct sk_buff *skb)
{
struct nci_conn_info *conn_info;
- u8 status = result;
conn_info = ndev->hci_dev->conn_info;
- if (!conn_info) {
- status = NCI_STATUS_REJECTED;
+ if (!conn_info)
goto exit;
- }
conn_info->rx_skb = skb;
{
switch (type) {
case NCI_HCI_HCP_RESPONSE:
- nci_hci_resp_received(ndev, pipe, instruction, skb);
+ nci_hci_resp_received(ndev, pipe, skb);
break;
case NCI_HCI_HCP_COMMAND:
nci_hci_cmd_received(ndev, pipe, instruction, skb);
if (!dev)
return -ENODEV;
- nla_strlcpy(firmware_name, info->attrs[NFC_ATTR_FIRMWARE_NAME],
+ nla_strscpy(firmware_name, info->attrs[NFC_ATTR_FIRMWARE_NAME],
sizeof(firmware_name));
rc = nfc_fw_download(dev, firmware_name);
__be32 lse;
int err;
+ if (!pskb_may_pull(skb, skb_network_offset(skb) + MPLS_HLEN))
+ return -ENOMEM;
+
stack = mpls_hdr(skb);
lse = OVS_MASKED(stack->label_stack_entry, *mpls_lse, *mask);
err = skb_mpls_update_lse(skb, lse);
{
/* The first action is always 'OVS_DEC_TTL_ATTR_ARG'. */
struct nlattr *dec_ttl_arg = nla_data(attr);
- int rem = nla_len(attr);
if (nla_len(dec_ttl_arg)) {
- struct nlattr *actions = nla_next(dec_ttl_arg, &rem);
+ struct nlattr *actions = nla_data(dec_ttl_arg);
if (actions)
- return clone_execute(dp, skb, key, 0, actions, rem,
- last, false);
+ return clone_execute(dp, skb, key, 0, nla_data(actions),
+ nla_len(actions), last, false);
}
consume_skb(skb);
return 0;
ovs_ct_helper(skb, info->family) != NF_ACCEPT) {
return -EINVAL;
}
+
+ if (nf_ct_protonum(ct) == IPPROTO_TCP &&
+ nf_ct_is_confirmed(ct) && nf_conntrack_tcp_established(ct)) {
+ /* Be liberal for tcp packets so that out-of-window
+ * packets are not marked invalid.
+ */
+ nf_ct_set_tcp_be_liberal(ct);
+ }
}
return 0;
__be16 eth_type, __be16 vlan_tci,
u32 mpls_label_count, bool log)
{
- int start, err;
- u32 nested = true;
+ const struct nlattr *attrs[OVS_DEC_TTL_ATTR_MAX + 1];
+ int start, action_start, err, rem;
+ const struct nlattr *a, *actions;
+
+ memset(attrs, 0, sizeof(attrs));
+ nla_for_each_nested(a, attr, rem) {
+ int type = nla_type(a);
- if (!nla_len(attr))
- return ovs_nla_add_action(sfa, OVS_ACTION_ATTR_DEC_TTL,
- NULL, 0, log);
+ /* Ignore unknown attributes to be future proof. */
+ if (type > OVS_DEC_TTL_ATTR_MAX)
+ continue;
+
+ if (!type || attrs[type])
+ return -EINVAL;
+
+ attrs[type] = a;
+ }
+
+ actions = attrs[OVS_DEC_TTL_ATTR_ACTION];
+ if (rem || !actions || (nla_len(actions) && nla_len(actions) < NLA_HDRLEN))
+ return -EINVAL;
start = add_nested_action_start(sfa, OVS_ACTION_ATTR_DEC_TTL, log);
if (start < 0)
return start;
- err = ovs_nla_add_action(sfa, OVS_DEC_TTL_ATTR_ACTION, &nested,
- sizeof(nested), log);
-
- if (err)
- return err;
+ action_start = add_nested_action_start(sfa, OVS_DEC_TTL_ATTR_ACTION, log);
+ if (action_start < 0)
+ return start;
- err = __ovs_nla_copy_actions(net, attr, key, sfa, eth_type,
+ err = __ovs_nla_copy_actions(net, actions, key, sfa, eth_type,
vlan_tci, mpls_label_count, log);
if (err)
return err;
+ add_nested_action_end(*sfa, action_start);
add_nested_action_end(*sfa, start);
return 0;
}
static int dec_ttl_action_to_attr(const struct nlattr *attr,
struct sk_buff *skb)
{
- int err = 0, rem = nla_len(attr);
- struct nlattr *start;
+ struct nlattr *start, *action_start;
+ const struct nlattr *a;
+ int err = 0, rem;
start = nla_nest_start_noflag(skb, OVS_ACTION_ATTR_DEC_TTL);
-
if (!start)
return -EMSGSIZE;
- err = ovs_nla_put_actions(nla_data(attr), rem, skb);
- if (err)
- nla_nest_cancel(skb, start);
- else
- nla_nest_end(skb, start);
+ nla_for_each_attr(a, nla_data(attr), nla_len(attr), rem) {
+ switch (nla_type(a)) {
+ case OVS_DEC_TTL_ATTR_ACTION:
+
+ action_start = nla_nest_start_noflag(skb, OVS_DEC_TTL_ATTR_ACTION);
+ if (!action_start) {
+ err = -EMSGSIZE;
+ goto out;
+ }
+
+ err = ovs_nla_put_actions(nla_data(a), nla_len(a), skb);
+ if (err)
+ goto out;
+
+ nla_nest_end(skb, action_start);
+ break;
+ default:
+ /* Ignore all other option to be future compatible */
+ break;
+ }
+ }
+
+ nla_nest_end(skb, start);
+ return 0;
+
+out:
+ nla_nest_cancel(skb, start);
return err;
}
{
int len, err;
+ /* store len value because skb can be freed inside ovs_vport_receive() */
len = skb->len;
+
rcu_read_lock();
err = ovs_vport_receive(internal_dev_priv(netdev)->vport, skb, NULL);
rcu_read_unlock();
- if (likely(!err)) {
- struct pcpu_sw_netstats *tstats = this_cpu_ptr(netdev->tstats);
-
- u64_stats_update_begin(&tstats->syncp);
- tstats->tx_bytes += len;
- tstats->tx_packets++;
- u64_stats_update_end(&tstats->syncp);
- } else {
+ if (likely(!err))
+ dev_sw_netstats_tx_add(netdev, 1, len);
+ else
netdev->stats.tx_errors++;
- }
+
return NETDEV_TX_OK;
}
ovs_vport_free(vport);
}
-static void
-internal_get_stats(struct net_device *dev, struct rtnl_link_stats64 *stats)
-{
- memset(stats, 0, sizeof(*stats));
- stats->rx_errors = dev->stats.rx_errors;
- stats->tx_errors = dev->stats.tx_errors;
- stats->tx_dropped = dev->stats.tx_dropped;
- stats->rx_dropped = dev->stats.rx_dropped;
-
- dev_fetch_sw_netstats(stats, dev->tstats);
-}
-
static const struct net_device_ops internal_dev_netdev_ops = {
.ndo_open = internal_dev_open,
.ndo_stop = internal_dev_stop,
.ndo_start_xmit = internal_dev_xmit,
.ndo_set_mac_address = eth_mac_addr,
- .ndo_get_stats64 = internal_get_stats,
+ .ndo_get_stats64 = dev_get_tstats64,
};
static struct rtnl_link_ops internal_dev_link_ops __read_mostly = {
* Copyright (C) 2011, <lokec@ccs.neu.edu>
*/
+#include <linux/ethtool.h>
#include <linux/types.h>
#include <linux/mm.h>
#include <linux/capability.h>
/*
Assumptions:
- - If the device has no dev->header_ops, there is no LL header visible
- above the device. In this case, its hard_header_len should be 0.
+ - If the device has no dev->header_ops->create, there is no LL header
+ visible above the device. In this case, its hard_header_len should be 0.
The device may prepend its own header internally. In this case, its
needed_headroom should be set to the space needed for it to add its
internal header.
On receive:
-----------
-Incoming, dev->header_ops != NULL
+Incoming, dev_has_header(dev) == true
mac_header -> ll header
data -> data
-Outgoing, dev->header_ops != NULL
+Outgoing, dev_has_header(dev) == true
mac_header -> ll header
data -> ll header
-Incoming, dev->header_ops == NULL
+Incoming, dev_has_header(dev) == false
mac_header -> data
However drivers often make it point to the ll header.
This is incorrect because the ll header should be invisible to us.
data -> data
-Outgoing, dev->header_ops == NULL
+Outgoing, dev_has_header(dev) == false
mac_header -> data. ll header is invisible to us.
data -> data
Resume
- If dev->header_ops == NULL we are unable to restore the ll header,
+ If dev_has_header(dev) == false we are unable to restore the ll header,
because it is invisible to us.
skb->dev = dev;
- if (dev->header_ops) {
+ if (dev_has_header(dev)) {
/* The device has an explicit notion of ll header,
* exported to higher levels.
*
if (!net_eq(dev_net(dev), sock_net(sk)))
goto drop;
- if (dev->header_ops) {
+ if (dev_has_header(dev)) {
if (sk->sk_type != SOCK_DGRAM)
skb_push(skb, skb->data - skb_mac_header(skb));
else if (skb->pkt_type == PACKET_OUTGOING) {
rfkill->suspended = false;
+ if (!rfkill->registered)
+ return 0;
+
if (!rfkill->persistent) {
cur = !!(rfkill->state & RFKILL_BLOCK_SW);
rfkill_set_block(rfkill, cur);
}
if (frametype == ROSE_CALL_REQUEST) {
- if ((dev = rose_dev_get(dest)) != NULL) {
- if (rose_rx_call_request(skb, dev, rose_loopback_neigh, lci_o) == 0)
- kfree_skb(skb);
- } else {
+ if (!rose_loopback_neigh->dev) {
+ kfree_skb(skb);
+ continue;
+ }
+
+ dev = rose_dev_get(dest);
+ if (!dev) {
+ kfree_skb(skb);
+ continue;
+ }
+
+ if (rose_rx_call_request(skb, dev, rose_loopback_neigh, lci_o) == 0) {
+ dev_put(dev);
kfree_skb(skb);
}
} else {
rtt.o \
security.o \
sendmsg.o \
+ server_key.o \
skbuff.o \
utils.o
#include <net/netns/generic.h>
#include <net/sock.h>
#include <net/af_rxrpc.h>
+#include <keys/rxrpc-type.h>
#include "protocol.h"
#if 0
#define rxrpc_queue_delayed_work(WS,D) \
queue_delayed_work(rxrpc_workqueue, (WS), (D))
+struct key_preparsed_payload;
struct rxrpc_connection;
/*
/* Clean up a security service */
void (*exit)(void);
+ /* Parse the information from a server key */
+ int (*preparse_server_key)(struct key_preparsed_payload *);
+
+ /* Clean up the preparse buffer after parsing a server key */
+ void (*free_preparse_server_key)(struct key_preparsed_payload *);
+
+ /* Destroy the payload of a server key */
+ void (*destroy_server_key)(struct key *);
+
+ /* Describe a server key */
+ void (*describe_server_key)(const struct key *, struct seq_file *);
+
/* initialise a connection's security */
- int (*init_connection_security)(struct rxrpc_connection *);
+ int (*init_connection_security)(struct rxrpc_connection *,
+ struct rxrpc_key_token *);
- /* prime a connection's packet security */
- int (*prime_packet_security)(struct rxrpc_connection *);
+ /* Work out how much data we can store in a packet, given an estimate
+ * of the amount of data remaining.
+ */
+ int (*how_much_data)(struct rxrpc_call *, size_t,
+ size_t *, size_t *, size_t *);
/* impose security on a packet */
- int (*secure_packet)(struct rxrpc_call *,
- struct sk_buff *,
- size_t,
- void *);
+ int (*secure_packet)(struct rxrpc_call *, struct sk_buff *, size_t);
/* verify the security on a received packet */
int (*verify_packet)(struct rxrpc_call *, struct sk_buff *,
struct list_head proc_link; /* link in procfs list */
struct list_head link; /* link in master connection list */
struct sk_buff_head rx_queue; /* received conn-level packets */
+
const struct rxrpc_security *security; /* applied security module */
- struct key *server_key; /* security for this service */
- struct crypto_sync_skcipher *cipher; /* encryption handle */
- struct rxrpc_crypt csum_iv; /* packet checksum base */
+ union {
+ struct {
+ struct crypto_sync_skcipher *cipher; /* encryption handle */
+ struct rxrpc_crypt csum_iv; /* packet checksum base */
+ u32 nonce; /* response re-use preventer */
+ } rxkad;
+ };
unsigned long flags;
unsigned long events;
unsigned long idle_timestamp; /* Time at which last became idle */
int debug_id; /* debug ID for printks */
atomic_t serial; /* packet serial number counter */
unsigned int hi_serial; /* highest serial number received */
- u32 security_nonce; /* response re-use preventer */
u32 service_id; /* Service ID, possibly upgraded */
- u8 size_align; /* data size alignment (for security) */
- u8 security_size; /* security header size */
u8 security_ix; /* security type */
u8 out_clientflag; /* RXRPC_CLIENT_INITIATED if we are client */
u8 bundle_shift; /* Index into bundle->avail_chans */
struct sk_buff *);
struct rxrpc_connection *rxrpc_prealloc_service_connection(struct rxrpc_net *, gfp_t);
void rxrpc_new_incoming_connection(struct rxrpc_sock *, struct rxrpc_connection *,
- const struct rxrpc_security *, struct key *,
- struct sk_buff *);
+ const struct rxrpc_security *, struct sk_buff *);
void rxrpc_unpublish_service_conn(struct rxrpc_connection *);
/*
* key.c
*/
extern struct key_type key_type_rxrpc;
-extern struct key_type key_type_rxrpc_s;
int rxrpc_request_key(struct rxrpc_sock *, sockptr_t , int);
-int rxrpc_server_keyring(struct rxrpc_sock *, sockptr_t, int);
int rxrpc_get_server_data_key(struct rxrpc_connection *, const void *, time64_t,
u32);
* security.c
*/
int __init rxrpc_init_security(void);
+const struct rxrpc_security *rxrpc_security_lookup(u8);
void rxrpc_exit_security(void);
int rxrpc_init_client_conn_security(struct rxrpc_connection *);
-bool rxrpc_look_up_server_security(struct rxrpc_local *, struct rxrpc_sock *,
- const struct rxrpc_security **, struct key **,
- struct sk_buff *);
+const struct rxrpc_security *rxrpc_get_incoming_security(struct rxrpc_sock *,
+ struct sk_buff *);
+struct key *rxrpc_look_up_server_security(struct rxrpc_connection *,
+ struct sk_buff *, u32, u32);
/*
* sendmsg.c
*/
int rxrpc_do_sendmsg(struct rxrpc_sock *, struct msghdr *, size_t);
+/*
+ * server_key.c
+ */
+extern struct key_type key_type_rxrpc_s;
+
+int rxrpc_server_keyring(struct rxrpc_sock *, sockptr_t, int);
+
/*
* skbuff.c
*/
struct rxrpc_peer *peer,
struct rxrpc_connection *conn,
const struct rxrpc_security *sec,
- struct key *key,
struct sk_buff *skb)
{
struct rxrpc_backlog *b = rx->backlog;
conn->params.local = rxrpc_get_local(local);
conn->params.peer = peer;
rxrpc_see_connection(conn);
- rxrpc_new_incoming_connection(rx, conn, sec, key, skb);
+ rxrpc_new_incoming_connection(rx, conn, sec, skb);
} else {
rxrpc_get_connection(conn);
}
struct rxrpc_connection *conn;
struct rxrpc_peer *peer = NULL;
struct rxrpc_call *call = NULL;
- struct key *key = NULL;
_enter("");
*/
conn = rxrpc_find_connection_rcu(local, skb, &peer);
- if (!conn && !rxrpc_look_up_server_security(local, rx, &sec, &key, skb))
- goto no_call;
+ if (!conn) {
+ sec = rxrpc_get_incoming_security(rx, skb);
+ if (!sec)
+ goto no_call;
+ }
- call = rxrpc_alloc_incoming_call(rx, local, peer, conn, sec, key, skb);
- key_put(key);
+ call = rxrpc_alloc_incoming_call(rx, local, peer, conn, sec, skb);
if (!call) {
skb->mark = RXRPC_SKB_MARK_REJECT_BUSY;
goto no_call;
if (ret < 0)
goto error_1;
- ret = conn->security->prime_packet_security(conn);
- if (ret < 0)
- goto error_2;
-
atomic_inc(&rxnet->nr_conns);
write_lock(&rxnet->conn_lock);
list_add_tail(&conn->proc_link, &rxnet->conn_proc_list);
_leave(" = %p", conn);
return conn;
-error_2:
- conn->security->clear(conn);
error_1:
rxrpc_put_client_connection_id(conn);
error_0:
if (ret < 0)
return ret;
- ret = conn->security->init_connection_security(conn);
- if (ret < 0)
- return ret;
-
- ret = conn->security->prime_packet_security(conn);
+ ret = conn->security->init_connection_security(
+ conn, conn->params.key->payload.data[0]);
if (ret < 0)
return ret;
_enter("{%d}", conn->debug_id);
ASSERT(conn->security_ix != 0);
- ASSERT(conn->server_key);
if (conn->security->issue_challenge(conn) < 0) {
abort_code = RX_CALL_DEAD;
conn->security = &rxrpc_no_security;
spin_lock_init(&conn->state_lock);
conn->debug_id = atomic_inc_return(&rxrpc_debug_id);
- conn->size_align = 4;
conn->idle_timestamp = jiffies;
}
conn->security->clear(conn);
key_put(conn->params.key);
- key_put(conn->server_key);
rxrpc_put_bundle(conn->bundle);
rxrpc_put_peer(conn->params.peer);
void rxrpc_new_incoming_connection(struct rxrpc_sock *rx,
struct rxrpc_connection *conn,
const struct rxrpc_security *sec,
- struct key *key,
struct sk_buff *skb)
{
struct rxrpc_skb_priv *sp = rxrpc_skb(skb);
conn->security_ix = sp->hdr.securityIndex;
conn->out_clientflag = 0;
conn->security = sec;
- conn->server_key = key_get(key);
if (conn->security_ix)
conn->state = RXRPC_CONN_SERVICE_UNSECURED;
else
#include <net/af_rxrpc.h>
#include "ar-internal.h"
-static int none_init_connection_security(struct rxrpc_connection *conn)
+static int none_init_connection_security(struct rxrpc_connection *conn,
+ struct rxrpc_key_token *token)
{
return 0;
}
-static int none_prime_packet_security(struct rxrpc_connection *conn)
+/*
+ * Work out how much data we can put in an unsecured packet.
+ */
+static int none_how_much_data(struct rxrpc_call *call, size_t remain,
+ size_t *_buf_size, size_t *_data_size, size_t *_offset)
{
+ *_buf_size = *_data_size = min_t(size_t, remain, RXRPC_JUMBO_DATALEN);
+ *_offset = 0;
return 0;
}
-static int none_secure_packet(struct rxrpc_call *call,
- struct sk_buff *skb,
- size_t data_size,
- void *sechdr)
+static int none_secure_packet(struct rxrpc_call *call, struct sk_buff *skb,
+ size_t data_size)
{
return 0;
}
.init = none_init,
.exit = none_exit,
.init_connection_security = none_init_connection_security,
- .prime_packet_security = none_prime_packet_security,
.free_call_crypto = none_free_call_crypto,
+ .how_much_data = none_how_much_data,
.secure_packet = none_secure_packet,
.verify_packet = none_verify_packet,
.locate_data = none_locate_data,
* Written by David Howells (dhowells@redhat.com)
*
* RxRPC keys should have a description of describing their purpose:
- * "afs@CAMBRIDGE.REDHAT.COM>
+ * "afs@example.com"
*/
#define pr_fmt(fmt) KBUILD_MODNAME ": " fmt
#include <keys/user-type.h>
#include "ar-internal.h"
-static int rxrpc_vet_description_s(const char *);
static int rxrpc_preparse(struct key_preparsed_payload *);
-static int rxrpc_preparse_s(struct key_preparsed_payload *);
static void rxrpc_free_preparse(struct key_preparsed_payload *);
-static void rxrpc_free_preparse_s(struct key_preparsed_payload *);
static void rxrpc_destroy(struct key *);
-static void rxrpc_destroy_s(struct key *);
static void rxrpc_describe(const struct key *, struct seq_file *);
static long rxrpc_read(const struct key *, char *, size_t);
};
EXPORT_SYMBOL(key_type_rxrpc);
-/*
- * rxrpc server defined keys take "<serviceId>:<securityIndex>" as the
- * description and an 8-byte decryption key as the payload
- */
-struct key_type key_type_rxrpc_s = {
- .name = "rxrpc_s",
- .flags = KEY_TYPE_NET_DOMAIN,
- .vet_description = rxrpc_vet_description_s,
- .preparse = rxrpc_preparse_s,
- .free_preparse = rxrpc_free_preparse_s,
- .instantiate = generic_key_instantiate,
- .destroy = rxrpc_destroy_s,
- .describe = rxrpc_describe,
-};
-
-/*
- * Vet the description for an RxRPC server key
- */
-static int rxrpc_vet_description_s(const char *desc)
-{
- unsigned long num;
- char *p;
-
- num = simple_strtoul(desc, &p, 10);
- if (*p != ':' || num > 65535)
- return -EINVAL;
- num = simple_strtoul(p + 1, &p, 10);
- if (*p || num < 1 || num > 255)
- return -EINVAL;
- return 0;
-}
-
/*
* parse an RxKAD type XDR format token
* - the caller guarantees we have at least 4 words
return 0;
}
-static void rxrpc_free_krb5_principal(struct krb5_principal *princ)
-{
- int loop;
-
- if (princ->name_parts) {
- for (loop = princ->n_name_parts - 1; loop >= 0; loop--)
- kfree(princ->name_parts[loop]);
- kfree(princ->name_parts);
- }
- kfree(princ->realm);
-}
-
-static void rxrpc_free_krb5_tagged(struct krb5_tagged_data *td)
-{
- kfree(td->data);
-}
-
-/*
- * free up an RxK5 token
- */
-static void rxrpc_rxk5_free(struct rxk5_key *rxk5)
-{
- int loop;
-
- rxrpc_free_krb5_principal(&rxk5->client);
- rxrpc_free_krb5_principal(&rxk5->server);
- rxrpc_free_krb5_tagged(&rxk5->session);
-
- if (rxk5->addresses) {
- for (loop = rxk5->n_addresses - 1; loop >= 0; loop--)
- rxrpc_free_krb5_tagged(&rxk5->addresses[loop]);
- kfree(rxk5->addresses);
- }
- if (rxk5->authdata) {
- for (loop = rxk5->n_authdata - 1; loop >= 0; loop--)
- rxrpc_free_krb5_tagged(&rxk5->authdata[loop]);
- kfree(rxk5->authdata);
- }
-
- kfree(rxk5->ticket);
- kfree(rxk5->ticket2);
- kfree(rxk5);
-}
-
-/*
- * extract a krb5 principal
- */
-static int rxrpc_krb5_decode_principal(struct krb5_principal *princ,
- const __be32 **_xdr,
- unsigned int *_toklen)
-{
- const __be32 *xdr = *_xdr;
- unsigned int toklen = *_toklen, n_parts, loop, tmp, paddedlen;
-
- /* there must be at least one name, and at least #names+1 length
- * words */
- if (toklen <= 12)
- return -EINVAL;
-
- _enter(",{%x,%x,%x},%u",
- ntohl(xdr[0]), ntohl(xdr[1]), ntohl(xdr[2]), toklen);
-
- n_parts = ntohl(*xdr++);
- toklen -= 4;
- if (n_parts <= 0 || n_parts > AFSTOKEN_K5_COMPONENTS_MAX)
- return -EINVAL;
- princ->n_name_parts = n_parts;
-
- if (toklen <= (n_parts + 1) * 4)
- return -EINVAL;
-
- princ->name_parts = kcalloc(n_parts, sizeof(char *), GFP_KERNEL);
- if (!princ->name_parts)
- return -ENOMEM;
-
- for (loop = 0; loop < n_parts; loop++) {
- if (toklen < 4)
- return -EINVAL;
- tmp = ntohl(*xdr++);
- toklen -= 4;
- if (tmp <= 0 || tmp > AFSTOKEN_STRING_MAX)
- return -EINVAL;
- paddedlen = (tmp + 3) & ~3;
- if (paddedlen > toklen)
- return -EINVAL;
- princ->name_parts[loop] = kmalloc(tmp + 1, GFP_KERNEL);
- if (!princ->name_parts[loop])
- return -ENOMEM;
- memcpy(princ->name_parts[loop], xdr, tmp);
- princ->name_parts[loop][tmp] = 0;
- toklen -= paddedlen;
- xdr += paddedlen >> 2;
- }
-
- if (toklen < 4)
- return -EINVAL;
- tmp = ntohl(*xdr++);
- toklen -= 4;
- if (tmp <= 0 || tmp > AFSTOKEN_K5_REALM_MAX)
- return -EINVAL;
- paddedlen = (tmp + 3) & ~3;
- if (paddedlen > toklen)
- return -EINVAL;
- princ->realm = kmalloc(tmp + 1, GFP_KERNEL);
- if (!princ->realm)
- return -ENOMEM;
- memcpy(princ->realm, xdr, tmp);
- princ->realm[tmp] = 0;
- toklen -= paddedlen;
- xdr += paddedlen >> 2;
-
- _debug("%s/...@%s", princ->name_parts[0], princ->realm);
-
- *_xdr = xdr;
- *_toklen = toklen;
- _leave(" = 0 [toklen=%u]", toklen);
- return 0;
-}
-
-/*
- * extract a piece of krb5 tagged data
- */
-static int rxrpc_krb5_decode_tagged_data(struct krb5_tagged_data *td,
- size_t max_data_size,
- const __be32 **_xdr,
- unsigned int *_toklen)
-{
- const __be32 *xdr = *_xdr;
- unsigned int toklen = *_toklen, len, paddedlen;
-
- /* there must be at least one tag and one length word */
- if (toklen <= 8)
- return -EINVAL;
-
- _enter(",%zu,{%x,%x},%u",
- max_data_size, ntohl(xdr[0]), ntohl(xdr[1]), toklen);
-
- td->tag = ntohl(*xdr++);
- len = ntohl(*xdr++);
- toklen -= 8;
- if (len > max_data_size)
- return -EINVAL;
- paddedlen = (len + 3) & ~3;
- if (paddedlen > toklen)
- return -EINVAL;
- td->data_len = len;
-
- if (len > 0) {
- td->data = kmemdup(xdr, len, GFP_KERNEL);
- if (!td->data)
- return -ENOMEM;
- toklen -= paddedlen;
- xdr += paddedlen >> 2;
- }
-
- _debug("tag %x len %x", td->tag, td->data_len);
-
- *_xdr = xdr;
- *_toklen = toklen;
- _leave(" = 0 [toklen=%u]", toklen);
- return 0;
-}
-
-/*
- * extract an array of tagged data
- */
-static int rxrpc_krb5_decode_tagged_array(struct krb5_tagged_data **_td,
- u8 *_n_elem,
- u8 max_n_elem,
- size_t max_elem_size,
- const __be32 **_xdr,
- unsigned int *_toklen)
-{
- struct krb5_tagged_data *td;
- const __be32 *xdr = *_xdr;
- unsigned int toklen = *_toklen, n_elem, loop;
- int ret;
-
- /* there must be at least one count */
- if (toklen < 4)
- return -EINVAL;
-
- _enter(",,%u,%zu,{%x},%u",
- max_n_elem, max_elem_size, ntohl(xdr[0]), toklen);
-
- n_elem = ntohl(*xdr++);
- toklen -= 4;
- if (n_elem > max_n_elem)
- return -EINVAL;
- *_n_elem = n_elem;
- if (n_elem > 0) {
- if (toklen <= (n_elem + 1) * 4)
- return -EINVAL;
-
- _debug("n_elem %d", n_elem);
-
- td = kcalloc(n_elem, sizeof(struct krb5_tagged_data),
- GFP_KERNEL);
- if (!td)
- return -ENOMEM;
- *_td = td;
-
- for (loop = 0; loop < n_elem; loop++) {
- ret = rxrpc_krb5_decode_tagged_data(&td[loop],
- max_elem_size,
- &xdr, &toklen);
- if (ret < 0)
- return ret;
- }
- }
-
- *_xdr = xdr;
- *_toklen = toklen;
- _leave(" = 0 [toklen=%u]", toklen);
- return 0;
-}
-
-/*
- * extract a krb5 ticket
- */
-static int rxrpc_krb5_decode_ticket(u8 **_ticket, u16 *_tktlen,
- const __be32 **_xdr, unsigned int *_toklen)
-{
- const __be32 *xdr = *_xdr;
- unsigned int toklen = *_toklen, len, paddedlen;
-
- /* there must be at least one length word */
- if (toklen <= 4)
- return -EINVAL;
-
- _enter(",{%x},%u", ntohl(xdr[0]), toklen);
-
- len = ntohl(*xdr++);
- toklen -= 4;
- if (len > AFSTOKEN_K5_TIX_MAX)
- return -EINVAL;
- paddedlen = (len + 3) & ~3;
- if (paddedlen > toklen)
- return -EINVAL;
- *_tktlen = len;
-
- _debug("ticket len %u", len);
-
- if (len > 0) {
- *_ticket = kmemdup(xdr, len, GFP_KERNEL);
- if (!*_ticket)
- return -ENOMEM;
- toklen -= paddedlen;
- xdr += paddedlen >> 2;
- }
-
- *_xdr = xdr;
- *_toklen = toklen;
- _leave(" = 0 [toklen=%u]", toklen);
- return 0;
-}
-
-/*
- * parse an RxK5 type XDR format token
- * - the caller guarantees we have at least 4 words
- */
-static int rxrpc_preparse_xdr_rxk5(struct key_preparsed_payload *prep,
- size_t datalen,
- const __be32 *xdr, unsigned int toklen)
-{
- struct rxrpc_key_token *token, **pptoken;
- struct rxk5_key *rxk5;
- const __be32 *end_xdr = xdr + (toklen >> 2);
- time64_t expiry;
- int ret;
-
- _enter(",{%x,%x,%x,%x},%u",
- ntohl(xdr[0]), ntohl(xdr[1]), ntohl(xdr[2]), ntohl(xdr[3]),
- toklen);
-
- /* reserve some payload space for this subkey - the length of the token
- * is a reasonable approximation */
- prep->quotalen = datalen + toklen;
-
- token = kzalloc(sizeof(*token), GFP_KERNEL);
- if (!token)
- return -ENOMEM;
-
- rxk5 = kzalloc(sizeof(*rxk5), GFP_KERNEL);
- if (!rxk5) {
- kfree(token);
- return -ENOMEM;
- }
-
- token->security_index = RXRPC_SECURITY_RXK5;
- token->k5 = rxk5;
-
- /* extract the principals */
- ret = rxrpc_krb5_decode_principal(&rxk5->client, &xdr, &toklen);
- if (ret < 0)
- goto error;
- ret = rxrpc_krb5_decode_principal(&rxk5->server, &xdr, &toklen);
- if (ret < 0)
- goto error;
-
- /* extract the session key and the encoding type (the tag field ->
- * ENCTYPE_xxx) */
- ret = rxrpc_krb5_decode_tagged_data(&rxk5->session, AFSTOKEN_DATA_MAX,
- &xdr, &toklen);
- if (ret < 0)
- goto error;
-
- if (toklen < 4 * 8 + 2 * 4)
- goto inval;
- rxk5->authtime = be64_to_cpup((const __be64 *) xdr);
- xdr += 2;
- rxk5->starttime = be64_to_cpup((const __be64 *) xdr);
- xdr += 2;
- rxk5->endtime = be64_to_cpup((const __be64 *) xdr);
- xdr += 2;
- rxk5->renew_till = be64_to_cpup((const __be64 *) xdr);
- xdr += 2;
- rxk5->is_skey = ntohl(*xdr++);
- rxk5->flags = ntohl(*xdr++);
- toklen -= 4 * 8 + 2 * 4;
-
- _debug("times: a=%llx s=%llx e=%llx rt=%llx",
- rxk5->authtime, rxk5->starttime, rxk5->endtime,
- rxk5->renew_till);
- _debug("is_skey=%x flags=%x", rxk5->is_skey, rxk5->flags);
-
- /* extract the permitted client addresses */
- ret = rxrpc_krb5_decode_tagged_array(&rxk5->addresses,
- &rxk5->n_addresses,
- AFSTOKEN_K5_ADDRESSES_MAX,
- AFSTOKEN_DATA_MAX,
- &xdr, &toklen);
- if (ret < 0)
- goto error;
-
- ASSERTCMP((end_xdr - xdr) << 2, ==, toklen);
-
- /* extract the tickets */
- ret = rxrpc_krb5_decode_ticket(&rxk5->ticket, &rxk5->ticket_len,
- &xdr, &toklen);
- if (ret < 0)
- goto error;
- ret = rxrpc_krb5_decode_ticket(&rxk5->ticket2, &rxk5->ticket2_len,
- &xdr, &toklen);
- if (ret < 0)
- goto error;
-
- ASSERTCMP((end_xdr - xdr) << 2, ==, toklen);
-
- /* extract the typed auth data */
- ret = rxrpc_krb5_decode_tagged_array(&rxk5->authdata,
- &rxk5->n_authdata,
- AFSTOKEN_K5_AUTHDATA_MAX,
- AFSTOKEN_BDATALN_MAX,
- &xdr, &toklen);
- if (ret < 0)
- goto error;
-
- ASSERTCMP((end_xdr - xdr) << 2, ==, toklen);
-
- if (toklen != 0)
- goto inval;
-
- /* attach the payload */
- for (pptoken = (struct rxrpc_key_token **)&prep->payload.data[0];
- *pptoken;
- pptoken = &(*pptoken)->next)
- continue;
- *pptoken = token;
- expiry = rxrpc_u32_to_time64(token->k5->endtime);
- if (expiry < prep->expiry)
- prep->expiry = expiry;
-
- _leave(" = 0");
- return 0;
-
-inval:
- ret = -EINVAL;
-error:
- rxrpc_rxk5_free(rxk5);
- kfree(token);
- _leave(" = %d", ret);
- return ret;
-}
-
/*
* attempt to parse the data as the XDR format
* - the caller guarantees we have more than 7 words
*/
static int rxrpc_preparse_xdr(struct key_preparsed_payload *prep)
{
- const __be32 *xdr = prep->data, *token;
+ const __be32 *xdr = prep->data, *token, *p;
const char *cp;
unsigned int len, paddedlen, loop, ntoken, toklen, sec_ix;
size_t datalen = prep->datalen;
- int ret;
+ int ret, ret2;
_enter(",{%x,%x,%x,%x},%zu",
ntohl(xdr[0]), ntohl(xdr[1]), ntohl(xdr[2]), ntohl(xdr[3]),
goto not_xdr;
/* check each token wrapper */
- token = xdr;
+ p = xdr;
loop = ntoken;
do {
if (datalen < 8)
goto not_xdr;
- toklen = ntohl(*xdr++);
- sec_ix = ntohl(*xdr);
+ toklen = ntohl(*p++);
+ sec_ix = ntohl(*p);
datalen -= 4;
_debug("token: [%x/%zx] %x", toklen, datalen, sec_ix);
paddedlen = (toklen + 3) & ~3;
if (toklen < 20 || toklen > datalen || paddedlen > datalen)
goto not_xdr;
datalen -= paddedlen;
- xdr += paddedlen >> 2;
+ p += paddedlen >> 2;
} while (--loop > 0);
/* okay: we're going to assume it's valid XDR format
* - we ignore the cellname, relying on the key to be correctly named
*/
+ ret = -EPROTONOSUPPORT;
do {
- xdr = token;
toklen = ntohl(*xdr++);
- token = xdr + ((toklen + 3) >> 2);
- sec_ix = ntohl(*xdr++);
+ token = xdr;
+ xdr += (toklen + 3) / 4;
+
+ sec_ix = ntohl(*token++);
toklen -= 4;
- _debug("TOKEN type=%u [%p-%p]", sec_ix, xdr, token);
+ _debug("TOKEN type=%x len=%x", sec_ix, toklen);
switch (sec_ix) {
case RXRPC_SECURITY_RXKAD:
- ret = rxrpc_preparse_xdr_rxkad(prep, datalen, xdr, toklen);
- if (ret != 0)
- goto error;
+ ret2 = rxrpc_preparse_xdr_rxkad(prep, datalen, token, toklen);
break;
+ default:
+ ret2 = -EPROTONOSUPPORT;
+ break;
+ }
- case RXRPC_SECURITY_RXK5:
- ret = rxrpc_preparse_xdr_rxk5(prep, datalen, xdr, toklen);
+ switch (ret2) {
+ case 0:
+ ret = 0;
+ break;
+ case -EPROTONOSUPPORT:
+ break;
+ case -ENOPKG:
if (ret != 0)
- goto error;
+ ret = -ENOPKG;
break;
-
default:
- ret = -EPROTONOSUPPORT;
+ ret = ret2;
goto error;
}
} while (--ntoken > 0);
- _leave(" = 0");
- return 0;
+error:
+ _leave(" = %d", ret);
+ return ret;
not_xdr:
_leave(" = -EPROTO");
return -EPROTO;
-error:
- _leave(" = %d", ret);
- return ret;
}
/*
case RXRPC_SECURITY_RXKAD:
kfree(token->kad);
break;
- case RXRPC_SECURITY_RXK5:
- if (token->k5)
- rxrpc_rxk5_free(token->k5);
- break;
default:
pr_err("Unknown token type %x on rxrpc key\n",
token->security_index);
rxrpc_free_token_list(prep->payload.data[0]);
}
-/*
- * Preparse a server secret key.
- *
- * The data should be the 8-byte secret key.
- */
-static int rxrpc_preparse_s(struct key_preparsed_payload *prep)
-{
- struct crypto_skcipher *ci;
-
- _enter("%zu", prep->datalen);
-
- if (prep->datalen != 8)
- return -EINVAL;
-
- memcpy(&prep->payload.data[2], prep->data, 8);
-
- ci = crypto_alloc_skcipher("pcbc(des)", 0, CRYPTO_ALG_ASYNC);
- if (IS_ERR(ci)) {
- _leave(" = %ld", PTR_ERR(ci));
- return PTR_ERR(ci);
- }
-
- if (crypto_skcipher_setkey(ci, prep->data, 8) < 0)
- BUG();
-
- prep->payload.data[0] = ci;
- _leave(" = 0");
- return 0;
-}
-
-/*
- * Clean up preparse data.
- */
-static void rxrpc_free_preparse_s(struct key_preparsed_payload *prep)
-{
- if (prep->payload.data[0])
- crypto_free_skcipher(prep->payload.data[0]);
-}
-
/*
* dispose of the data dangling from the corpse of a rxrpc key
*/
rxrpc_free_token_list(key->payload.data[0]);
}
-/*
- * dispose of the data dangling from the corpse of a rxrpc key
- */
-static void rxrpc_destroy_s(struct key *key)
-{
- if (key->payload.data[0]) {
- crypto_free_skcipher(key->payload.data[0]);
- key->payload.data[0] = NULL;
- }
-}
-
/*
* describe the rxrpc key
*/
static void rxrpc_describe(const struct key *key, struct seq_file *m)
{
+ const struct rxrpc_key_token *token;
+ const char *sep = ": ";
+
seq_puts(m, key->description);
+
+ for (token = key->payload.data[0]; token; token = token->next) {
+ seq_puts(m, sep);
+
+ switch (token->security_index) {
+ case RXRPC_SECURITY_RXKAD:
+ seq_puts(m, "ka");
+ break;
+ default: /* we have a ticket we can't encode */
+ seq_printf(m, "%u", token->security_index);
+ break;
+ }
+
+ sep = " ";
+ }
}
/*
return 0;
}
-/*
- * grab the security keyring for a server socket
- */
-int rxrpc_server_keyring(struct rxrpc_sock *rx, sockptr_t optval, int optlen)
-{
- struct key *key;
- char *description;
-
- _enter("");
-
- if (optlen <= 0 || optlen > PAGE_SIZE - 1)
- return -EINVAL;
-
- description = memdup_sockptr_nul(optval, optlen);
- if (IS_ERR(description))
- return PTR_ERR(description);
-
- key = request_key(&key_type_keyring, description, NULL);
- if (IS_ERR(key)) {
- kfree(description);
- _leave(" = %ld", PTR_ERR(key));
- return PTR_ERR(key);
- }
-
- rx->securities = key;
- kfree(description);
- _leave(" = 0 [key %x]", key->serial);
- return 0;
-}
-
/*
* generate a server data key
*/
char *buffer, size_t buflen)
{
const struct rxrpc_key_token *token;
- const struct krb5_principal *princ;
size_t size;
__be32 *xdr, *oldxdr;
u32 cnlen, toksize, ntoks, tok, zero;
u16 toksizes[AFSTOKEN_MAX];
- int loop;
_enter("");
case RXRPC_SECURITY_RXKAD:
toksize += 8 * 4; /* viceid, kvno, key*2, begin,
* end, primary, tktlen */
- toksize += RND(token->kad->ticket_len);
- break;
-
- case RXRPC_SECURITY_RXK5:
- princ = &token->k5->client;
- toksize += 4 + princ->n_name_parts * 4;
- for (loop = 0; loop < princ->n_name_parts; loop++)
- toksize += RND(strlen(princ->name_parts[loop]));
- toksize += 4 + RND(strlen(princ->realm));
-
- princ = &token->k5->server;
- toksize += 4 + princ->n_name_parts * 4;
- for (loop = 0; loop < princ->n_name_parts; loop++)
- toksize += RND(strlen(princ->name_parts[loop]));
- toksize += 4 + RND(strlen(princ->realm));
-
- toksize += 8 + RND(token->k5->session.data_len);
-
- toksize += 4 * 8 + 2 * 4;
-
- toksize += 4 + token->k5->n_addresses * 8;
- for (loop = 0; loop < token->k5->n_addresses; loop++)
- toksize += RND(token->k5->addresses[loop].data_len);
-
- toksize += 4 + RND(token->k5->ticket_len);
- toksize += 4 + RND(token->k5->ticket2_len);
-
- toksize += 4 + token->k5->n_authdata * 8;
- for (loop = 0; loop < token->k5->n_authdata; loop++)
- toksize += RND(token->k5->authdata[loop].data_len);
+ if (!token->no_leak_key)
+ toksize += RND(token->kad->ticket_len);
break;
default: /* we have a ticket we can't encode */
ENCODE(token->kad->start);
ENCODE(token->kad->expiry);
ENCODE(token->kad->primary_flag);
- ENCODE_DATA(token->kad->ticket_len, token->kad->ticket);
- break;
-
- case RXRPC_SECURITY_RXK5:
- princ = &token->k5->client;
- ENCODE(princ->n_name_parts);
- for (loop = 0; loop < princ->n_name_parts; loop++)
- ENCODE_STR(princ->name_parts[loop]);
- ENCODE_STR(princ->realm);
-
- princ = &token->k5->server;
- ENCODE(princ->n_name_parts);
- for (loop = 0; loop < princ->n_name_parts; loop++)
- ENCODE_STR(princ->name_parts[loop]);
- ENCODE_STR(princ->realm);
-
- ENCODE(token->k5->session.tag);
- ENCODE_DATA(token->k5->session.data_len,
- token->k5->session.data);
-
- ENCODE64(token->k5->authtime);
- ENCODE64(token->k5->starttime);
- ENCODE64(token->k5->endtime);
- ENCODE64(token->k5->renew_till);
- ENCODE(token->k5->is_skey);
- ENCODE(token->k5->flags);
-
- ENCODE(token->k5->n_addresses);
- for (loop = 0; loop < token->k5->n_addresses; loop++) {
- ENCODE(token->k5->addresses[loop].tag);
- ENCODE_DATA(token->k5->addresses[loop].data_len,
- token->k5->addresses[loop].data);
- }
-
- ENCODE_DATA(token->k5->ticket_len, token->k5->ticket);
- ENCODE_DATA(token->k5->ticket2_len, token->k5->ticket2);
-
- ENCODE(token->k5->n_authdata);
- for (loop = 0; loop < token->k5->n_authdata; loop++) {
- ENCODE(token->k5->authdata[loop].tag);
- ENCODE_DATA(token->k5->authdata[loop].data_len,
- token->k5->authdata[loop].data);
- }
+ if (token->no_leak_key)
+ ENCODE(0);
+ else
+ ENCODE_DATA(token->kad->ticket_len, token->kad->ticket);
break;
default:
#include <linux/scatterlist.h>
#include <linux/ctype.h>
#include <linux/slab.h>
+#include <linux/key-type.h>
#include <net/sock.h>
#include <net/af_rxrpc.h>
#include <keys/rxrpc-type.h>
#define INST_SZ 40 /* size of principal's instance */
#define REALM_SZ 40 /* size of principal's auth domain */
#define SNAME_SZ 40 /* size of service name */
+#define RXKAD_ALIGN 8
struct rxkad_level1_hdr {
__be32 data_size; /* true data size (excluding padding) */
__be32 checksum; /* decrypted data checksum */
};
+static int rxkad_prime_packet_security(struct rxrpc_connection *conn,
+ struct crypto_sync_skcipher *ci);
+
/*
* this holds a pinned cipher so that keventd doesn't get called by the cipher
* alloc routine, but since we have it to hand, we use it to decrypt RESPONSE
static struct skcipher_request *rxkad_ci_req;
static DEFINE_MUTEX(rxkad_ci_mutex);
+/*
+ * Parse the information from a server key
+ *
+ * The data should be the 8-byte secret key.
+ */
+static int rxkad_preparse_server_key(struct key_preparsed_payload *prep)
+{
+ struct crypto_skcipher *ci;
+
+ if (prep->datalen != 8)
+ return -EINVAL;
+
+ memcpy(&prep->payload.data[2], prep->data, 8);
+
+ ci = crypto_alloc_skcipher("pcbc(des)", 0, CRYPTO_ALG_ASYNC);
+ if (IS_ERR(ci)) {
+ _leave(" = %ld", PTR_ERR(ci));
+ return PTR_ERR(ci);
+ }
+
+ if (crypto_skcipher_setkey(ci, prep->data, 8) < 0)
+ BUG();
+
+ prep->payload.data[0] = ci;
+ _leave(" = 0");
+ return 0;
+}
+
+static void rxkad_free_preparse_server_key(struct key_preparsed_payload *prep)
+{
+
+ if (prep->payload.data[0])
+ crypto_free_skcipher(prep->payload.data[0]);
+}
+
+static void rxkad_destroy_server_key(struct key *key)
+{
+ if (key->payload.data[0]) {
+ crypto_free_skcipher(key->payload.data[0]);
+ key->payload.data[0] = NULL;
+ }
+}
+
/*
* initialise connection security
*/
-static int rxkad_init_connection_security(struct rxrpc_connection *conn)
+static int rxkad_init_connection_security(struct rxrpc_connection *conn,
+ struct rxrpc_key_token *token)
{
struct crypto_sync_skcipher *ci;
- struct rxrpc_key_token *token;
int ret;
_enter("{%d},{%x}", conn->debug_id, key_serial(conn->params.key));
- token = conn->params.key->payload.data[0];
conn->security_ix = token->security_index;
ci = crypto_alloc_sync_skcipher("pcbc(fcrypt)", 0, 0);
switch (conn->params.security_level) {
case RXRPC_SECURITY_PLAIN:
- break;
case RXRPC_SECURITY_AUTH:
- conn->size_align = 8;
- conn->security_size = sizeof(struct rxkad_level1_hdr);
- break;
case RXRPC_SECURITY_ENCRYPT:
- conn->size_align = 8;
- conn->security_size = sizeof(struct rxkad_level2_hdr);
break;
default:
ret = -EKEYREJECTED;
goto error;
}
- conn->cipher = ci;
- ret = 0;
+ ret = rxkad_prime_packet_security(conn, ci);
+ if (ret < 0)
+ goto error_ci;
+
+ conn->rxkad.cipher = ci;
+ return 0;
+
+error_ci:
+ crypto_free_sync_skcipher(ci);
error:
_leave(" = %d", ret);
return ret;
}
+/*
+ * Work out how much data we can put in a packet.
+ */
+static int rxkad_how_much_data(struct rxrpc_call *call, size_t remain,
+ size_t *_buf_size, size_t *_data_size, size_t *_offset)
+{
+ size_t shdr, buf_size, chunk;
+
+ switch (call->conn->params.security_level) {
+ default:
+ buf_size = chunk = min_t(size_t, remain, RXRPC_JUMBO_DATALEN);
+ shdr = 0;
+ goto out;
+ case RXRPC_SECURITY_AUTH:
+ shdr = sizeof(struct rxkad_level1_hdr);
+ break;
+ case RXRPC_SECURITY_ENCRYPT:
+ shdr = sizeof(struct rxkad_level2_hdr);
+ break;
+ }
+
+ buf_size = round_down(RXRPC_JUMBO_DATALEN, RXKAD_ALIGN);
+
+ chunk = buf_size - shdr;
+ if (remain < chunk)
+ buf_size = round_up(shdr + remain, RXKAD_ALIGN);
+
+out:
+ *_buf_size = buf_size;
+ *_data_size = chunk;
+ *_offset = shdr;
+ return 0;
+}
+
/*
* prime the encryption state with the invariant parts of a connection's
* description
*/
-static int rxkad_prime_packet_security(struct rxrpc_connection *conn)
+static int rxkad_prime_packet_security(struct rxrpc_connection *conn,
+ struct crypto_sync_skcipher *ci)
{
struct skcipher_request *req;
struct rxrpc_key_token *token;
if (!tmpbuf)
return -ENOMEM;
- req = skcipher_request_alloc(&conn->cipher->base, GFP_NOFS);
+ req = skcipher_request_alloc(&ci->base, GFP_NOFS);
if (!req) {
kfree(tmpbuf);
return -ENOMEM;
tmpbuf[3] = htonl(conn->security_ix);
sg_init_one(&sg, tmpbuf, tmpsize);
- skcipher_request_set_sync_tfm(req, conn->cipher);
+ skcipher_request_set_sync_tfm(req, ci);
skcipher_request_set_callback(req, 0, NULL, NULL);
skcipher_request_set_crypt(req, &sg, &sg, tmpsize, iv.x);
crypto_skcipher_encrypt(req);
skcipher_request_free(req);
- memcpy(&conn->csum_iv, tmpbuf + 2, sizeof(conn->csum_iv));
+ memcpy(&conn->rxkad.csum_iv, tmpbuf + 2, sizeof(conn->rxkad.csum_iv));
kfree(tmpbuf);
_leave(" = 0");
return 0;
*/
static struct skcipher_request *rxkad_get_call_crypto(struct rxrpc_call *call)
{
- struct crypto_skcipher *tfm = &call->conn->cipher->base;
+ struct crypto_skcipher *tfm = &call->conn->rxkad.cipher->base;
struct skcipher_request *cipher_req = call->cipher_req;
if (!cipher_req) {
* partially encrypt a packet (level 1 security)
*/
static int rxkad_secure_packet_auth(const struct rxrpc_call *call,
- struct sk_buff *skb,
- u32 data_size,
- void *sechdr,
+ struct sk_buff *skb, u32 data_size,
struct skcipher_request *req)
{
struct rxrpc_skb_priv *sp = rxrpc_skb(skb);
struct rxkad_level1_hdr hdr;
struct rxrpc_crypt iv;
struct scatterlist sg;
+ size_t pad;
u16 check;
_enter("");
data_size |= (u32)check << 16;
hdr.data_size = htonl(data_size);
- memcpy(sechdr, &hdr, sizeof(hdr));
+ memcpy(skb->head, &hdr, sizeof(hdr));
+
+ pad = sizeof(struct rxkad_level1_hdr) + data_size;
+ pad = RXKAD_ALIGN - pad;
+ pad &= RXKAD_ALIGN - 1;
+ if (pad)
+ skb_put_zero(skb, pad);
/* start the encryption afresh */
memset(&iv, 0, sizeof(iv));
- sg_init_one(&sg, sechdr, 8);
- skcipher_request_set_sync_tfm(req, call->conn->cipher);
+ sg_init_one(&sg, skb->head, 8);
+ skcipher_request_set_sync_tfm(req, call->conn->rxkad.cipher);
skcipher_request_set_callback(req, 0, NULL, NULL);
skcipher_request_set_crypt(req, &sg, &sg, 8, iv.x);
crypto_skcipher_encrypt(req);
static int rxkad_secure_packet_encrypt(const struct rxrpc_call *call,
struct sk_buff *skb,
u32 data_size,
- void *sechdr,
struct skcipher_request *req)
{
const struct rxrpc_key_token *token;
struct rxrpc_crypt iv;
struct scatterlist sg[16];
unsigned int len;
+ size_t pad;
u16 check;
int err;
rxkhdr.data_size = htonl(data_size | (u32)check << 16);
rxkhdr.checksum = 0;
- memcpy(sechdr, &rxkhdr, sizeof(rxkhdr));
+ memcpy(skb->head, &rxkhdr, sizeof(rxkhdr));
+
+ pad = sizeof(struct rxkad_level2_hdr) + data_size;
+ pad = RXKAD_ALIGN - pad;
+ pad &= RXKAD_ALIGN - 1;
+ if (pad)
+ skb_put_zero(skb, pad);
/* encrypt from the session key */
token = call->conn->params.key->payload.data[0];
memcpy(&iv, token->kad->session_key, sizeof(iv));
- sg_init_one(&sg[0], sechdr, sizeof(rxkhdr));
- skcipher_request_set_sync_tfm(req, call->conn->cipher);
+ sg_init_one(&sg[0], skb->head, sizeof(rxkhdr));
+ skcipher_request_set_sync_tfm(req, call->conn->rxkad.cipher);
skcipher_request_set_callback(req, 0, NULL, NULL);
skcipher_request_set_crypt(req, &sg[0], &sg[0], sizeof(rxkhdr), iv.x);
crypto_skcipher_encrypt(req);
if (skb_shinfo(skb)->nr_frags > 16)
goto out;
- len = data_size + call->conn->size_align - 1;
- len &= ~(call->conn->size_align - 1);
+ len = round_up(data_size, RXKAD_ALIGN);
sg_init_table(sg, ARRAY_SIZE(sg));
- err = skb_to_sgvec(skb, sg, 0, len);
+ err = skb_to_sgvec(skb, sg, 8, len);
if (unlikely(err < 0))
goto out;
skcipher_request_set_crypt(req, sg, sg, len, iv.x);
*/
static int rxkad_secure_packet(struct rxrpc_call *call,
struct sk_buff *skb,
- size_t data_size,
- void *sechdr)
+ size_t data_size)
{
struct rxrpc_skb_priv *sp;
struct skcipher_request *req;
call->debug_id, key_serial(call->conn->params.key),
sp->hdr.seq, data_size);
- if (!call->conn->cipher)
+ if (!call->conn->rxkad.cipher)
return 0;
ret = key_validate(call->conn->params.key);
return -ENOMEM;
/* continue encrypting from where we left off */
- memcpy(&iv, call->conn->csum_iv.x, sizeof(iv));
+ memcpy(&iv, call->conn->rxkad.csum_iv.x, sizeof(iv));
/* calculate the security checksum */
x = (call->cid & RXRPC_CHANNELMASK) << (32 - RXRPC_CIDSHIFT);
call->crypto_buf[1] = htonl(x);
sg_init_one(&sg, call->crypto_buf, 8);
- skcipher_request_set_sync_tfm(req, call->conn->cipher);
+ skcipher_request_set_sync_tfm(req, call->conn->rxkad.cipher);
skcipher_request_set_callback(req, 0, NULL, NULL);
skcipher_request_set_crypt(req, &sg, &sg, 8, iv.x);
crypto_skcipher_encrypt(req);
ret = 0;
break;
case RXRPC_SECURITY_AUTH:
- ret = rxkad_secure_packet_auth(call, skb, data_size, sechdr,
- req);
+ ret = rxkad_secure_packet_auth(call, skb, data_size, req);
break;
case RXRPC_SECURITY_ENCRYPT:
- ret = rxkad_secure_packet_encrypt(call, skb, data_size,
- sechdr, req);
+ ret = rxkad_secure_packet_encrypt(call, skb, data_size, req);
break;
default:
ret = -EPERM;
/* start the decryption afresh */
memset(&iv, 0, sizeof(iv));
- skcipher_request_set_sync_tfm(req, call->conn->cipher);
+ skcipher_request_set_sync_tfm(req, call->conn->rxkad.cipher);
skcipher_request_set_callback(req, 0, NULL, NULL);
skcipher_request_set_crypt(req, sg, sg, 8, iv.x);
crypto_skcipher_decrypt(req);
token = call->conn->params.key->payload.data[0];
memcpy(&iv, token->kad->session_key, sizeof(iv));
- skcipher_request_set_sync_tfm(req, call->conn->cipher);
+ skcipher_request_set_sync_tfm(req, call->conn->rxkad.cipher);
skcipher_request_set_callback(req, 0, NULL, NULL);
skcipher_request_set_crypt(req, sg, sg, len, iv.x);
crypto_skcipher_decrypt(req);
_enter("{%d{%x}},{#%u}",
call->debug_id, key_serial(call->conn->params.key), seq);
- if (!call->conn->cipher)
+ if (!call->conn->rxkad.cipher)
return 0;
req = rxkad_get_call_crypto(call);
return -ENOMEM;
/* continue encrypting from where we left off */
- memcpy(&iv, call->conn->csum_iv.x, sizeof(iv));
+ memcpy(&iv, call->conn->rxkad.csum_iv.x, sizeof(iv));
/* validate the security checksum */
x = (call->cid & RXRPC_CHANNELMASK) << (32 - RXRPC_CIDSHIFT);
call->crypto_buf[1] = htonl(x);
sg_init_one(&sg, call->crypto_buf, 8);
- skcipher_request_set_sync_tfm(req, call->conn->cipher);
+ skcipher_request_set_sync_tfm(req, call->conn->rxkad.cipher);
skcipher_request_set_callback(req, 0, NULL, NULL);
skcipher_request_set_crypt(req, &sg, &sg, 8, iv.x);
crypto_skcipher_encrypt(req);
u32 serial;
int ret;
- _enter("{%d,%x}", conn->debug_id, key_serial(conn->server_key));
+ _enter("{%d}", conn->debug_id);
- ret = key_validate(conn->server_key);
- if (ret < 0)
- return ret;
-
- get_random_bytes(&conn->security_nonce, sizeof(conn->security_nonce));
+ get_random_bytes(&conn->rxkad.nonce, sizeof(conn->rxkad.nonce));
challenge.version = htonl(2);
- challenge.nonce = htonl(conn->security_nonce);
+ challenge.nonce = htonl(conn->rxkad.nonce);
challenge.min_level = htonl(0);
challenge.__padding = 0;
struct rxrpc_crypt iv;
struct scatterlist sg[1];
- req = skcipher_request_alloc(&conn->cipher->base, GFP_NOFS);
+ req = skcipher_request_alloc(&conn->rxkad.cipher->base, GFP_NOFS);
if (!req)
return -ENOMEM;
sg_init_table(sg, 1);
sg_set_buf(sg, &resp->encrypted, sizeof(resp->encrypted));
- skcipher_request_set_sync_tfm(req, conn->cipher);
+ skcipher_request_set_sync_tfm(req, conn->rxkad.cipher);
skcipher_request_set_callback(req, 0, NULL, NULL);
skcipher_request_set_crypt(req, sg, sg, sizeof(resp->encrypted), iv.x);
crypto_skcipher_encrypt(req);
* decrypt the kerberos IV ticket in the response
*/
static int rxkad_decrypt_ticket(struct rxrpc_connection *conn,
+ struct key *server_key,
struct sk_buff *skb,
void *ticket, size_t ticket_len,
struct rxrpc_crypt *_session_key,
u32 abort_code;
u8 *p, *q, *name, *end;
- _enter("{%d},{%x}", conn->debug_id, key_serial(conn->server_key));
+ _enter("{%d},{%x}", conn->debug_id, key_serial(server_key));
*_expiry = 0;
- ret = key_validate(conn->server_key);
- if (ret < 0) {
- switch (ret) {
- case -EKEYEXPIRED:
- abort_code = RXKADEXPIRED;
- goto other_error;
- default:
- abort_code = RXKADNOAUTH;
- goto other_error;
- }
- }
-
- ASSERT(conn->server_key->payload.data[0] != NULL);
+ ASSERT(server_key->payload.data[0] != NULL);
ASSERTCMP((unsigned long) ticket & 7UL, ==, 0);
- memcpy(&iv, &conn->server_key->payload.data[2], sizeof(iv));
+ memcpy(&iv, &server_key->payload.data[2], sizeof(iv));
ret = -ENOMEM;
- req = skcipher_request_alloc(conn->server_key->payload.data[0],
- GFP_NOFS);
+ req = skcipher_request_alloc(server_key->payload.data[0], GFP_NOFS);
if (!req)
goto temporary_error;
struct rxkad_response *response;
struct rxrpc_skb_priv *sp = rxrpc_skb(skb);
struct rxrpc_crypt session_key;
+ struct key *server_key;
const char *eproto;
time64_t expiry;
void *ticket;
__be32 csum;
int ret, i;
- _enter("{%d,%x}", conn->debug_id, key_serial(conn->server_key));
+ _enter("{%d}", conn->debug_id);
+
+ server_key = rxrpc_look_up_server_security(conn, skb, 0, 0);
+ if (IS_ERR(server_key)) {
+ switch (PTR_ERR(server_key)) {
+ case -ENOKEY:
+ abort_code = RXKADUNKNOWNKEY;
+ break;
+ case -EKEYEXPIRED:
+ abort_code = RXKADEXPIRED;
+ break;
+ default:
+ abort_code = RXKADNOAUTH;
+ break;
+ }
+ trace_rxrpc_abort(0, "SVK",
+ sp->hdr.cid, sp->hdr.callNumber, sp->hdr.seq,
+ abort_code, PTR_ERR(server_key));
+ *_abort_code = abort_code;
+ return -EPROTO;
+ }
ret = -ENOMEM;
response = kzalloc(sizeof(struct rxkad_response), GFP_NOFS);
if (skb_copy_bits(skb, sizeof(struct rxrpc_wire_header),
response, sizeof(*response)) < 0)
goto protocol_error;
- if (!pskb_pull(skb, sizeof(*response)))
- BUG();
version = ntohl(response->version);
ticket_len = ntohl(response->ticket_len);
eproto = tracepoint_string("rxkad_tkt_short");
abort_code = RXKADPACKETSHORT;
- if (skb_copy_bits(skb, sizeof(struct rxrpc_wire_header),
+ if (skb_copy_bits(skb, sizeof(struct rxrpc_wire_header) + sizeof(*response),
ticket, ticket_len) < 0)
goto protocol_error_free;
- ret = rxkad_decrypt_ticket(conn, skb, ticket, ticket_len, &session_key,
- &expiry, _abort_code);
+ ret = rxkad_decrypt_ticket(conn, server_key, skb, ticket, ticket_len,
+ &session_key, &expiry, _abort_code);
if (ret < 0)
goto temporary_error_free_ticket;
eproto = tracepoint_string("rxkad_rsp_seq");
abort_code = RXKADOUTOFSEQUENCE;
- if (ntohl(response->encrypted.inc_nonce) != conn->security_nonce + 1)
+ if (ntohl(response->encrypted.inc_nonce) != conn->rxkad.nonce + 1)
goto protocol_error_free;
eproto = tracepoint_string("rxkad_rsp_level");
protocol_error:
kfree(response);
trace_rxrpc_rx_eproto(NULL, sp->hdr.serial, eproto);
+ key_put(server_key);
*_abort_code = abort_code;
return -EPROTO;
* ENOMEM. We just want to send the challenge again. Note that we
* also come out this way if the ticket decryption fails.
*/
+ key_put(server_key);
return ret;
}
{
_enter("");
- if (conn->cipher)
- crypto_free_sync_skcipher(conn->cipher);
+ if (conn->rxkad.cipher)
+ crypto_free_sync_skcipher(conn->rxkad.cipher);
}
/*
.no_key_abort = RXKADUNKNOWNKEY,
.init = rxkad_init,
.exit = rxkad_exit,
+ .preparse_server_key = rxkad_preparse_server_key,
+ .free_preparse_server_key = rxkad_free_preparse_server_key,
+ .destroy_server_key = rxkad_destroy_server_key,
.init_connection_security = rxkad_init_connection_security,
- .prime_packet_security = rxkad_prime_packet_security,
+ .how_much_data = rxkad_how_much_data,
.secure_packet = rxkad_secure_packet,
.verify_packet = rxkad_verify_packet,
.free_call_crypto = rxkad_free_call_crypto,
/*
* look up an rxrpc security module
*/
-static const struct rxrpc_security *rxrpc_security_lookup(u8 security_index)
+const struct rxrpc_security *rxrpc_security_lookup(u8 security_index)
{
if (security_index >= ARRAY_SIZE(rxrpc_security_types))
return NULL;
if (ret < 0)
return ret;
- token = key->payload.data[0];
- if (!token)
- return -EKEYREJECTED;
+ for (token = key->payload.data[0]; token; token = token->next) {
+ sec = rxrpc_security_lookup(token->security_index);
+ if (sec)
+ goto found;
+ }
+ return -EKEYREJECTED;
- sec = rxrpc_security_lookup(token->security_index);
- if (!sec)
- return -EKEYREJECTED;
+found:
conn->security = sec;
- ret = conn->security->init_connection_security(conn);
+ ret = conn->security->init_connection_security(conn, token);
if (ret < 0) {
conn->security = &rxrpc_no_security;
return ret;
}
/*
- * Find the security key for a server connection.
+ * Set the ops a server connection.
*/
-bool rxrpc_look_up_server_security(struct rxrpc_local *local, struct rxrpc_sock *rx,
- const struct rxrpc_security **_sec,
- struct key **_key,
- struct sk_buff *skb)
+const struct rxrpc_security *rxrpc_get_incoming_security(struct rxrpc_sock *rx,
+ struct sk_buff *skb)
{
const struct rxrpc_security *sec;
struct rxrpc_skb_priv *sp = rxrpc_skb(skb);
- key_ref_t kref = NULL;
- char kdesc[5 + 1 + 3 + 1];
_enter("");
- sprintf(kdesc, "%u:%u", sp->hdr.serviceId, sp->hdr.securityIndex);
-
sec = rxrpc_security_lookup(sp->hdr.securityIndex);
if (!sec) {
trace_rxrpc_abort(0, "SVS",
RX_INVALID_OPERATION, EKEYREJECTED);
skb->mark = RXRPC_SKB_MARK_REJECT_ABORT;
skb->priority = RX_INVALID_OPERATION;
- return false;
+ return NULL;
}
- if (sp->hdr.securityIndex == RXRPC_SECURITY_NONE)
- goto out;
-
- if (!rx->securities) {
+ if (sp->hdr.securityIndex != RXRPC_SECURITY_NONE &&
+ !rx->securities) {
trace_rxrpc_abort(0, "SVR",
sp->hdr.cid, sp->hdr.callNumber, sp->hdr.seq,
RX_INVALID_OPERATION, EKEYREJECTED);
skb->mark = RXRPC_SKB_MARK_REJECT_ABORT;
- skb->priority = RX_INVALID_OPERATION;
- return false;
+ skb->priority = sec->no_key_abort;
+ return NULL;
}
+ return sec;
+}
+
+/*
+ * Find the security key for a server connection.
+ */
+struct key *rxrpc_look_up_server_security(struct rxrpc_connection *conn,
+ struct sk_buff *skb,
+ u32 kvno, u32 enctype)
+{
+ struct rxrpc_skb_priv *sp = rxrpc_skb(skb);
+ struct rxrpc_sock *rx;
+ struct key *key = ERR_PTR(-EKEYREJECTED);
+ key_ref_t kref = NULL;
+ char kdesc[5 + 1 + 3 + 1 + 12 + 1 + 12 + 1];
+ int ret;
+
+ _enter("");
+
+ if (enctype)
+ sprintf(kdesc, "%u:%u:%u:%u",
+ sp->hdr.serviceId, sp->hdr.securityIndex, kvno, enctype);
+ else if (kvno)
+ sprintf(kdesc, "%u:%u:%u",
+ sp->hdr.serviceId, sp->hdr.securityIndex, kvno);
+ else
+ sprintf(kdesc, "%u:%u",
+ sp->hdr.serviceId, sp->hdr.securityIndex);
+
+ rcu_read_lock();
+
+ rx = rcu_dereference(conn->params.local->service);
+ if (!rx)
+ goto out;
+
/* look through the service's keyring */
kref = keyring_search(make_key_ref(rx->securities, 1UL),
&key_type_rxrpc_s, kdesc, true);
if (IS_ERR(kref)) {
- trace_rxrpc_abort(0, "SVK",
- sp->hdr.cid, sp->hdr.callNumber, sp->hdr.seq,
- sec->no_key_abort, EKEYREJECTED);
- skb->mark = RXRPC_SKB_MARK_REJECT_ABORT;
- skb->priority = sec->no_key_abort;
- return false;
+ key = ERR_CAST(kref);
+ goto out;
+ }
+
+ key = key_ref_to_ptr(kref);
+
+ ret = key_validate(key);
+ if (ret < 0) {
+ key_put(key);
+ key = ERR_PTR(ret);
+ goto out;
}
out:
- *_sec = sec;
- *_key = key_ref_to_ptr(kref);
- return true;
+ rcu_read_unlock();
+ return key;
}
rxrpc_send_ack_packet(call, false, NULL);
if (!skb) {
- size_t size, chunk, max, space;
+ size_t remain, bufsize, chunk, offset;
_debug("alloc");
goto maybe_error;
}
- max = RXRPC_JUMBO_DATALEN;
- max -= call->conn->security_size;
- max &= ~(call->conn->size_align - 1UL);
-
- chunk = max;
- if (chunk > msg_data_left(msg) && !more)
- chunk = msg_data_left(msg);
-
- space = chunk + call->conn->size_align;
- space &= ~(call->conn->size_align - 1UL);
-
- size = space + call->conn->security_size;
+ /* Work out the maximum size of a packet. Assume that
+ * the security header is going to be in the padded
+ * region (enc blocksize), but the trailer is not.
+ */
+ remain = more ? INT_MAX : msg_data_left(msg);
+ ret = call->conn->security->how_much_data(call, remain,
+ &bufsize, &chunk, &offset);
+ if (ret < 0)
+ goto maybe_error;
- _debug("SIZE: %zu/%zu/%zu", chunk, space, size);
+ _debug("SIZE: %zu/%zu @%zu", chunk, bufsize, offset);
/* create a buffer that we can retain until it's ACK'd */
skb = sock_alloc_send_skb(
- sk, size, msg->msg_flags & MSG_DONTWAIT, &ret);
+ sk, bufsize, msg->msg_flags & MSG_DONTWAIT, &ret);
if (!skb)
goto maybe_error;
ASSERTCMP(skb->mark, ==, 0);
- _debug("HS: %u", call->conn->security_size);
- skb_reserve(skb, call->conn->security_size);
- skb->len += call->conn->security_size;
+ __skb_put(skb, offset);
sp->remain = chunk;
if (sp->remain > skb_tailroom(skb))
(msg_data_left(msg) == 0 && !more)) {
struct rxrpc_connection *conn = call->conn;
uint32_t seq;
- size_t pad;
-
- /* pad out if we're using security */
- if (conn->security_ix) {
- pad = conn->security_size + skb->mark;
- pad = conn->size_align - pad;
- pad &= conn->size_align - 1;
- _debug("pad %zu", pad);
- if (pad)
- skb_put_zero(skb, pad);
- }
seq = call->tx_top + 1;
call->tx_winsize)
sp->hdr.flags |= RXRPC_MORE_PACKETS;
- ret = call->security->secure_packet(
- call, skb, skb->mark, skb->head);
+ ret = call->security->secure_packet(call, skb, skb->mark);
if (ret < 0)
goto out;
--- /dev/null
+// SPDX-License-Identifier: GPL-2.0-or-later
+/* RxRPC key management
+ *
+ * Copyright (C) 2007 Red Hat, Inc. All Rights Reserved.
+ * Written by David Howells (dhowells@redhat.com)
+ *
+ * RxRPC keys should have a description of describing their purpose:
+ * "afs@CAMBRIDGE.REDHAT.COM>
+ */
+
+#define pr_fmt(fmt) KBUILD_MODNAME ": " fmt
+
+#include <crypto/skcipher.h>
+#include <linux/module.h>
+#include <linux/net.h>
+#include <linux/skbuff.h>
+#include <linux/key-type.h>
+#include <linux/ctype.h>
+#include <linux/slab.h>
+#include <net/sock.h>
+#include <net/af_rxrpc.h>
+#include <keys/rxrpc-type.h>
+#include <keys/user-type.h>
+#include "ar-internal.h"
+
+static int rxrpc_vet_description_s(const char *);
+static int rxrpc_preparse_s(struct key_preparsed_payload *);
+static void rxrpc_free_preparse_s(struct key_preparsed_payload *);
+static void rxrpc_destroy_s(struct key *);
+static void rxrpc_describe_s(const struct key *, struct seq_file *);
+
+/*
+ * rxrpc server keys take "<serviceId>:<securityIndex>[:<sec-specific>]" as the
+ * description and the key material as the payload.
+ */
+struct key_type key_type_rxrpc_s = {
+ .name = "rxrpc_s",
+ .flags = KEY_TYPE_NET_DOMAIN,
+ .vet_description = rxrpc_vet_description_s,
+ .preparse = rxrpc_preparse_s,
+ .free_preparse = rxrpc_free_preparse_s,
+ .instantiate = generic_key_instantiate,
+ .destroy = rxrpc_destroy_s,
+ .describe = rxrpc_describe_s,
+};
+
+/*
+ * Vet the description for an RxRPC server key.
+ */
+static int rxrpc_vet_description_s(const char *desc)
+{
+ unsigned long service, sec_class;
+ char *p;
+
+ service = simple_strtoul(desc, &p, 10);
+ if (*p != ':' || service > 65535)
+ return -EINVAL;
+ sec_class = simple_strtoul(p + 1, &p, 10);
+ if ((*p && *p != ':') || sec_class < 1 || sec_class > 255)
+ return -EINVAL;
+ return 0;
+}
+
+/*
+ * Preparse a server secret key.
+ */
+static int rxrpc_preparse_s(struct key_preparsed_payload *prep)
+{
+ const struct rxrpc_security *sec;
+ unsigned int service, sec_class;
+ int n;
+
+ _enter("%zu", prep->datalen);
+
+ if (!prep->orig_description)
+ return -EINVAL;
+
+ if (sscanf(prep->orig_description, "%u:%u%n", &service, &sec_class, &n) != 2)
+ return -EINVAL;
+
+ sec = rxrpc_security_lookup(sec_class);
+ if (!sec)
+ return -ENOPKG;
+
+ prep->payload.data[1] = (struct rxrpc_security *)sec;
+
+ return sec->preparse_server_key(prep);
+}
+
+static void rxrpc_free_preparse_s(struct key_preparsed_payload *prep)
+{
+ const struct rxrpc_security *sec = prep->payload.data[1];
+
+ if (sec)
+ sec->free_preparse_server_key(prep);
+}
+
+static void rxrpc_destroy_s(struct key *key)
+{
+ const struct rxrpc_security *sec = key->payload.data[1];
+
+ if (sec)
+ sec->destroy_server_key(key);
+}
+
+static void rxrpc_describe_s(const struct key *key, struct seq_file *m)
+{
+ const struct rxrpc_security *sec = key->payload.data[1];
+
+ seq_puts(m, key->description);
+ if (sec && sec->describe_server_key)
+ sec->describe_server_key(key, m);
+}
+
+/*
+ * grab the security keyring for a server socket
+ */
+int rxrpc_server_keyring(struct rxrpc_sock *rx, sockptr_t optval, int optlen)
+{
+ struct key *key;
+ char *description;
+
+ _enter("");
+
+ if (optlen <= 0 || optlen > PAGE_SIZE - 1)
+ return -EINVAL;
+
+ description = memdup_sockptr_nul(optval, optlen);
+ if (IS_ERR(description))
+ return PTR_ERR(description);
+
+ key = request_key(&key_type_keyring, description, NULL);
+ if (IS_ERR(key)) {
+ kfree(description);
+ _leave(" = %ld", PTR_ERR(key));
+ return PTR_ERR(key);
+ }
+
+ rx->securities = key;
+ kfree(description);
+ _leave(" = 0 [key %x]", key->serial);
+ return 0;
+}
obj-y := sch_generic.o sch_mq.o
+obj-$(CONFIG_INET) += sch_frag.o
obj-$(CONFIG_NET_SCHED) += sch_api.o sch_blackhole.o
obj-$(CONFIG_NET_CLS) += cls_api.o
obj-$(CONFIG_NET_CLS_ACT) += act_api.o
#include <net/act_api.h>
#include <net/netlink.h>
+#ifdef CONFIG_INET
+DEFINE_STATIC_KEY_FALSE(tcf_frag_xmit_count);
+EXPORT_SYMBOL_GPL(tcf_frag_xmit_count);
+#endif
+
+int tcf_dev_queue_xmit(struct sk_buff *skb, int (*xmit)(struct sk_buff *skb))
+{
+#ifdef CONFIG_INET
+ if (static_branch_unlikely(&tcf_frag_xmit_count))
+ return sch_frag_xmit_hook(skb, xmit);
+#endif
+
+ return xmit(skb);
+}
+EXPORT_SYMBOL_GPL(tcf_dev_queue_xmit);
+
static void tcf_action_goto_chain_exec(const struct tc_action *a,
struct tcf_result *res)
{
index--;
goto nla_put_failure;
}
- err = (act_flags & TCA_FLAG_TERSE_DUMP) ?
+ err = (act_flags & TCA_ACT_FLAG_TERSE_DUMP) ?
tcf_action_dump_terse(skb, p, true) :
tcf_action_dump_1(skb, p, 0, 0);
if (err < 0) {
}
nla_nest_end(skb, nest);
n_i++;
- if (!(act_flags & TCA_FLAG_LARGE_DUMP_ON) &&
+ if (!(act_flags & TCA_ACT_FLAG_LARGE_DUMP_ON) &&
n_i >= TCA_ACT_MAX_PRIO)
goto done;
}
mutex_unlock(&idrinfo->lock);
if (n_i) {
- if (act_flags & TCA_FLAG_LARGE_DUMP_ON)
+ if (act_flags & TCA_ACT_FLAG_LARGE_DUMP_ON)
cb->args[1] = n_i;
}
return n_i;
NL_SET_ERR_MSG(extack, "TC action kind must be specified");
goto err_out;
}
- if (nla_strlcpy(act_name, kind, IFNAMSIZ) >= IFNAMSIZ) {
+ if (nla_strscpy(act_name, kind, IFNAMSIZ) < 0) {
NL_SET_ERR_MSG(extack, "TC action name too long");
goto err_out;
}
}
static const struct nla_policy tcaa_policy[TCA_ROOT_MAX + 1] = {
- [TCA_ROOT_FLAGS] = NLA_POLICY_BITFIELD32(TCA_FLAG_LARGE_DUMP_ON |
- TCA_FLAG_TERSE_DUMP),
+ [TCA_ROOT_FLAGS] = NLA_POLICY_BITFIELD32(TCA_ACT_FLAG_LARGE_DUMP_ON |
+ TCA_ACT_FLAG_TERSE_DUMP),
[TCA_ROOT_TIME_DELTA] = { .type = NLA_U32 },
};
goto err_insert;
ct_ft->nf_ft.type = &flowtable_ct;
- ct_ft->nf_ft.flags |= NF_FLOWTABLE_HW_OFFLOAD;
+ ct_ft->nf_ft.flags |= NF_FLOWTABLE_HW_OFFLOAD |
+ NF_FLOWTABLE_COUNTER;
err = nf_flow_table_init(&ct_ft->nf_ft);
if (err)
goto err_init;
flow_offload_refresh(nf_ft, flow);
nf_conntrack_get(&ct->ct_general);
nf_ct_set(skb, ct, ctinfo);
- nf_ct_acct_update(ct, dir, skb->len);
+ if (nf_ft->flags & NF_FLOWTABLE_COUNTER)
+ nf_ct_acct_update(ct, dir, skb->len);
return true;
}
if (err)
goto err_register;
+ static_branch_inc(&tcf_frag_xmit_count);
+
return 0;
err_register:
static void __exit ct_cleanup_module(void)
{
+ static_branch_dec(&tcf_frag_xmit_count);
tcf_unregister_action(&act_ct_ops, &ct_net_ops);
tcf_ct_flow_tables_uninit();
destroy_workqueue(act_ct_wq);
if (unlikely(!tname))
goto err1;
if (tb[TCA_IPT_TABLE] == NULL ||
- nla_strlcpy(tname, tb[TCA_IPT_TABLE], IFNAMSIZ) >= IFNAMSIZ)
+ nla_strscpy(tname, tb[TCA_IPT_TABLE], IFNAMSIZ) >= IFNAMSIZ)
strcpy(tname, "mangle");
t = kmemdup(td, td->u.target_size, GFP_KERNEL);
return err;
}
+static int tcf_mirred_forward(bool want_ingress, struct sk_buff *skb)
+{
+ int err;
+
+ if (!want_ingress)
+ err = tcf_dev_queue_xmit(skb, dev_queue_xmit);
+ else
+ err = netif_receive_skb(skb);
+
+ return err;
+}
+
static int tcf_mirred_act(struct sk_buff *skb, const struct tc_action *a,
struct tcf_result *res)
{
/* let's the caller reinsert the packet, if possible */
if (use_reinsert) {
res->ingress = want_ingress;
- if (skb_tc_reinsert(skb, res))
+ err = tcf_mirred_forward(res->ingress, skb);
+ if (err)
tcf_action_inc_overlimit_qstats(&m->common);
__this_cpu_dec(mirred_rec_level);
return TC_ACT_CONSUMED;
}
}
- if (!want_ingress)
- err = dev_queue_xmit(skb2);
- else
- err = netif_receive_skb(skb2);
-
+ err = tcf_mirred_forward(want_ingress, skb2);
if (err) {
out:
tcf_action_inc_overlimit_qstats(&m->common);
goto drop;
break;
case TCA_MPLS_ACT_MODIFY:
+ if (!pskb_may_pull(skb,
+ skb_network_offset(skb) + MPLS_HLEN))
+ goto drop;
new_lse = tcf_mpls_get_lse(mpls_hdr(skb), p, false);
if (skb_mpls_update_lse(skb, new_lse))
goto drop;
d->tcfd_defdata = kzalloc(SIMP_MAX_DATA, GFP_KERNEL);
if (unlikely(!d->tcfd_defdata))
return -ENOMEM;
- nla_strlcpy(d->tcfd_defdata, defdata, SIMP_MAX_DATA);
+ nla_strscpy(d->tcfd_defdata, defdata, SIMP_MAX_DATA);
return 0;
}
spin_lock_bh(&d->tcf_lock);
goto_ch = tcf_action_set_ctrlact(a, p->action, goto_ch);
memset(d->tcfd_defdata, 0, SIMP_MAX_DATA);
- nla_strlcpy(d->tcfd_defdata, defdata, SIMP_MAX_DATA);
+ nla_strscpy(d->tcfd_defdata, defdata, SIMP_MAX_DATA);
spin_unlock_bh(&d->tcf_lock);
if (goto_ch)
tcf_chain_put_by_act(goto_ch);
static bool tcf_proto_check_kind(struct nlattr *kind, char *name)
{
if (kind)
- return nla_strlcpy(name, kind, IFNAMSIZ) >= IFNAMSIZ;
+ return nla_strscpy(name, kind, IFNAMSIZ) < 0;
memset(name, 0, IFNAMSIZ);
return false;
}
*/
struct tcf_proto *
-tcf_get_next_proto(struct tcf_chain *chain, struct tcf_proto *tp,
- bool rtnl_held)
+tcf_get_next_proto(struct tcf_chain *chain, struct tcf_proto *tp)
{
struct tcf_proto *tp_next = __tcf_get_next_proto(chain, tp);
if (tp)
- tcf_proto_put(tp, rtnl_held, NULL);
+ tcf_proto_put(tp, true, NULL);
return tp_next;
}
static void tfilter_notify_chain(struct net *net, struct sk_buff *oskb,
struct tcf_block *block, struct Qdisc *q,
u32 parent, struct nlmsghdr *n,
- struct tcf_chain *chain, int event,
- bool rtnl_held)
+ struct tcf_chain *chain, int event)
{
struct tcf_proto *tp;
- for (tp = tcf_get_next_proto(chain, NULL, rtnl_held);
- tp; tp = tcf_get_next_proto(chain, tp, rtnl_held))
+ for (tp = tcf_get_next_proto(chain, NULL);
+ tp; tp = tcf_get_next_proto(chain, tp))
tfilter_notify(net, oskb, n, tp, block,
- q, parent, NULL, event, false, rtnl_held);
+ q, parent, NULL, event, false, true);
}
static void tfilter_put(struct tcf_proto *tp, void *fh)
if (prio == 0) {
tfilter_notify_chain(net, skb, block, q, parent, n,
- chain, RTM_DELTFILTER, rtnl_held);
+ chain, RTM_DELTFILTER);
tcf_chain_flush(chain, rtnl_held);
err = 0;
goto errout;
break;
case RTM_DELCHAIN:
tfilter_notify_chain(net, skb, block, q, parent, n,
- chain, RTM_DELTFILTER, true);
+ chain, RTM_DELTFILTER);
/* Flush the chain first as the user requested chain removal. */
tcf_chain_flush(chain, true);
/* In case the chain was successfully deleted, put a reference
#ifdef CONFIG_MODULES
if (ops == NULL && kind != NULL) {
char name[IFNAMSIZ];
- if (nla_strlcpy(name, kind, IFNAMSIZ) < IFNAMSIZ) {
+ if (nla_strscpy(name, kind, IFNAMSIZ) >= 0) {
/* We dropped the RTNL semaphore in order to
* perform the module load. So, even if we
* succeeded in loading the module we have to
chain = tcf_get_next_chain(block, chain)) {
struct tcf_proto *tp;
- for (tp = tcf_get_next_proto(chain, NULL, true);
- tp; tp = tcf_get_next_proto(chain, tp, true)) {
+ for (tp = tcf_get_next_proto(chain, NULL);
+ tp; tp = tcf_get_next_proto(chain, tp)) {
struct tcf_bind_args arg = {};
arg.w.fn = tcf_node_bind;
* locredit = max_frame_size * (sendslope / port_transmit_rate)
*/
+#include <linux/ethtool.h>
#include <linux/module.h>
#include <linux/types.h>
#include <linux/kernel.h>
--- /dev/null
+// SPDX-License-Identifier: GPL-2.0 OR Linux-OpenIB
+#include <net/netlink.h>
+#include <net/sch_generic.h>
+#include <net/dst.h>
+#include <net/ip.h>
+#include <net/ip6_fib.h>
+
+struct sch_frag_data {
+ unsigned long dst;
+ struct qdisc_skb_cb cb;
+ __be16 inner_protocol;
+ u16 vlan_tci;
+ __be16 vlan_proto;
+ unsigned int l2_len;
+ u8 l2_data[VLAN_ETH_HLEN];
+ int (*xmit)(struct sk_buff *skb);
+};
+
+static DEFINE_PER_CPU(struct sch_frag_data, sch_frag_data_storage);
+
+static int sch_frag_xmit(struct net *net, struct sock *sk, struct sk_buff *skb)
+{
+ struct sch_frag_data *data = this_cpu_ptr(&sch_frag_data_storage);
+
+ if (skb_cow_head(skb, data->l2_len) < 0) {
+ kfree_skb(skb);
+ return -ENOMEM;
+ }
+
+ __skb_dst_copy(skb, data->dst);
+ *qdisc_skb_cb(skb) = data->cb;
+ skb->inner_protocol = data->inner_protocol;
+ if (data->vlan_tci & VLAN_CFI_MASK)
+ __vlan_hwaccel_put_tag(skb, data->vlan_proto,
+ data->vlan_tci & ~VLAN_CFI_MASK);
+ else
+ __vlan_hwaccel_clear_tag(skb);
+
+ /* Reconstruct the MAC header. */
+ skb_push(skb, data->l2_len);
+ memcpy(skb->data, &data->l2_data, data->l2_len);
+ skb_postpush_rcsum(skb, skb->data, data->l2_len);
+ skb_reset_mac_header(skb);
+
+ return data->xmit(skb);
+}
+
+static void sch_frag_prepare_frag(struct sk_buff *skb,
+ int (*xmit)(struct sk_buff *skb))
+{
+ unsigned int hlen = skb_network_offset(skb);
+ struct sch_frag_data *data;
+
+ data = this_cpu_ptr(&sch_frag_data_storage);
+ data->dst = skb->_skb_refdst;
+ data->cb = *qdisc_skb_cb(skb);
+ data->xmit = xmit;
+ data->inner_protocol = skb->inner_protocol;
+ if (skb_vlan_tag_present(skb))
+ data->vlan_tci = skb_vlan_tag_get(skb) | VLAN_CFI_MASK;
+ else
+ data->vlan_tci = 0;
+ data->vlan_proto = skb->vlan_proto;
+ data->l2_len = hlen;
+ memcpy(&data->l2_data, skb->data, hlen);
+
+ memset(IPCB(skb), 0, sizeof(struct inet_skb_parm));
+ skb_pull(skb, hlen);
+}
+
+static unsigned int
+sch_frag_dst_get_mtu(const struct dst_entry *dst)
+{
+ return dst->dev->mtu;
+}
+
+static struct dst_ops sch_frag_dst_ops = {
+ .family = AF_UNSPEC,
+ .mtu = sch_frag_dst_get_mtu,
+};
+
+static int sch_fragment(struct net *net, struct sk_buff *skb,
+ u16 mru, int (*xmit)(struct sk_buff *skb))
+{
+ int ret = -1;
+
+ if (skb_network_offset(skb) > VLAN_ETH_HLEN) {
+ net_warn_ratelimited("L2 header too long to fragment\n");
+ goto err;
+ }
+
+ if (skb_protocol(skb, true) == htons(ETH_P_IP)) {
+ struct dst_entry sch_frag_dst;
+ unsigned long orig_dst;
+
+ sch_frag_prepare_frag(skb, xmit);
+ dst_init(&sch_frag_dst, &sch_frag_dst_ops, NULL, 1,
+ DST_OBSOLETE_NONE, DST_NOCOUNT);
+ sch_frag_dst.dev = skb->dev;
+
+ orig_dst = skb->_skb_refdst;
+ skb_dst_set_noref(skb, &sch_frag_dst);
+ IPCB(skb)->frag_max_size = mru;
+
+ ret = ip_do_fragment(net, skb->sk, skb, sch_frag_xmit);
+ refdst_drop(orig_dst);
+ } else if (skb_protocol(skb, true) == htons(ETH_P_IPV6)) {
+ unsigned long orig_dst;
+ struct rt6_info sch_frag_rt;
+
+ sch_frag_prepare_frag(skb, xmit);
+ memset(&sch_frag_rt, 0, sizeof(sch_frag_rt));
+ dst_init(&sch_frag_rt.dst, &sch_frag_dst_ops, NULL, 1,
+ DST_OBSOLETE_NONE, DST_NOCOUNT);
+ sch_frag_rt.dst.dev = skb->dev;
+
+ orig_dst = skb->_skb_refdst;
+ skb_dst_set_noref(skb, &sch_frag_rt.dst);
+ IP6CB(skb)->frag_max_size = mru;
+
+ ret = ipv6_stub->ipv6_fragment(net, skb->sk, skb,
+ sch_frag_xmit);
+ refdst_drop(orig_dst);
+ } else {
+ net_warn_ratelimited("Fail frag %s: eth=%x, MRU=%d, MTU=%d\n",
+ netdev_name(skb->dev),
+ ntohs(skb_protocol(skb, true)), mru,
+ skb->dev->mtu);
+ goto err;
+ }
+
+ return ret;
+err:
+ kfree_skb(skb);
+ return ret;
+}
+
+int sch_frag_xmit_hook(struct sk_buff *skb, int (*xmit)(struct sk_buff *skb))
+{
+ u16 mru = qdisc_skb_cb(skb)->mru;
+ int err;
+
+ if (mru && skb->len > mru + skb->dev->hard_header_len)
+ err = sch_fragment(dev_net(skb->dev), skb, mru, xmit);
+ else
+ err = xmit(skb);
+
+ return err;
+}
+EXPORT_SYMBOL_GPL(sch_frag_xmit_hook);
*
*/
+#include <linux/ethtool.h>
#include <linux/types.h>
#include <linux/slab.h>
#include <linux/kernel.h>
else {
if (!mod_timer(&t->proto_unreach_timer,
jiffies + (HZ/20)))
- sctp_association_hold(asoc);
+ sctp_transport_hold(t);
}
} else {
struct net *net = sock_net(sk);
"encountered!\n", __func__);
if (del_timer(&t->proto_unreach_timer))
- sctp_association_put(asoc);
+ sctp_transport_put(t);
sctp_do_sm(net, SCTP_EVENT_T_OTHER,
SCTP_ST_OTHER(SCTP_EVENT_ICMP_PROTO_UNREACH),
/* Try again later. */
if (!mod_timer(&transport->proto_unreach_timer,
jiffies + (HZ/20)))
- sctp_association_hold(asoc);
+ sctp_transport_hold(transport);
goto out_unlock;
}
out_unlock:
bh_unlock_sock(sk);
- sctp_association_put(asoc);
+ sctp_transport_put(transport);
}
/* Handle the timeout of the RE-CONFIG timer. */
*
* This file is part of the SCTP kernel implementation
*
- * This module provides the abstraction for an SCTP tranport representing
+ * This module provides the abstraction for an SCTP transport representing
* a remote transport address. For local transport addresses, we just use
* union sctp_addr.
*
/* Delete the T3_rtx timer if it's active.
* There is no point in not doing this now and letting
* structure hang around in memory since we know
- * the tranport is going away.
+ * the transport is going away.
*/
if (del_timer(&transport->T3_rtx_timer))
sctp_transport_put(transport);
/* Delete the ICMP proto unreachable timer if it's active. */
if (del_timer(&transport->proto_unreach_timer))
- sctp_association_put(transport->asoc);
+ sctp_transport_put(transport);
sctp_transport_put(transport);
}
obj-$(CONFIG_SMC) += smc.o
obj-$(CONFIG_SMC_DIAG) += smc_diag.o
smc-y := af_smc.o smc_pnet.o smc_ib.o smc_clc.o smc_core.o smc_wr.o smc_llc.o
-smc-y += smc_cdc.o smc_tx.o smc_rx.o smc_close.o smc_ism.o
+smc-y += smc_cdc.o smc_tx.o smc_rx.o smc_close.o smc_ism.o smc_netlink.o
#include "smc_ib.h"
#include "smc_ism.h"
#include "smc_pnet.h"
+#include "smc_netlink.h"
#include "smc_tx.h"
#include "smc_rx.h"
#include "smc_close.h"
return smc_connect_fallback(smc, reason_code);
}
-/* abort connecting */
-static void smc_connect_abort(struct smc_sock *smc, int local_first)
+static void smc_conn_abort(struct smc_sock *smc, int local_first)
{
if (local_first)
smc_lgr_cleanup_early(&smc->conn);
ini->smc_type_v1 = SMC_TYPE_N;
} /* else RDMA is supported for this connection */
}
- if (smc_ism_v2_capable && smc_find_ism_v2_device_clnt(smc, ini))
+ if (smc_ism_is_v2_capable() && smc_find_ism_v2_device_clnt(smc, ini))
ini->smc_type_v2 = SMC_TYPE_N;
/* if neither ISM nor RDMA are supported, fallback */
return 0;
connect_abort:
- smc_connect_abort(smc, ini->first_contact_local);
+ smc_conn_abort(smc, ini->first_contact_local);
mutex_unlock(&smc_client_lgr_pending);
smc->connect_nonblock = 0;
return 0;
connect_abort:
- smc_connect_abort(smc, ini->first_contact_local);
+ smc_conn_abort(smc, ini->first_contact_local);
mutex_unlock(&smc_server_lgr_pending);
smc->connect_nonblock = 0;
/* perform steps before actually connecting */
static int __smc_connect(struct smc_sock *smc)
{
- u8 version = smc_ism_v2_capable ? SMC_V2 : SMC_V1;
+ u8 version = smc_ism_is_v2_capable() ? SMC_V2 : SMC_V1;
struct smc_clc_msg_accept_confirm_v2 *aclc2;
struct smc_clc_msg_accept_confirm *aclc;
struct smc_init_info *ini = NULL;
version);
ini->smcd_version = SMC_V1;
- ini->smcd_version |= smc_ism_v2_capable ? SMC_V2 : 0;
+ ini->smcd_version |= smc_ism_is_v2_capable() ? SMC_V2 : 0;
ini->smc_type_v1 = SMC_TYPE_B;
- ini->smc_type_v2 = smc_ism_v2_capable ? SMC_TYPE_D : SMC_TYPE_N;
+ ini->smc_type_v2 = smc_ism_is_v2_capable() ? SMC_TYPE_D : SMC_TYPE_N;
/* get vlan id from IP device */
if (smc_vlan_by_tcpsk(smc->clcsock, ini)) {
/* check if smc modes and versions of CLC proposal and accept match */
rc = smc_connect_check_aclc(ini, aclc);
- version = aclc->hdr.version == SMC_V1 ? SMC_V1 : version;
+ version = aclc->hdr.version == SMC_V1 ? SMC_V1 : SMC_V2;
+ ini->smcd_version = version;
if (rc)
goto vlan_cleanup;
int local_first, u8 version)
{
/* RDMA setup failed, switch back to TCP */
- if (local_first)
- smc_lgr_cleanup_early(&new_smc->conn);
- else
- smc_conn_free(&new_smc->conn);
+ smc_conn_abort(new_smc, local_first);
if (reason_code < 0) { /* error, no fallback possible */
smc_listen_out_err(new_smc);
return;
rc = SMC_CLC_DECL_PEERNOSMC;
goto out;
}
- if (!smc_ism_v2_capable) {
+ if (!smc_ism_is_v2_capable()) {
ini->smcd_version &= ~SMC_V2;
rc = SMC_CLC_DECL_NOISM2SUPP;
goto out;
/* Create send and receive buffers */
rc = smc_buf_create(new_smc, true);
if (rc) {
- if (ini->first_contact_local)
- smc_lgr_cleanup_early(&new_smc->conn);
- else
- smc_conn_free(&new_smc->conn);
+ smc_conn_abort(new_smc, ini->first_contact_local);
return (rc == -ENOSPC) ? SMC_CLC_DECL_MAX_DMB :
SMC_CLC_DECL_MEM;
}
{
struct smc_sock *new_smc = container_of(work, struct smc_sock,
smc_listen_work);
- u8 version = smc_ism_v2_capable ? SMC_V2 : SMC_V1;
+ u8 version = smc_ism_is_v2_capable() ? SMC_V2 : SMC_V1;
struct socket *newclcsock = new_smc->clcsock;
struct smc_clc_msg_accept_confirm *cclc;
struct smc_clc_msg_proposal_area *buf;
smc_ism_init();
smc_clc_init();
- rc = smc_pnet_init();
+ rc = smc_nl_init();
if (rc)
goto out_pernet_subsys;
+ rc = smc_pnet_init();
+ if (rc)
+ goto out_nl;
+
rc = -ENOMEM;
smc_hs_wq = alloc_workqueue("smc_hs_wq", 0, 0);
if (!smc_hs_wq)
destroy_workqueue(smc_hs_wq);
out_pnet:
smc_pnet_exit();
+out_nl:
+ smc_nl_exit();
out_pernet_subsys:
unregister_pernet_subsys(&smc_net_ops);
proto_unregister(&smc_proto6);
proto_unregister(&smc_proto);
smc_pnet_exit();
+ smc_nl_exit();
unregister_pernet_subsys(&smc_net_ops);
rcu_barrier();
}
return len > 0 ? 0 : len;
}
+void smc_clc_get_hostname(u8 **host)
+{
+ *host = &smc_hostname[0];
+}
+
void __init smc_clc_init(void)
{
struct new_utsname *u;
int smc_clc_send_accept(struct smc_sock *smc, bool srv_first_contact,
u8 version);
void smc_clc_init(void) __init;
+void smc_clc_get_hostname(u8 **host);
#endif
#include <linux/wait.h>
#include <linux/reboot.h>
#include <linux/mutex.h>
+#include <linux/list.h>
+#include <linux/smc.h>
#include <net/tcp.h>
#include <net/sock.h>
#include <rdma/ib_verbs.h>
#include "smc_cdc.h"
#include "smc_close.h"
#include "smc_ism.h"
+#include "smc_netlink.h"
#define SMC_LGR_NUM_INCR 256
#define SMC_LGR_FREE_DELAY_SERV (600 * HZ)
#define SMC_LGR_FREE_DELAY_CLNT (SMC_LGR_FREE_DELAY_SERV + 10 * HZ)
-static struct smc_lgr_list smc_lgr_list = { /* established link groups */
+struct smc_lgr_list smc_lgr_list = { /* established link groups */
.lock = __SPIN_LOCK_UNLOCKED(smc_lgr_list.lock),
.list = LIST_HEAD_INIT(smc_lgr_list.list),
.num = 0,
return &smc_lgr_list.list;
}
+static void smc_ibdev_cnt_inc(struct smc_link *lnk)
+{
+ atomic_inc(&lnk->smcibdev->lnk_cnt_by_port[lnk->ibport - 1]);
+}
+
+static void smc_ibdev_cnt_dec(struct smc_link *lnk)
+{
+ atomic_dec(&lnk->smcibdev->lnk_cnt_by_port[lnk->ibport - 1]);
+}
+
static void smc_lgr_schedule_free_work(struct smc_link_group *lgr)
{
/* client link group creation always follows the server link group
}
if (!conn->lnk)
return SMC_CLC_DECL_NOACTLINK;
+ atomic_inc(&conn->lnk->conn_cnt);
return 0;
}
struct smc_link_group *lgr = conn->lgr;
rb_erase(&conn->alert_node, &lgr->conns_all);
+ if (conn->lnk)
+ atomic_dec(&conn->lnk->conn_cnt);
lgr->conns_num--;
conn->alert_token_local = 0;
sock_put(&smc->sk); /* sock_hold in smc_lgr_register_conn() */
conn->lgr = NULL;
}
+int smc_nl_get_sys_info(struct sk_buff *skb, struct netlink_callback *cb)
+{
+ struct smc_nl_dmp_ctx *cb_ctx = smc_nl_dmp_ctx(cb);
+ char hostname[SMC_MAX_HOSTNAME_LEN + 1];
+ char smc_seid[SMC_MAX_EID_LEN + 1];
+ struct smcd_dev *smcd_dev;
+ struct nlattr *attrs;
+ u8 *seid = NULL;
+ u8 *host = NULL;
+ void *nlh;
+
+ nlh = genlmsg_put(skb, NETLINK_CB(cb->skb).portid, cb->nlh->nlmsg_seq,
+ &smc_gen_nl_family, NLM_F_MULTI,
+ SMC_NETLINK_GET_SYS_INFO);
+ if (!nlh)
+ goto errmsg;
+ if (cb_ctx->pos[0])
+ goto errout;
+ attrs = nla_nest_start(skb, SMC_GEN_SYS_INFO);
+ if (!attrs)
+ goto errout;
+ if (nla_put_u8(skb, SMC_NLA_SYS_VER, SMC_V2))
+ goto errattr;
+ if (nla_put_u8(skb, SMC_NLA_SYS_REL, SMC_RELEASE))
+ goto errattr;
+ if (nla_put_u8(skb, SMC_NLA_SYS_IS_ISM_V2, smc_ism_is_v2_capable()))
+ goto errattr;
+ smc_clc_get_hostname(&host);
+ if (host) {
+ snprintf(hostname, sizeof(hostname), "%s", host);
+ if (nla_put_string(skb, SMC_NLA_SYS_LOCAL_HOST, hostname))
+ goto errattr;
+ }
+ mutex_lock(&smcd_dev_list.mutex);
+ smcd_dev = list_first_entry_or_null(&smcd_dev_list.list,
+ struct smcd_dev, list);
+ if (smcd_dev)
+ smc_ism_get_system_eid(smcd_dev, &seid);
+ mutex_unlock(&smcd_dev_list.mutex);
+ if (seid && smc_ism_is_v2_capable()) {
+ snprintf(smc_seid, sizeof(smc_seid), "%s", seid);
+ if (nla_put_string(skb, SMC_NLA_SYS_SEID, smc_seid))
+ goto errattr;
+ }
+ nla_nest_end(skb, attrs);
+ genlmsg_end(skb, nlh);
+ cb_ctx->pos[0] = 1;
+ return skb->len;
+
+errattr:
+ nla_nest_cancel(skb, attrs);
+errout:
+ genlmsg_cancel(skb, nlh);
+errmsg:
+ return skb->len;
+}
+
+static int smc_nl_fill_lgr(struct smc_link_group *lgr,
+ struct sk_buff *skb,
+ struct netlink_callback *cb)
+{
+ char smc_target[SMC_MAX_PNETID_LEN + 1];
+ struct nlattr *attrs;
+
+ attrs = nla_nest_start(skb, SMC_GEN_LGR_SMCR);
+ if (!attrs)
+ goto errout;
+
+ if (nla_put_u32(skb, SMC_NLA_LGR_R_ID, *((u32 *)&lgr->id)))
+ goto errattr;
+ if (nla_put_u32(skb, SMC_NLA_LGR_R_CONNS_NUM, lgr->conns_num))
+ goto errattr;
+ if (nla_put_u8(skb, SMC_NLA_LGR_R_ROLE, lgr->role))
+ goto errattr;
+ if (nla_put_u8(skb, SMC_NLA_LGR_R_TYPE, lgr->type))
+ goto errattr;
+ if (nla_put_u8(skb, SMC_NLA_LGR_R_VLAN_ID, lgr->vlan_id))
+ goto errattr;
+ snprintf(smc_target, sizeof(smc_target), "%s", lgr->pnet_id);
+ if (nla_put_string(skb, SMC_NLA_LGR_R_PNETID, smc_target))
+ goto errattr;
+
+ nla_nest_end(skb, attrs);
+ return 0;
+errattr:
+ nla_nest_cancel(skb, attrs);
+errout:
+ return -EMSGSIZE;
+}
+
+static int smc_nl_fill_lgr_link(struct smc_link_group *lgr,
+ struct smc_link *link,
+ struct sk_buff *skb,
+ struct netlink_callback *cb)
+{
+ char smc_ibname[IB_DEVICE_NAME_MAX + 1];
+ u8 smc_gid_target[41];
+ struct nlattr *attrs;
+ u32 link_uid = 0;
+ void *nlh;
+
+ nlh = genlmsg_put(skb, NETLINK_CB(cb->skb).portid, cb->nlh->nlmsg_seq,
+ &smc_gen_nl_family, NLM_F_MULTI,
+ SMC_NETLINK_GET_LINK_SMCR);
+ if (!nlh)
+ goto errmsg;
+
+ attrs = nla_nest_start(skb, SMC_GEN_LINK_SMCR);
+ if (!attrs)
+ goto errout;
+
+ if (nla_put_u8(skb, SMC_NLA_LINK_ID, link->link_id))
+ goto errattr;
+ if (nla_put_u32(skb, SMC_NLA_LINK_STATE, link->state))
+ goto errattr;
+ if (nla_put_u32(skb, SMC_NLA_LINK_CONN_CNT,
+ atomic_read(&link->conn_cnt)))
+ goto errattr;
+ if (nla_put_u8(skb, SMC_NLA_LINK_IB_PORT, link->ibport))
+ goto errattr;
+ if (nla_put_u32(skb, SMC_NLA_LINK_NET_DEV, link->ndev_ifidx))
+ goto errattr;
+ snprintf(smc_ibname, sizeof(smc_ibname), "%s", link->ibname);
+ if (nla_put_string(skb, SMC_NLA_LINK_IB_DEV, smc_ibname))
+ goto errattr;
+ memcpy(&link_uid, link->link_uid, sizeof(link_uid));
+ if (nla_put_u32(skb, SMC_NLA_LINK_UID, link_uid))
+ goto errattr;
+ memcpy(&link_uid, link->peer_link_uid, sizeof(link_uid));
+ if (nla_put_u32(skb, SMC_NLA_LINK_PEER_UID, link_uid))
+ goto errattr;
+ memset(smc_gid_target, 0, sizeof(smc_gid_target));
+ smc_gid_be16_convert(smc_gid_target, link->gid);
+ if (nla_put_string(skb, SMC_NLA_LINK_GID, smc_gid_target))
+ goto errattr;
+ memset(smc_gid_target, 0, sizeof(smc_gid_target));
+ smc_gid_be16_convert(smc_gid_target, link->peer_gid);
+ if (nla_put_string(skb, SMC_NLA_LINK_PEER_GID, smc_gid_target))
+ goto errattr;
+
+ nla_nest_end(skb, attrs);
+ genlmsg_end(skb, nlh);
+ return 0;
+errattr:
+ nla_nest_cancel(skb, attrs);
+errout:
+ genlmsg_cancel(skb, nlh);
+errmsg:
+ return -EMSGSIZE;
+}
+
+static int smc_nl_handle_lgr(struct smc_link_group *lgr,
+ struct sk_buff *skb,
+ struct netlink_callback *cb,
+ bool list_links)
+{
+ void *nlh;
+ int i;
+
+ nlh = genlmsg_put(skb, NETLINK_CB(cb->skb).portid, cb->nlh->nlmsg_seq,
+ &smc_gen_nl_family, NLM_F_MULTI,
+ SMC_NETLINK_GET_LGR_SMCR);
+ if (!nlh)
+ goto errmsg;
+ if (smc_nl_fill_lgr(lgr, skb, cb))
+ goto errout;
+
+ genlmsg_end(skb, nlh);
+ if (!list_links)
+ goto out;
+ for (i = 0; i < SMC_LINKS_PER_LGR_MAX; i++) {
+ if (!smc_link_usable(&lgr->lnk[i]))
+ continue;
+ if (smc_nl_fill_lgr_link(lgr, &lgr->lnk[i], skb, cb))
+ goto errout;
+ }
+out:
+ return 0;
+
+errout:
+ genlmsg_cancel(skb, nlh);
+errmsg:
+ return -EMSGSIZE;
+}
+
+static void smc_nl_fill_lgr_list(struct smc_lgr_list *smc_lgr,
+ struct sk_buff *skb,
+ struct netlink_callback *cb,
+ bool list_links)
+{
+ struct smc_nl_dmp_ctx *cb_ctx = smc_nl_dmp_ctx(cb);
+ struct smc_link_group *lgr;
+ int snum = cb_ctx->pos[0];
+ int num = 0;
+
+ spin_lock_bh(&smc_lgr->lock);
+ list_for_each_entry(lgr, &smc_lgr->list, list) {
+ if (num < snum)
+ goto next;
+ if (smc_nl_handle_lgr(lgr, skb, cb, list_links))
+ goto errout;
+next:
+ num++;
+ }
+errout:
+ spin_unlock_bh(&smc_lgr->lock);
+ cb_ctx->pos[0] = num;
+}
+
+static int smc_nl_fill_smcd_lgr(struct smc_link_group *lgr,
+ struct sk_buff *skb,
+ struct netlink_callback *cb)
+{
+ char smc_host[SMC_MAX_HOSTNAME_LEN + 1];
+ char smc_pnet[SMC_MAX_PNETID_LEN + 1];
+ char smc_eid[SMC_MAX_EID_LEN + 1];
+ struct nlattr *v2_attrs;
+ struct nlattr *attrs;
+ void *nlh;
+
+ nlh = genlmsg_put(skb, NETLINK_CB(cb->skb).portid, cb->nlh->nlmsg_seq,
+ &smc_gen_nl_family, NLM_F_MULTI,
+ SMC_NETLINK_GET_LGR_SMCD);
+ if (!nlh)
+ goto errmsg;
+
+ attrs = nla_nest_start(skb, SMC_GEN_LGR_SMCD);
+ if (!attrs)
+ goto errout;
+
+ if (nla_put_u32(skb, SMC_NLA_LGR_D_ID, *((u32 *)&lgr->id)))
+ goto errattr;
+ if (nla_put_u64_64bit(skb, SMC_NLA_LGR_D_GID, lgr->smcd->local_gid,
+ SMC_NLA_LGR_D_PAD))
+ goto errattr;
+ if (nla_put_u64_64bit(skb, SMC_NLA_LGR_D_PEER_GID, lgr->peer_gid,
+ SMC_NLA_LGR_D_PAD))
+ goto errattr;
+ if (nla_put_u8(skb, SMC_NLA_LGR_D_VLAN_ID, lgr->vlan_id))
+ goto errattr;
+ if (nla_put_u32(skb, SMC_NLA_LGR_D_CONNS_NUM, lgr->conns_num))
+ goto errattr;
+ if (nla_put_u32(skb, SMC_NLA_LGR_D_CHID, smc_ism_get_chid(lgr->smcd)))
+ goto errattr;
+ snprintf(smc_pnet, sizeof(smc_pnet), "%s", lgr->smcd->pnetid);
+ if (nla_put_string(skb, SMC_NLA_LGR_D_PNETID, smc_pnet))
+ goto errattr;
+
+ v2_attrs = nla_nest_start(skb, SMC_NLA_LGR_V2);
+ if (!v2_attrs)
+ goto errattr;
+ if (nla_put_u8(skb, SMC_NLA_LGR_V2_VER, lgr->smc_version))
+ goto errv2attr;
+ if (nla_put_u8(skb, SMC_NLA_LGR_V2_REL, lgr->peer_smc_release))
+ goto errv2attr;
+ if (nla_put_u8(skb, SMC_NLA_LGR_V2_OS, lgr->peer_os))
+ goto errv2attr;
+ snprintf(smc_host, sizeof(smc_host), "%s", lgr->peer_hostname);
+ if (nla_put_string(skb, SMC_NLA_LGR_V2_PEER_HOST, smc_host))
+ goto errv2attr;
+ snprintf(smc_eid, sizeof(smc_eid), "%s", lgr->negotiated_eid);
+ if (nla_put_string(skb, SMC_NLA_LGR_V2_NEG_EID, smc_eid))
+ goto errv2attr;
+
+ nla_nest_end(skb, v2_attrs);
+ nla_nest_end(skb, attrs);
+ genlmsg_end(skb, nlh);
+ return 0;
+
+errv2attr:
+ nla_nest_cancel(skb, v2_attrs);
+errattr:
+ nla_nest_cancel(skb, attrs);
+errout:
+ genlmsg_cancel(skb, nlh);
+errmsg:
+ return -EMSGSIZE;
+}
+
+static int smc_nl_handle_smcd_lgr(struct smcd_dev *dev,
+ struct sk_buff *skb,
+ struct netlink_callback *cb)
+{
+ struct smc_nl_dmp_ctx *cb_ctx = smc_nl_dmp_ctx(cb);
+ struct smc_link_group *lgr;
+ int snum = cb_ctx->pos[1];
+ int rc = 0, num = 0;
+
+ spin_lock_bh(&dev->lgr_lock);
+ list_for_each_entry(lgr, &dev->lgr_list, list) {
+ if (!lgr->is_smcd)
+ continue;
+ if (num < snum)
+ goto next;
+ rc = smc_nl_fill_smcd_lgr(lgr, skb, cb);
+ if (rc)
+ goto errout;
+next:
+ num++;
+ }
+errout:
+ spin_unlock_bh(&dev->lgr_lock);
+ cb_ctx->pos[1] = num;
+ return rc;
+}
+
+static int smc_nl_fill_smcd_dev(struct smcd_dev_list *dev_list,
+ struct sk_buff *skb,
+ struct netlink_callback *cb)
+{
+ struct smc_nl_dmp_ctx *cb_ctx = smc_nl_dmp_ctx(cb);
+ struct smcd_dev *smcd_dev;
+ int snum = cb_ctx->pos[0];
+ int rc = 0, num = 0;
+
+ mutex_lock(&dev_list->mutex);
+ list_for_each_entry(smcd_dev, &dev_list->list, list) {
+ if (list_empty(&smcd_dev->lgr_list))
+ continue;
+ if (num < snum)
+ goto next;
+ rc = smc_nl_handle_smcd_lgr(smcd_dev, skb, cb);
+ if (rc)
+ goto errout;
+next:
+ num++;
+ }
+errout:
+ mutex_unlock(&dev_list->mutex);
+ cb_ctx->pos[0] = num;
+ return rc;
+}
+
+int smcr_nl_get_lgr(struct sk_buff *skb, struct netlink_callback *cb)
+{
+ bool list_links = false;
+
+ smc_nl_fill_lgr_list(&smc_lgr_list, skb, cb, list_links);
+ return skb->len;
+}
+
+int smcr_nl_get_link(struct sk_buff *skb, struct netlink_callback *cb)
+{
+ bool list_links = true;
+
+ smc_nl_fill_lgr_list(&smc_lgr_list, skb, cb, list_links);
+ return skb->len;
+}
+
+int smcd_nl_get_lgr(struct sk_buff *skb, struct netlink_callback *cb)
+{
+ smc_nl_fill_smcd_dev(&smcd_dev_list, skb, cb);
+ return skb->len;
+}
+
void smc_lgr_cleanup_early(struct smc_connection *conn)
{
struct smc_link_group *lgr = conn->lgr;
return link_id;
}
+static void smcr_copy_dev_info_to_link(struct smc_link *link)
+{
+ struct smc_ib_device *smcibdev = link->smcibdev;
+
+ snprintf(link->ibname, sizeof(link->ibname), "%s",
+ smcibdev->ibdev->name);
+ link->ndev_ifidx = smcibdev->ndev_ifidx[link->ibport - 1];
+}
+
int smcr_link_init(struct smc_link_group *lgr, struct smc_link *lnk,
u8 link_idx, struct smc_init_info *ini)
{
lnk->link_idx = link_idx;
lnk->smcibdev = ini->ib_dev;
lnk->ibport = ini->ib_port;
+ smc_ibdev_cnt_inc(lnk);
+ smcr_copy_dev_info_to_link(lnk);
lnk->path_mtu = ini->ib_dev->pattr[ini->ib_port - 1].active_mtu;
+ atomic_set(&lnk->conn_cnt, 0);
smc_llc_link_set_uid(lnk);
INIT_WORK(&lnk->link_down_wrk, smc_link_down_work);
if (!ini->ib_dev->initialized) {
clear_llc_lnk:
smc_llc_link_clear(lnk, false);
out:
+ smc_ibdev_cnt_dec(lnk);
put_device(&ini->ib_dev->ibdev->dev);
memset(lnk, 0, sizeof(struct smc_link));
lnk->state = SMC_LNK_UNUSED;
return rc;
}
+static void smc_switch_link_and_count(struct smc_connection *conn,
+ struct smc_link *to_lnk)
+{
+ atomic_dec(&conn->lnk->conn_cnt);
+ conn->lnk = to_lnk;
+ atomic_inc(&conn->lnk->conn_cnt);
+}
+
struct smc_link *smc_switch_conns(struct smc_link_group *lgr,
struct smc_link *from_lnk, bool is_dev_err)
{
smc->sk.sk_state == SMC_PEERABORTWAIT ||
smc->sk.sk_state == SMC_PROCESSABORT) {
spin_lock_bh(&conn->send_lock);
- conn->lnk = to_lnk;
+ smc_switch_link_and_count(conn, to_lnk);
spin_unlock_bh(&conn->send_lock);
continue;
}
}
/* avoid race with smcr_tx_sndbuf_nonempty() */
spin_lock_bh(&conn->send_lock);
- conn->lnk = to_lnk;
+ smc_switch_link_and_count(conn, to_lnk);
rc = smc_switch_cursor(smc, pend, wr_buf);
spin_unlock_bh(&conn->send_lock);
sock_put(&smc->sk);
smc_ib_destroy_queue_pair(lnk);
smc_ib_dealloc_protection_domain(lnk);
smc_wr_free_link_mem(lnk);
+ smc_ibdev_cnt_dec(lnk);
put_device(&lnk->smcibdev->ibdev->dev);
smcibdev = lnk->smcibdev;
memset(lnk, 0, sizeof(struct smc_link));
ini->ism_peer_gid[ini->ism_selected]) :
smcr_lgr_match(lgr, ini->ib_lcl, role, ini->ib_clcqpn)) &&
!lgr->sync_err &&
- lgr->vlan_id == ini->vlan_id &&
+ (ini->smcd_version == SMC_V2 ||
+ lgr->vlan_id == ini->vlan_id) &&
(role == SMC_CLNT || ini->is_smcd ||
lgr->conns_num < SMC_RMBS_PER_LGR_MAX)) {
/* link group found */
#define _SMC_CORE_H
#include <linux/atomic.h>
+#include <linux/smc.h>
+#include <linux/pci.h>
#include <rdma/ib_verbs.h>
+#include <net/genetlink.h>
#include "smc.h"
#include "smc_ib.h"
u8 link_is_asym; /* is link asymmetric? */
struct smc_link_group *lgr; /* parent link group */
struct work_struct link_down_wrk; /* wrk to bring link down */
+ char ibname[IB_DEVICE_NAME_MAX]; /* ib device name */
+ int ndev_ifidx; /* network device ifindex */
enum smc_link_state state; /* state of link */
struct delayed_work llc_testlink_wrk; /* testlink worker */
struct completion llc_testlink_resp; /* wait for rx of testlink */
int llc_testlink_time; /* testlink interval */
+ atomic_t conn_cnt; /* connections on this link */
};
/* For now we just allow one parallel link per link group. The SMC protocol
return lnk->state == SMC_LNK_ACTIVE;
}
+static inline void smc_gid_be16_convert(__u8 *buf, u8 *gid_raw)
+{
+ sprintf(buf, "%04x:%04x:%04x:%04x:%04x:%04x:%04x:%04x",
+ be16_to_cpu(((__be16 *)gid_raw)[0]),
+ be16_to_cpu(((__be16 *)gid_raw)[1]),
+ be16_to_cpu(((__be16 *)gid_raw)[2]),
+ be16_to_cpu(((__be16 *)gid_raw)[3]),
+ be16_to_cpu(((__be16 *)gid_raw)[4]),
+ be16_to_cpu(((__be16 *)gid_raw)[5]),
+ be16_to_cpu(((__be16 *)gid_raw)[6]),
+ be16_to_cpu(((__be16 *)gid_raw)[7]));
+}
+
+struct smc_pci_dev {
+ __u32 pci_fid;
+ __u16 pci_pchid;
+ __u16 pci_vendor;
+ __u16 pci_device;
+ __u8 pci_id[SMC_PCI_ID_STR_LEN];
+};
+
+static inline void smc_set_pci_values(struct pci_dev *pci_dev,
+ struct smc_pci_dev *smc_dev)
+{
+ smc_dev->pci_vendor = pci_dev->vendor;
+ smc_dev->pci_device = pci_dev->device;
+ snprintf(smc_dev->pci_id, sizeof(smc_dev->pci_id), "%s",
+ pci_name(pci_dev));
+#if IS_ENABLED(CONFIG_S390)
+ { /* Set s390 specific PCI information */
+ struct zpci_dev *zdev;
+
+ zdev = to_zpci(pci_dev);
+ smc_dev->pci_fid = zdev->fid;
+ smc_dev->pci_pchid = zdev->pchid;
+ }
+#endif
+}
+
struct smc_sock;
struct smc_clc_msg_accept_confirm;
struct smc_clc_msg_local;
struct smc_link *from_lnk, bool is_dev_err);
void smcr_link_down_cond(struct smc_link *lnk);
void smcr_link_down_cond_sched(struct smc_link *lnk);
+int smc_nl_get_sys_info(struct sk_buff *skb, struct netlink_callback *cb);
+int smcr_nl_get_lgr(struct sk_buff *skb, struct netlink_callback *cb);
+int smcr_nl_get_link(struct sk_buff *skb, struct netlink_callback *cb);
+int smcd_nl_get_lgr(struct sk_buff *skb, struct netlink_callback *cb);
static inline struct smc_link_group *smc_get_lgr(struct smc_link *link)
{
return (struct smc_diag_dump_ctx *)cb->ctx;
}
-static void smc_gid_be16_convert(__u8 *buf, u8 *gid_raw)
-{
- sprintf(buf, "%04x:%04x:%04x:%04x:%04x:%04x:%04x:%04x",
- be16_to_cpu(((__be16 *)gid_raw)[0]),
- be16_to_cpu(((__be16 *)gid_raw)[1]),
- be16_to_cpu(((__be16 *)gid_raw)[2]),
- be16_to_cpu(((__be16 *)gid_raw)[3]),
- be16_to_cpu(((__be16 *)gid_raw)[4]),
- be16_to_cpu(((__be16 *)gid_raw)[5]),
- be16_to_cpu(((__be16 *)gid_raw)[6]),
- be16_to_cpu(((__be16 *)gid_raw)[7]));
-}
-
static void smc_diag_msg_common_fill(struct smc_diag_msg *r, struct sock *sk)
{
struct smc_sock *smc = smc_sk(sk);
!list_empty(&smc->conn.lgr->list)) {
struct smc_diag_lgrinfo linfo = {
.role = smc->conn.lgr->role,
- .lnk[0].ibport = smc->conn.lgr->lnk[0].ibport,
- .lnk[0].link_id = smc->conn.lgr->lnk[0].link_id,
+ .lnk[0].ibport = smc->conn.lnk->ibport,
+ .lnk[0].link_id = smc->conn.lnk->link_id,
};
memcpy(linfo.lnk[0].ibname,
smc->conn.lgr->lnk[0].smcibdev->ibdev->name,
- sizeof(smc->conn.lgr->lnk[0].smcibdev->ibdev->name));
+ sizeof(smc->conn.lnk->smcibdev->ibdev->name));
smc_gid_be16_convert(linfo.lnk[0].gid,
- smc->conn.lgr->lnk[0].gid);
+ smc->conn.lnk->gid);
smc_gid_be16_convert(linfo.lnk[0].peer_gid,
- smc->conn.lgr->lnk[0].peer_gid);
+ smc->conn.lnk->peer_gid);
if (nla_put(skb, SMC_DIAG_LGRINFO, sizeof(linfo), &linfo) < 0)
goto errout;
#include "smc_core.h"
#include "smc_wr.h"
#include "smc.h"
+#include "smc_netlink.h"
#define SMC_MAX_CQE 32766 /* max. # of completion queue elements */
rcu_read_lock();
ndev = rdma_read_gid_attr_ndev_rcu(attr);
if (!IS_ERR(ndev) &&
- ((!vlan_id && !is_vlan_dev(attr->ndev)) ||
- (vlan_id && is_vlan_dev(attr->ndev) &&
- vlan_dev_vlan_id(attr->ndev) == vlan_id)) &&
+ ((!vlan_id && !is_vlan_dev(ndev)) ||
+ (vlan_id && is_vlan_dev(ndev) &&
+ vlan_dev_vlan_id(ndev) == vlan_id)) &&
attr->gid_type == IB_GID_TYPE_ROCE) {
rcu_read_unlock();
if (gid)
return rc;
}
+static bool smcr_diag_is_dev_critical(struct smc_lgr_list *smc_lgr,
+ struct smc_ib_device *smcibdev)
+{
+ struct smc_link_group *lgr;
+ bool rc = false;
+ int i;
+
+ spin_lock_bh(&smc_lgr->lock);
+ list_for_each_entry(lgr, &smc_lgr->list, list) {
+ if (lgr->is_smcd)
+ continue;
+ for (i = 0; i < SMC_LINKS_PER_LGR_MAX; i++) {
+ if (lgr->lnk[i].state == SMC_LNK_UNUSED ||
+ lgr->lnk[i].smcibdev != smcibdev)
+ continue;
+ if (lgr->type == SMC_LGR_SINGLE ||
+ lgr->type == SMC_LGR_ASYMMETRIC_LOCAL) {
+ rc = true;
+ goto out;
+ }
+ }
+ }
+out:
+ spin_unlock_bh(&smc_lgr->lock);
+ return rc;
+}
+
+static int smc_nl_handle_dev_port(struct sk_buff *skb,
+ struct ib_device *ibdev,
+ struct smc_ib_device *smcibdev,
+ int port)
+{
+ char smc_pnet[SMC_MAX_PNETID_LEN + 1];
+ struct nlattr *port_attrs;
+ unsigned char port_state;
+ int lnk_count = 0;
+
+ port_attrs = nla_nest_start(skb, SMC_NLA_DEV_PORT + port);
+ if (!port_attrs)
+ goto errout;
+
+ if (nla_put_u8(skb, SMC_NLA_DEV_PORT_PNET_USR,
+ smcibdev->pnetid_by_user[port]))
+ goto errattr;
+ snprintf(smc_pnet, sizeof(smc_pnet), "%s",
+ (char *)&smcibdev->pnetid[port]);
+ if (nla_put_string(skb, SMC_NLA_DEV_PORT_PNETID, smc_pnet))
+ goto errattr;
+ if (nla_put_u32(skb, SMC_NLA_DEV_PORT_NETDEV,
+ smcibdev->ndev_ifidx[port]))
+ goto errattr;
+ if (nla_put_u8(skb, SMC_NLA_DEV_PORT_VALID, 1))
+ goto errattr;
+ port_state = smc_ib_port_active(smcibdev, port + 1);
+ if (nla_put_u8(skb, SMC_NLA_DEV_PORT_STATE, port_state))
+ goto errattr;
+ lnk_count = atomic_read(&smcibdev->lnk_cnt_by_port[port]);
+ if (nla_put_u32(skb, SMC_NLA_DEV_PORT_LNK_CNT, lnk_count))
+ goto errattr;
+ nla_nest_end(skb, port_attrs);
+ return 0;
+errattr:
+ nla_nest_cancel(skb, port_attrs);
+errout:
+ return -EMSGSIZE;
+}
+
+static int smc_nl_handle_smcr_dev(struct smc_ib_device *smcibdev,
+ struct sk_buff *skb,
+ struct netlink_callback *cb)
+{
+ char smc_ibname[IB_DEVICE_NAME_MAX + 1];
+ struct smc_pci_dev smc_pci_dev;
+ struct pci_dev *pci_dev;
+ unsigned char is_crit;
+ struct nlattr *attrs;
+ void *nlh;
+ int i;
+
+ nlh = genlmsg_put(skb, NETLINK_CB(cb->skb).portid, cb->nlh->nlmsg_seq,
+ &smc_gen_nl_family, NLM_F_MULTI,
+ SMC_NETLINK_GET_DEV_SMCR);
+ if (!nlh)
+ goto errmsg;
+ attrs = nla_nest_start(skb, SMC_GEN_DEV_SMCR);
+ if (!attrs)
+ goto errout;
+ is_crit = smcr_diag_is_dev_critical(&smc_lgr_list, smcibdev);
+ if (nla_put_u8(skb, SMC_NLA_DEV_IS_CRIT, is_crit))
+ goto errattr;
+ memset(&smc_pci_dev, 0, sizeof(smc_pci_dev));
+ pci_dev = to_pci_dev(smcibdev->ibdev->dev.parent);
+ smc_set_pci_values(pci_dev, &smc_pci_dev);
+ if (nla_put_u32(skb, SMC_NLA_DEV_PCI_FID, smc_pci_dev.pci_fid))
+ goto errattr;
+ if (nla_put_u16(skb, SMC_NLA_DEV_PCI_CHID, smc_pci_dev.pci_pchid))
+ goto errattr;
+ if (nla_put_u16(skb, SMC_NLA_DEV_PCI_VENDOR, smc_pci_dev.pci_vendor))
+ goto errattr;
+ if (nla_put_u16(skb, SMC_NLA_DEV_PCI_DEVICE, smc_pci_dev.pci_device))
+ goto errattr;
+ if (nla_put_string(skb, SMC_NLA_DEV_PCI_ID, smc_pci_dev.pci_id))
+ goto errattr;
+ snprintf(smc_ibname, sizeof(smc_ibname), "%s", smcibdev->ibdev->name);
+ if (nla_put_string(skb, SMC_NLA_DEV_IB_NAME, smc_ibname))
+ goto errattr;
+ for (i = 1; i <= SMC_MAX_PORTS; i++) {
+ if (!rdma_is_port_valid(smcibdev->ibdev, i))
+ continue;
+ if (smc_nl_handle_dev_port(skb, smcibdev->ibdev,
+ smcibdev, i - 1))
+ goto errattr;
+ }
+
+ nla_nest_end(skb, attrs);
+ genlmsg_end(skb, nlh);
+ return 0;
+
+errattr:
+ nla_nest_cancel(skb, attrs);
+errout:
+ genlmsg_cancel(skb, nlh);
+errmsg:
+ return -EMSGSIZE;
+}
+
+static void smc_nl_prep_smcr_dev(struct smc_ib_devices *dev_list,
+ struct sk_buff *skb,
+ struct netlink_callback *cb)
+{
+ struct smc_nl_dmp_ctx *cb_ctx = smc_nl_dmp_ctx(cb);
+ struct smc_ib_device *smcibdev;
+ int snum = cb_ctx->pos[0];
+ int num = 0;
+
+ mutex_lock(&dev_list->mutex);
+ list_for_each_entry(smcibdev, &dev_list->list, list) {
+ if (num < snum)
+ goto next;
+ if (smc_nl_handle_smcr_dev(smcibdev, skb, cb))
+ goto errout;
+next:
+ num++;
+ }
+errout:
+ mutex_unlock(&dev_list->mutex);
+ cb_ctx->pos[0] = num;
+}
+
+int smcr_nl_get_device(struct sk_buff *skb, struct netlink_callback *cb)
+{
+ smc_nl_prep_smcr_dev(&smc_ib_devices, skb, cb);
+ return skb->len;
+}
+
static void smc_ib_qp_event_handler(struct ib_event *ibevent, void *priv)
{
struct smc_link *lnk = (struct smc_link *)priv;
static struct ib_client smc_ib_client;
+static void smc_copy_netdev_ifindex(struct smc_ib_device *smcibdev, int port)
+{
+ struct ib_device *ibdev = smcibdev->ibdev;
+ struct net_device *ndev;
+
+ if (!ibdev->ops.get_netdev)
+ return;
+ ndev = ibdev->ops.get_netdev(ibdev, port + 1);
+ if (ndev) {
+ smcibdev->ndev_ifidx[port] = ndev->ifindex;
+ dev_put(ndev);
+ }
+}
+
+void smc_ib_ndev_change(struct net_device *ndev, unsigned long event)
+{
+ struct smc_ib_device *smcibdev;
+ struct ib_device *libdev;
+ struct net_device *lndev;
+ u8 port_cnt;
+ int i;
+
+ mutex_lock(&smc_ib_devices.mutex);
+ list_for_each_entry(smcibdev, &smc_ib_devices.list, list) {
+ port_cnt = smcibdev->ibdev->phys_port_cnt;
+ for (i = 0; i < min_t(size_t, port_cnt, SMC_MAX_PORTS); i++) {
+ libdev = smcibdev->ibdev;
+ if (!libdev->ops.get_netdev)
+ continue;
+ lndev = libdev->ops.get_netdev(libdev, i + 1);
+ if (lndev)
+ dev_put(lndev);
+ if (lndev != ndev)
+ continue;
+ if (event == NETDEV_REGISTER)
+ smcibdev->ndev_ifidx[i] = ndev->ifindex;
+ if (event == NETDEV_UNREGISTER)
+ smcibdev->ndev_ifidx[i] = 0;
+ }
+ }
+ mutex_unlock(&smc_ib_devices.mutex);
+}
+
/* callback function for ib_register_client() */
static int smc_ib_add_dev(struct ib_device *ibdev)
{
if (smc_pnetid_by_dev_port(ibdev->dev.parent, i,
smcibdev->pnetid[i]))
smc_pnetid_by_table_ib(smcibdev, i + 1);
+ smc_copy_netdev_ifindex(smcibdev, i);
pr_warn_ratelimited("smc: ib device %s port %d has pnetid "
"%.16s%s\n",
smcibdev->ibdev->name, i + 1,
};
extern struct smc_ib_devices smc_ib_devices; /* list of smc ib devices */
+extern struct smc_lgr_list smc_lgr_list; /* list of linkgroups */
struct smc_ib_device { /* ib-device infos for smc */
struct list_head list;
atomic_t lnk_cnt; /* number of links on ibdev */
wait_queue_head_t lnks_deleted; /* wait 4 removal of all links*/
struct mutex mutex; /* protect dev setup+cleanup */
+ atomic_t lnk_cnt_by_port[SMC_MAX_PORTS];
+ /* number of links per port */
+ int ndev_ifidx[SMC_MAX_PORTS]; /* ndev if indexes */
};
struct smc_buf_desc;
struct smc_link;
+void smc_ib_ndev_change(struct net_device *ndev, unsigned long event);
int smc_ib_register_client(void) __init;
void smc_ib_unregister_client(void);
bool smc_ib_port_active(struct smc_ib_device *smcibdev, u8 ibport);
int smc_ib_determine_gid(struct smc_ib_device *smcibdev, u8 ibport,
unsigned short vlan_id, u8 gid[], u8 *sgid_index);
bool smc_ib_is_valid_local_systemid(void);
+int smcr_nl_get_device(struct sk_buff *skb, struct netlink_callback *cb);
#endif
#include "smc_core.h"
#include "smc_ism.h"
#include "smc_pnet.h"
+#include "smc_netlink.h"
struct smcd_dev_list smcd_dev_list = {
.list = LIST_HEAD_INIT(smcd_dev_list.list),
.mutex = __MUTEX_INITIALIZER(smcd_dev_list.mutex)
};
-bool smc_ism_v2_capable;
+static bool smc_ism_v2_capable;
/* Test if an ISM communication is possible - same CPC */
int smc_ism_cantalk(u64 peer_gid, unsigned short vlan_id, struct smcd_dev *smcd)
return smcd->ops->get_chid(smcd);
}
+/* HW supports ISM V2 and thus System EID is defined */
+bool smc_ism_is_v2_capable(void)
+{
+ return smc_ism_v2_capable;
+}
+
/* Set a connection using this DMBE. */
void smc_ism_set_conn(struct smc_connection *conn)
{
return rc;
}
+static int smc_nl_handle_smcd_dev(struct smcd_dev *smcd,
+ struct sk_buff *skb,
+ struct netlink_callback *cb)
+{
+ char smc_pnet[SMC_MAX_PNETID_LEN + 1];
+ struct smc_pci_dev smc_pci_dev;
+ struct nlattr *port_attrs;
+ struct nlattr *attrs;
+ int use_cnt = 0;
+ void *nlh;
+
+ nlh = genlmsg_put(skb, NETLINK_CB(cb->skb).portid, cb->nlh->nlmsg_seq,
+ &smc_gen_nl_family, NLM_F_MULTI,
+ SMC_NETLINK_GET_DEV_SMCD);
+ if (!nlh)
+ goto errmsg;
+ attrs = nla_nest_start(skb, SMC_GEN_DEV_SMCD);
+ if (!attrs)
+ goto errout;
+ use_cnt = atomic_read(&smcd->lgr_cnt);
+ if (nla_put_u32(skb, SMC_NLA_DEV_USE_CNT, use_cnt))
+ goto errattr;
+ if (nla_put_u8(skb, SMC_NLA_DEV_IS_CRIT, use_cnt > 0))
+ goto errattr;
+ memset(&smc_pci_dev, 0, sizeof(smc_pci_dev));
+ smc_set_pci_values(to_pci_dev(smcd->dev.parent), &smc_pci_dev);
+ if (nla_put_u32(skb, SMC_NLA_DEV_PCI_FID, smc_pci_dev.pci_fid))
+ goto errattr;
+ if (nla_put_u16(skb, SMC_NLA_DEV_PCI_CHID, smc_pci_dev.pci_pchid))
+ goto errattr;
+ if (nla_put_u16(skb, SMC_NLA_DEV_PCI_VENDOR, smc_pci_dev.pci_vendor))
+ goto errattr;
+ if (nla_put_u16(skb, SMC_NLA_DEV_PCI_DEVICE, smc_pci_dev.pci_device))
+ goto errattr;
+ if (nla_put_string(skb, SMC_NLA_DEV_PCI_ID, smc_pci_dev.pci_id))
+ goto errattr;
+
+ port_attrs = nla_nest_start(skb, SMC_NLA_DEV_PORT);
+ if (!port_attrs)
+ goto errattr;
+ if (nla_put_u8(skb, SMC_NLA_DEV_PORT_PNET_USR, smcd->pnetid_by_user))
+ goto errportattr;
+ snprintf(smc_pnet, sizeof(smc_pnet), "%s", smcd->pnetid);
+ if (nla_put_string(skb, SMC_NLA_DEV_PORT_PNETID, smc_pnet))
+ goto errportattr;
+
+ nla_nest_end(skb, port_attrs);
+ nla_nest_end(skb, attrs);
+ genlmsg_end(skb, nlh);
+ return 0;
+
+errportattr:
+ nla_nest_cancel(skb, port_attrs);
+errattr:
+ nla_nest_cancel(skb, attrs);
+errout:
+ nlmsg_cancel(skb, nlh);
+errmsg:
+ return -EMSGSIZE;
+}
+
+static void smc_nl_prep_smcd_dev(struct smcd_dev_list *dev_list,
+ struct sk_buff *skb,
+ struct netlink_callback *cb)
+{
+ struct smc_nl_dmp_ctx *cb_ctx = smc_nl_dmp_ctx(cb);
+ int snum = cb_ctx->pos[0];
+ struct smcd_dev *smcd;
+ int num = 0;
+
+ mutex_lock(&dev_list->mutex);
+ list_for_each_entry(smcd, &dev_list->list, list) {
+ if (num < snum)
+ goto next;
+ if (smc_nl_handle_smcd_dev(smcd, skb, cb))
+ goto errout;
+next:
+ num++;
+ }
+errout:
+ mutex_unlock(&dev_list->mutex);
+ cb_ctx->pos[0] = num;
+}
+
+int smcd_nl_get_device(struct sk_buff *skb, struct netlink_callback *cb)
+{
+ smc_nl_prep_smcd_dev(&smcd_dev_list, skb, cb);
+ return skb->len;
+}
+
struct smc_ism_event_work {
struct work_struct work;
struct smcd_dev *smcd;
#define SMCD_ISM_H
#include <linux/uio.h>
+#include <linux/types.h>
#include <linux/mutex.h>
#include "smc.h"
};
extern struct smcd_dev_list smcd_dev_list; /* list of smcd devices */
-extern bool smc_ism_v2_capable; /* HW supports ISM V2 and thus
- * System EID is defined
- */
struct smc_ism_vlanid { /* VLAN id set on ISM device */
struct list_head list;
int smc_ism_signal_shutdown(struct smc_link_group *lgr);
void smc_ism_get_system_eid(struct smcd_dev *dev, u8 **eid);
u16 smc_ism_get_chid(struct smcd_dev *dev);
+bool smc_ism_is_v2_capable(void);
void smc_ism_init(void);
+int smcd_nl_get_device(struct sk_buff *skb, struct netlink_callback *cb);
#endif
--- /dev/null
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Shared Memory Communications over RDMA (SMC-R) and RoCE
+ *
+ * Generic netlink support functions to interact with SMC module
+ *
+ * Copyright IBM Corp. 2020
+ *
+ * Author(s): Guvenc Gulce <guvenc@linux.ibm.com>
+ */
+
+#include <linux/module.h>
+#include <linux/list.h>
+#include <linux/ctype.h>
+#include <linux/mutex.h>
+#include <linux/if.h>
+#include <linux/smc.h>
+
+#include "smc_core.h"
+#include "smc_ism.h"
+#include "smc_ib.h"
+#include "smc_netlink.h"
+
+#define SMC_CMD_MAX_ATTR 1
+
+/* SMC_GENL generic netlink operation definition */
+static const struct genl_ops smc_gen_nl_ops[] = {
+ {
+ .cmd = SMC_NETLINK_GET_SYS_INFO,
+ /* can be retrieved by unprivileged users */
+ .dumpit = smc_nl_get_sys_info,
+ },
+ {
+ .cmd = SMC_NETLINK_GET_LGR_SMCR,
+ /* can be retrieved by unprivileged users */
+ .dumpit = smcr_nl_get_lgr,
+ },
+ {
+ .cmd = SMC_NETLINK_GET_LINK_SMCR,
+ /* can be retrieved by unprivileged users */
+ .dumpit = smcr_nl_get_link,
+ },
+ {
+ .cmd = SMC_NETLINK_GET_LGR_SMCD,
+ /* can be retrieved by unprivileged users */
+ .dumpit = smcd_nl_get_lgr,
+ },
+ {
+ .cmd = SMC_NETLINK_GET_DEV_SMCD,
+ /* can be retrieved by unprivileged users */
+ .dumpit = smcd_nl_get_device,
+ },
+ {
+ .cmd = SMC_NETLINK_GET_DEV_SMCR,
+ /* can be retrieved by unprivileged users */
+ .dumpit = smcr_nl_get_device,
+ },
+};
+
+static const struct nla_policy smc_gen_nl_policy[2] = {
+ [SMC_CMD_MAX_ATTR] = { .type = NLA_REJECT, },
+};
+
+/* SMC_GENL family definition */
+struct genl_family smc_gen_nl_family __ro_after_init = {
+ .hdrsize = 0,
+ .name = SMC_GENL_FAMILY_NAME,
+ .version = SMC_GENL_FAMILY_VERSION,
+ .maxattr = SMC_CMD_MAX_ATTR,
+ .policy = smc_gen_nl_policy,
+ .netnsok = true,
+ .module = THIS_MODULE,
+ .ops = smc_gen_nl_ops,
+ .n_ops = ARRAY_SIZE(smc_gen_nl_ops)
+};
+
+int __init smc_nl_init(void)
+{
+ return genl_register_family(&smc_gen_nl_family);
+}
+
+void smc_nl_exit(void)
+{
+ genl_unregister_family(&smc_gen_nl_family);
+}
--- /dev/null
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Shared Memory Communications over RDMA (SMC-R) and RoCE
+ *
+ * SMC Generic netlink operations
+ *
+ * Copyright IBM Corp. 2020
+ *
+ * Author(s): Guvenc Gulce <guvenc@linux.ibm.com>
+ */
+
+#ifndef _SMC_NETLINK_H
+#define _SMC_NETLINK_H
+
+#include <net/netlink.h>
+#include <net/genetlink.h>
+
+extern struct genl_family smc_gen_nl_family;
+
+struct smc_nl_dmp_ctx {
+ int pos[2];
+};
+
+static inline struct smc_nl_dmp_ctx *smc_nl_dmp_ctx(struct netlink_callback *c)
+{
+ return (struct smc_nl_dmp_ctx *)c->ctx;
+}
+
+int smc_nl_init(void) __init;
+void smc_nl_exit(void);
+
+#endif
case NETDEV_REBOOT:
case NETDEV_UNREGISTER:
smc_pnet_remove_by_ndev(event_dev);
+ smc_ib_ndev_change(event_dev, event);
return NOTIFY_OK;
case NETDEV_REGISTER:
smc_pnet_add_by_ndev(event_dev);
+ smc_ib_ndev_change(event_dev, event);
return NOTIFY_OK;
case NETDEV_UP:
smc_pnet_add_base_pnetid(net, event_dev, ndev_pnetid);
* Based upon Swansea University Computer Society NET3.039
*/
+#include <linux/ethtool.h>
#include <linux/mm.h>
#include <linux/socket.h>
#include <linux/file.h>
#include <linux/seq_file.h>
#include <linux/mutex.h>
#include <linux/if_bridge.h>
-#include <linux/if_frad.h>
#include <linux/if_vlan.h>
#include <linux/ptp_classify.h>
#include <linux/init.h>
}
EXPORT_SYMBOL(vlan_ioctl_set);
-static DEFINE_MUTEX(dlci_ioctl_mutex);
-static int (*dlci_ioctl_hook) (unsigned int, void __user *);
-
-void dlci_ioctl_set(int (*hook) (unsigned int, void __user *))
-{
- mutex_lock(&dlci_ioctl_mutex);
- dlci_ioctl_hook = hook;
- mutex_unlock(&dlci_ioctl_mutex);
-}
-EXPORT_SYMBOL(dlci_ioctl_set);
-
static long sock_do_ioctl(struct net *net, struct socket *sock,
unsigned int cmd, unsigned long arg)
{
err = vlan_ioctl_hook(net, argp);
mutex_unlock(&vlan_ioctl_mutex);
break;
- case SIOCADDDLCI:
- case SIOCDELDLCI:
- err = -ENOPKG;
- if (!dlci_ioctl_hook)
- request_module("dlci");
-
- mutex_lock(&dlci_ioctl_mutex);
- if (dlci_ioctl_hook)
- err = dlci_ioctl_hook(cmd, argp);
- mutex_unlock(&dlci_ioctl_mutex);
- break;
case SIOCGSKNS:
err = -EPERM;
if (!ns_capable(net->user_ns, CAP_NET_ADMIN))
case SIOCBRDELBR:
case SIOCGIFVLAN:
case SIOCSIFVLAN:
- case SIOCADDDLCI:
- case SIOCDELDLCI:
case SIOCGSKNS:
case SIOCGSTAMP_NEW:
case SIOCGSTAMPNS_NEW:
}
/**
- * rpc_mkpipe - make an rpc_pipefs file for kernel<->userspace communication
+ * rpc_mkpipe_dentry - make an rpc_pipefs file for kernel<->userspace
+ * communication
* @parent: dentry of directory to create new "pipe" in
* @name: name of pipe
* @private: private data to associate with the pipe, for the caller's use
void tipc_set_node_id(struct net *net, u8 *id)
{
struct tipc_net *tn = tipc_net(net);
- u32 *tmp = (u32 *)id;
memcpy(tn->node_id, id, NODE_ID_LEN);
tipc_nodeid2string(tn->node_id_string, id);
- tn->trial_addr = tmp[0] ^ tmp[1] ^ tmp[2] ^ tmp[3];
- pr_info("Own node identity %s, cluster identity %u\n",
+ tn->trial_addr = hash128to32(id);
+ pr_info("Node identity %s, cluster identity %u\n",
tipc_own_id_string(net), tn->net_id);
}
}
tn->trial_addr = addr;
tn->addr_trial_end = jiffies;
- pr_info("32-bit node address hash set to %x\n", addr);
+ pr_info("Node number set to %u\n", addr);
}
char *tipc_nodeid2string(char *str, u8 *id)
*
* Copyright (c) 2000-2006, 2018, Ericsson AB
* Copyright (c) 2004-2005, Wind River Systems
+ * Copyright (c) 2020, Red Hat Inc
* All rights reserved.
*
* Redistribution and use in source and binary forms, with or without
/**
* tipc_media_find - locates specified media object by name
+ * @name: name to locate
*/
struct tipc_media *tipc_media_find(const char *name)
{
/**
* media_find_id - locates specified media object by type identifier
+ * @type: type identifier to locate
*/
static struct tipc_media *media_find_id(u8 type)
{
/**
* tipc_media_addr_printf - record media address in print buffer
+ * @buf: output buffer
+ * @len: output buffer size remaining
+ * @a: input media address
*/
int tipc_media_addr_printf(char *buf, int len, struct tipc_media_addr *a)
{
* @name: ptr to bearer name string
* @name_parts: ptr to area for bearer name components (or NULL if not needed)
*
- * Returns 1 if bearer name is valid, otherwise 0.
+ * Return: 1 if bearer name is valid, otherwise 0.
*/
static int bearer_name_validate(const char *name,
struct tipc_bearer_names *name_parts)
/**
* tipc_bearer_find - locates bearer object with matching bearer name
+ * @net: the applicable net namespace
+ * @name: bearer name to locate
*/
struct tipc_bearer *tipc_bearer_find(struct net *net, const char *name)
{
/**
* tipc_enable_bearer - enable bearer with the given name
+ * @net: the applicable net namespace
+ * @name: bearer name to enable
+ * @disc_domain: bearer domain
+ * @prio: bearer priority
+ * @attr: nlattr array
*/
static int tipc_enable_bearer(struct net *net, const char *name,
u32 disc_domain, u32 prio,
/**
* tipc_reset_bearer - Reset all links established over this bearer
+ * @net: the applicable net namespace
+ * @b: the target bearer
*/
static int tipc_reset_bearer(struct net *net, struct tipc_bearer *b)
{
}
/**
- * bearer_disable
+ * bearer_disable - disable this bearer
+ * @net: the applicable net namespace
+ * @b: the bearer to disable
*
* Note: This routine assumes caller holds RTNL lock.
*/
}
/* tipc_disable_l2_media - detach TIPC bearer from an L2 interface
+ * @b: the target bearer
*
* Mark L2 bearer as inactive so that incoming buffers are thrown away
*/
/**
* tipc_l2_send_msg - send a TIPC packet out over an L2 interface
+ * @net: the associated network namespace
* @skb: the packet to be sent
* @b: the bearer through which the packet is to be sent
* @dest: peer destination address
* @raw2addr: convert from raw addr format to media addr format
* @priority: default link (and bearer) priority
* @tolerance: default time (in ms) before declaring link failure
- * @window: default window (in packets) before declaring link congestion
+ * @min_win: minimum window (in packets) before declaring link congestion
+ * @max_win: maximum window (in packets) before declaring link congestion
* @mtu: max packet size bearer can support for media type not dependent on
* underlying device MTU
* @type_id: TIPC media identifier
* @pt: packet type for bearer
* @rcu: rcu struct for tipc_bearer
* @priority: default link priority for bearer
- * @window: default window size for bearer
+ * @min_win: minimum window (in packets) before declaring link congestion
+ * @max_win: maximum window (in packets) before declaring link congestion
* @tolerance: default link tolerance for bearer
* @domain: network domain to which links can be established
* @identity: array index of this bearer within TIPC bearer array
- * @link_req: ptr to (optional) structure making periodic link setup requests
+ * @disc: ptr to link setup request
* @net_plane: network plane ('A' through 'H') currently associated with bearer
+ * @up: bearer up flag (bit 0)
+ * @refcnt: tipc_bearer reference counter
*
* Note: media-specific code is responsible for initialization of the fields
* indicated below when a bearer is enabled; TIPC's generic bearer code takes
*
* Copyright (c) 2005-2006, 2013-2018 Ericsson AB
* Copyright (c) 2005-2007, 2010-2013, Wind River Systems
+ * Copyright (c) 2020, Red Hat Inc
* All rights reserved.
*
* Redistribution and use in source and binary forms, with or without
return net_hash_mix(&init_net) ^ net_hash_mix(net) ^ tn_rand;
}
+static inline u32 hash128to32(char *bytes)
+{
+ __be32 *tmp = (__be32 *)bytes;
+ u32 res;
+
+ res = ntohl(tmp[0] ^ tmp[1] ^ tmp[2] ^ tmp[3]);
+ if (likely(res))
+ return res;
+ return ntohl(tmp[0] | tmp[1] | tmp[2] | tmp[3]);
+}
+
#ifdef CONFIG_SYSCTL
int tipc_register_sysctl(void);
void tipc_unregister_sysctl(void);
// SPDX-License-Identifier: GPL-2.0
-/**
+/*
* net/tipc/crypto.c: TIPC crypto for key handling & packet en/decryption
*
* Copyright (c) 2019, Ericsson AB
#define TIPC_REKEYING_INTV_DEF (60 * 24) /* default: 1 day */
-/**
+/*
* TIPC Key ids
*/
enum {
KEY_MAX = KEY_3,
};
-/**
+/*
* TIPC Crypto statistics
*/
enum {
/* Key exchange switch, default: on */
int sysctl_tipc_key_exchange_enabled __read_mostly = 1;
-/**
+/*
* struct tipc_key - TIPC keys' status indicator
*
* 7 6 5 4 3 2 1 0
/**
* struct tipc_tfm - TIPC TFM structure to form a list of TFMs
+ * @tfm: cipher handle/key
+ * @list: linked list of TFMs
*/
struct tipc_tfm {
struct crypto_aead *tfm;
* @salt: the key's SALT value
* @authsize: authentication tag size (max = 16)
* @mode: crypto mode is applied to the key
- * @hint[]: a hint for user key
+ * @hint: a hint for user key
* @rcu: struct rcu_head
* @key: the aead key
* @gen: the key's generation
/**
* struct tipc_crypto_stats - TIPC Crypto statistics
+ * @stat: array of crypto statistics
*/
struct tipc_crypto_stats {
unsigned int stat[MAX_STATS];
* @key_master: flag indicates if master key exists
* @legacy_user: flag indicates if a peer joins w/o master key (for bwd comp.)
* @nokey: no key indication
+ * @flags: combined flags field
* @lock: tipc_key lock
*/
struct tipc_crypto {
/**
* tipc_aead_key_validate - Validate a AEAD user key
+ * @ukey: pointer to user key data
+ * @info: netlink info pointer
*/
int tipc_aead_key_validate(struct tipc_aead_key *ukey, struct genl_info *info)
{
/**
* tipc_aead_tfm_next - Move TFM entry to the next one in list and return it
+ * @aead: the AEAD key pointer
*/
static struct crypto_aead *tipc_aead_tfm_next(struct tipc_aead *aead)
{
* @__dnode: TIPC dest node if "known"
*
* Return:
- * 0 : if the encryption has completed
- * -EINPROGRESS/-EBUSY : if a callback will be performed
- * < 0 : the encryption has failed
+ * * 0 : if the encryption has completed
+ * * -EINPROGRESS/-EBUSY : if a callback will be performed
+ * * < 0 : the encryption has failed
*/
static int tipc_aead_encrypt(struct tipc_aead *aead, struct sk_buff *skb,
struct tipc_bearer *b,
* @b: TIPC bearer where the message has been received
*
* Return:
- * 0 : if the decryption has completed
- * -EINPROGRESS/-EBUSY : if a callback will be performed
- * < 0 : the decryption has failed
+ * * 0 : if the decryption has completed
+ * * -EINPROGRESS/-EBUSY : if a callback will be performed
+ * * < 0 : the decryption has failed
*/
static int tipc_aead_decrypt(struct net *net, struct tipc_aead *aead,
struct sk_buff *skb, struct tipc_bearer *b)
* tipc_ehdr_validate - Validate an encryption message
* @skb: the message buffer
*
- * Returns "true" if this is a valid encryption message, otherwise "false"
+ * Return: "true" if this is a valid encryption message, otherwise "false"
*/
bool tipc_ehdr_validate(struct sk_buff *skb)
{
* Otherwise, the skb is freed!
*
* Return:
- * 0 : the encryption has succeeded (or no encryption)
- * -EINPROGRESS/-EBUSY : the encryption is ongoing, a callback will be made
- * -ENOKEK : the encryption has failed due to no key
- * -EKEYREVOKED : the encryption has failed due to key revoked
- * -ENOMEM : the encryption has failed due to no memory
- * < 0 : the encryption has failed due to other reasons
+ * * 0 : the encryption has succeeded (or no encryption)
+ * * -EINPROGRESS/-EBUSY : the encryption is ongoing, a callback will be made
+ * * -ENOKEK : the encryption has failed due to no key
+ * * -EKEYREVOKED : the encryption has failed due to key revoked
+ * * -ENOMEM : the encryption has failed due to no memory
+ * * < 0 : the encryption has failed due to other reasons
*/
int tipc_crypto_xmit(struct net *net, struct sk_buff **skb,
struct tipc_bearer *b, struct tipc_media_addr *dst,
* cluster key(s) can be taken for decryption (- recursive).
*
* Return:
- * 0 : the decryption has successfully completed
- * -EINPROGRESS/-EBUSY : the decryption is ongoing, a callback will be made
- * -ENOKEY : the decryption has failed due to no key
- * -EBADMSG : the decryption has failed due to bad message
- * -ENOMEM : the decryption has failed due to no memory
- * < 0 : the decryption has failed due to other reasons
+ * * 0 : the decryption has successfully completed
+ * * -EINPROGRESS/-EBUSY : the decryption is ongoing, a callback will be made
+ * * -ENOKEY : the decryption has failed due to no key
+ * * -EBADMSG : the decryption has failed due to bad message
+ * * -ENOMEM : the decryption has failed due to no memory
+ * * < 0 : the decryption has failed due to other reasons
*/
int tipc_crypto_rcv(struct net *net, struct tipc_crypto *rx,
struct sk_buff **skb, struct tipc_bearer *b)
/* SPDX-License-Identifier: GPL-2.0 */
-/**
+/*
* net/tipc/crypto.h: Include file for TIPC crypto
*
* Copyright (c) 2019, Ericsson AB
#define TIPC_AES_GCM_IV_SIZE 12
#define TIPC_AES_GCM_TAG_SIZE 16
-/**
+/*
* TIPC crypto modes:
* - CLUSTER_KEY:
* One single key is used for both TX & RX in all nodes in the cluster.
extern int sysctl_tipc_max_tfms __read_mostly;
extern int sysctl_tipc_key_exchange_enabled __read_mostly;
-/**
+/*
* TIPC encryption message format:
*
* 3 3 2 2 2 2 2 2 2 2 2 2 1 1 1 1 1 1 1 1 1 1 0 0 0 0 0 0 0 0 0 0
/**
* tipc_disc_init_msg - initialize a link setup message
* @net: the applicable net namespace
+ * @skb: buffer containing message
* @mtyp: message type (request or response)
* @b: ptr to bearer issuing message
*/
* @dest: destination address for request messages
* @skb: pointer to created frame
*
- * Returns 0 if successful, otherwise -errno.
+ * Return: 0 if successful, otherwise -errno.
*/
int tipc_disc_create(struct net *net, struct tipc_bearer *b,
struct tipc_media_addr *dest, struct sk_buff **skb)
/**
* tipc_disc_delete - destroy object sending periodic link setup requests
- * @d: ptr to link duest structure
+ * @d: ptr to link dest structure
*/
void tipc_disc_delete(struct tipc_discoverer *d)
{
* net/tipc/group.c: TIPC group messaging code
*
* Copyright (c) 2017, Ericsson AB
+ * Copyright (c) 2020, Red Hat Inc
* All rights reserved.
*
* Redistribution and use in source and binary forms, with or without
return &grp->dests;
}
-void tipc_group_self(struct tipc_group *grp, struct tipc_name_seq *seq,
+void tipc_group_self(struct tipc_group *grp, struct tipc_service_range *seq,
int *scope)
{
seq->type = grp->type;
* net/tipc/group.h: Include file for TIPC group unicast/multicast functions
*
* Copyright (c) 2017, Ericsson AB
+ * Copyright (c) 2020, Red Hat Inc
* All rights reserved.
*
* Redistribution and use in source and binary forms, with or without
void tipc_group_add_member(struct tipc_group *grp, u32 node,
u32 port, u32 instance);
struct tipc_nlist *tipc_group_dests(struct tipc_group *grp);
-void tipc_group_self(struct tipc_group *grp, struct tipc_name_seq *seq,
+void tipc_group_self(struct tipc_group *grp, struct tipc_service_range *seq,
int *scope);
u32 tipc_group_exclude(struct tipc_group *grp);
void tipc_group_filter_msg(struct tipc_group *grp,
* @reasm_buf: head of partially reassembled inbound message fragments
* @bc_rcvr: marks that this is a broadcast receiver link
* @stats: collects statistics regarding link activity
+ * @session: session to be used by link
+ * @snd_nxt_state: next send seq number
+ * @rcv_nxt_state: next rcv seq number
+ * @in_session: have received ACTIVATE_MSG from peer
+ * @active: link is active
+ * @if_name: associated interface name
+ * @rst_cnt: link reset counter
+ * @drop_point: seq number for failover handling (FIXME)
+ * @failover_reasm_skb: saved failover msg ptr (FIXME)
+ * @failover_deferdq: deferred message queue for failover processing (FIXME)
+ * @transmq: the link's transmit queue
+ * @backlog: link's backlog by priority (importance)
+ * @snd_nxt: next sequence number to be used
+ * @rcv_unacked: # messages read by user, but not yet acked back to peer
+ * @deferdq: deferred receive queue
+ * @window: sliding window size for congestion handling
+ * @min_win: minimal send window to be used by link
+ * @ssthresh: slow start threshold for congestion handling
+ * @max_win: maximal send window to be used by link
+ * @cong_acks: congestion acks for congestion avoidance (FIXME)
+ * @checkpoint: seq number for congestion window size handling
+ * @reasm_tnlmsg: fragmentation/reassembly area for tunnel protocol message
+ * @last_gap: last gap ack blocks for bcast (FIXME)
+ * @last_ga: ptr to gap ack blocks
+ * @bc_rcvlink: the peer specific link used for broadcast reception
+ * @bc_sndlink: the namespace global link used for broadcast sending
+ * @nack_state: bcast nack state
+ * @bc_peer_is_up: peer has acked the bcast init msg
*/
struct tipc_link {
u32 addr;
* @min_win: minimal send window to be used by link
* @max_win: maximal send window to be used by link
* @session: session to be used by link
- * @ownnode: identity of own node
* @peer: node id of peer node
* @peer_caps: bitmap describing peer node capabilities
* @bc_sndlink: the namespace global link used for broadcast sending
* @inputq: queue to put messages ready for delivery
* @namedq: queue to put binding table update messages ready for delivery
* @link: return value, pointer to put the created link
+ * @self: local unicast link id
+ * @peer_id: 128-bit ID of peer
*
- * Returns true if link was created, otherwise false
+ * Return: true if link was created, otherwise false
*/
bool tipc_link_create(struct net *net, char *if_name, int bearer_id,
int tolerance, char net_plane, u32 mtu, int priority,
* @inputq: queue to put messages ready for delivery
* @namedq: queue to put binding table update messages ready for delivery
* @link: return value, pointer to put the created link
+ * @ownnode: identity of own node
+ * @peer: node id of peer node
+ * @peer_id: 128-bit ID of peer
+ * @peer_caps: bitmap describing peer node capabilities
+ * @bc_sndlink: the namespace global link used for broadcast sending
*
- * Returns true if link was created, otherwise false
+ * Return: true if link was created, otherwise false
*/
bool tipc_link_bc_create(struct net *net, u32 ownnode, u32 peer, u8 *peer_id,
int mtu, u32 min_win, u32 max_win, u16 peer_caps,
* tipc_link_too_silent - check if link is "too silent"
* @l: tipc link to be checked
*
- * Returns true if the link 'silent_intv_cnt' is about to reach the
+ * Return: true if the link 'silent_intv_cnt' is about to reach the
* 'abort_limit' value, otherwise false
*/
bool tipc_link_too_silent(struct tipc_link *l)
* @xmitq: returned list of packets to be sent by caller
*
* Consumes the buffer chain.
- * Returns 0 if success, or errno: -ELINKCONG, -EMSGSIZE or -ENOBUFS
* Messages at TIPC_SYSTEM_IMPORTANCE are always accepted
+ * Return: 0 if success, or errno: -ELINKCONG, -EMSGSIZE or -ENOBUFS
*/
int tipc_link_xmit(struct tipc_link *l, struct sk_buff_head *list,
struct sk_buff_head *xmitq)
if (!msg_peer_node_is_up(hdr))
return rc;
- /* Open when peer ackowledges our bcast init msg (pkt #1) */
+ /* Open when peer acknowledges our bcast init msg (pkt #1) */
if (msg_ack(hdr))
l->bc_peer_is_up = true;
/**
* tipc_buf_acquire - creates a TIPC message buffer
* @size: message size (including TIPC header)
+ * @gfp: memory allocation flags
*
- * Returns a new buffer with data pointers set to the specified size.
+ * Return: a new buffer with data pointers set to the specified size.
*
- * NOTE: Headroom is reserved to allow prepending of a data link header.
- * There may also be unrequested tailroom present at the buffer's end.
+ * NOTE:
+ * Headroom is reserved to allow prepending of a data link header.
+ * There may also be unrequested tailroom present at the buffer's end.
*/
struct sk_buff *tipc_buf_acquire(u32 size, gfp_t gfp)
{
* @m: the data to be appended
* @mss: max allowable size of buffer
* @dlen: size of data to be appended
- * @txq: queue to appand to
- * Returns the number og 1k blocks appended or errno value
+ * @txq: queue to append to
+ *
+ * Return: the number of 1k blocks appended or errno value
*/
int tipc_msg_append(struct tipc_msg *_hdr, struct msghdr *m, int dlen,
int mss, struct sk_buff_head *txq)
* @pktmax: max size of a fragment incl. the header
* @frags: returned fragment skb list
*
- * Returns 0 if the fragmentation is successful, otherwise: -EINVAL
+ * Return: 0 if the fragmentation is successful, otherwise: -EINVAL
* or -ENOMEM
*/
int tipc_msg_fragment(struct sk_buff *skb, const struct tipc_msg *hdr,
* tipc_msg_build - create buffer chain containing specified header and data
* @mhdr: Message header, to be prepended to data
* @m: User message
+ * @offset: buffer offset for fragmented messages (FIXME)
* @dsz: Total length of user data
* @pktmax: Max packet size that can be used
* @list: Buffer or chain of buffers to be returned to caller
* Note that the recursive call we are making here is safe, since it can
* logically go only one further level down.
*
- * Returns message data size or errno: -ENOMEM, -EFAULT
+ * Return: message data size or errno: -ENOMEM, -EFAULT
*/
int tipc_msg_build(struct tipc_msg *mhdr, struct msghdr *m, int offset,
int dsz, int pktmax, struct sk_buff_head *list)
* @msg: message to be appended
* @max: max allowable size for the bundle buffer
*
- * Returns "true" if bundling has been performed, otherwise "false"
+ * Return: "true" if bundling has been performed, otherwise "false"
*/
static bool tipc_msg_bundle(struct sk_buff *bskb, struct tipc_msg *msg,
u32 max)
* @skb: buffer to be extracted from.
* @iskb: extracted inner buffer, to be returned
* @pos: position in outer message of msg to be extracted.
- * Returns position of next msg
+ * Returns position of next msg.
* Consumes outer buffer when last packet extracted
- * Returns true when there is an extracted buffer, otherwise false
+ * Return: true when there is an extracted buffer, otherwise false
*/
bool tipc_msg_extract(struct sk_buff *skb, struct sk_buff **iskb, int *pos)
{
* @skb: buffer containing message to be reversed; will be consumed
* @err: error code to be set in message, if any
* Replaces consumed buffer with new one when successful
- * Returns true if success, otherwise false
+ * Return: true if success, otherwise false
*/
bool tipc_msg_reverse(u32 own_node, struct sk_buff **skb, int err)
{
/**
* tipc_msg_lookup_dest(): try to find new destination for named message
+ * @net: pointer to associated network namespace
* @skb: the buffer containing the message.
* @err: error code to be used by caller if lookup fails
* Does not consume buffer
- * Returns true if a destination is found, false otherwise
+ * Return: true if a destination is found, false otherwise
*/
bool tipc_msg_lookup_dest(struct net *net, struct sk_buff *skb, int *err)
{
/**
* publ_to_item - add publication info to a publication message
+ * @p: publication info
+ * @i: location of item in the message
*/
static void publ_to_item(struct distr_item *i, struct publication *p)
{
/**
* named_prepare_buf - allocate & initialize a publication message
+ * @net: the associated network namespace
+ * @type: message type
+ * @size: payload size
+ * @dest: destination node
*
* The buffer returned is of size INT_H_SIZE + payload size
*/
/**
* tipc_named_publish - tell other nodes about a new publication by this node
+ * @net: the associated network namespace
+ * @publ: the new publication
*/
struct sk_buff *tipc_named_publish(struct net *net, struct publication *publ)
{
/**
* tipc_named_withdraw - tell other nodes about a withdrawn publication by this node
+ * @net: the associated network namespace
+ * @publ: the withdrawn publication
*/
struct sk_buff *tipc_named_withdraw(struct net *net, struct publication *publ)
{
/**
* named_distribute - prepare name info for bulk distribution to another node
+ * @net: the associated network namespace
* @list: list of messages (buffers) to be returned from this function
* @dnode: node to be updated
* @pls: linked list of publication items to be packed into buffer chain
+ * @seqno: sequence number for this message
*/
static void named_distribute(struct net *net, struct sk_buff_head *list,
u32 dnode, struct list_head *pls, u16 seqno)
/**
* tipc_named_node_up - tell specified node about all publications by this node
+ * @net: the associated network namespace
+ * @dnode: destination node
+ * @capabilities: peer node's capabilities
*/
void tipc_named_node_up(struct net *net, u32 dnode, u16 capabilities)
{
/**
* tipc_publ_purge - remove publication associated with a failed node
+ * @net: the associated network namespace
+ * @publ: the publication to remove
+ * @addr: failed node's address
*
* Invoked for each publication issued by a newly failed node.
* Removes publication structure from name table & deletes it.
/**
* tipc_update_nametbl - try to process a nametable update and notify
* subscribers
+ * @net: the associated network namespace
+ * @i: location of item in the message
+ * @node: node address
+ * @dtype: name distributor message type
*
* tipc_nametbl_lock must be held.
- * Returns the publication item if successful, otherwise NULL.
+ * Return: the publication item if successful, otherwise NULL.
*/
static bool tipc_update_nametbl(struct net *net, struct distr_item *i,
u32 node, u32 dtype)
/**
* tipc_named_rcv - process name table update messages sent by another node
+ * @net: the associated network namespace
+ * @namedq: queue to receive from
+ * @rcv_nxt: store last received seqno here
+ * @open: last bulk msg was received (FIXME)
*/
void tipc_named_rcv(struct net *net, struct sk_buff_head *namedq,
u16 *rcv_nxt, bool *open)
/**
* tipc_named_reinit - re-initialize local publications
+ * @net: the associated network namespace
*
* This routine is called whenever TIPC networking is enabled.
* All name table entries published by this node are updated to reflect
* @type: name sequence type
* @lower: name sequence lower bound
* @upper: name sequence upper bound
- * @ref: publishing port reference
+ * @port: publishing port reference
* @key: publication key
*
* ===> All fields are stored in network byte order. <===
*
* Copyright (c) 2000-2006, 2014-2018, Ericsson AB
* Copyright (c) 2004-2008, 2010-2014, Wind River Systems
+ * Copyright (c) 2020, Red Hat Inc
* All rights reserved.
*
* Redistribution and use in source and binary forms, with or without
* range match
* @sr: the service range pointer as a loop cursor
* @sc: the pointer to tipc service which holds the service range rbtree
- * @start, end: the range (end >= start) for matching
+ * @start: beginning of the search range (end >= start) for matching
+ * @end: end of the search range (end >= start) for matching
*/
#define service_range_foreach_match(sr, sc, start, end) \
for (sr = service_range_match_first((sc)->ranges.rb_node, \
/**
* service_range_match_first - find first service range matching a range
* @n: the root node of service range rbtree for searching
- * @start, end: the range (end >= start) for matching
+ * @start: beginning of the search range (end >= start) for matching
+ * @end: end of the search range (end >= start) for matching
*
* Return: the leftmost service range node in the rbtree that overlaps the
* specific range if any. Otherwise, returns NULL.
/**
* service_range_match_next - find next service range matching a range
* @n: a node in service range rbtree from which the searching starts
- * @start, end: the range (end >= start) for matching
+ * @start: beginning of the search range (end >= start) for matching
+ * @end: end of the search range (end >= start) for matching
*
* Return: the next service range node to the given node in the rbtree that
* overlaps the specific range if any. Otherwise, returns NULL.
/**
* tipc_publ_create - create a publication structure
+ * @type: name sequence type
+ * @lower: name sequence lower bound
+ * @upper: name sequence upper bound
+ * @scope: publication scope
+ * @node: network address of publishing socket
+ * @port: publishing port
+ * @key: publication key
*/
static struct publication *tipc_publ_create(u32 type, u32 lower, u32 upper,
u32 scope, u32 node, u32 port,
/**
* tipc_service_create - create a service structure for the specified 'type'
+ * @type: service type
+ * @hd: name_table services list
*
* Allocates a single range structure and sets it to all 0's.
*/
/**
* tipc_service_remove_publ - remove a publication from a service
+ * @sr: service_range to remove publication from
+ * @node: target node
+ * @key: target publication key
*/
static struct publication *tipc_service_remove_publ(struct service_range *sr,
u32 node, u32 key)
return NULL;
}
-/**
+/*
* Code reused: time_after32() for the same purpose
*/
#define publication_after(pa, pb) time_after32((pa)->id, (pb)->id)
* tipc_service_subscribe - attach a subscription, and optionally
* issue the prescribed number of events if there is any service
* range overlapping with the requested range
+ * @service: the tipc_service to attach the @sub to
+ * @sub: the subscription to attach
*/
static void tipc_service_subscribe(struct tipc_service *service,
struct tipc_subscription *sub)
struct publication *p, *first, *tmp;
struct list_head publ_list;
struct service_range *sr;
- struct tipc_name_seq ns;
+ struct tipc_service_range r;
u32 filter;
- ns.type = tipc_sub_read(sb, seq.type);
- ns.lower = tipc_sub_read(sb, seq.lower);
- ns.upper = tipc_sub_read(sb, seq.upper);
+ r.type = tipc_sub_read(sb, seq.type);
+ r.lower = tipc_sub_read(sb, seq.lower);
+ r.upper = tipc_sub_read(sb, seq.upper);
filter = tipc_sub_read(sb, filter);
tipc_sub_get(sub);
return;
INIT_LIST_HEAD(&publ_list);
- service_range_foreach_match(sr, service, ns.lower, ns.upper) {
+ service_range_foreach_match(sr, service, r.lower, r.upper) {
first = NULL;
list_for_each_entry(p, &sr->all_publ, all_publ) {
if (filter & TIPC_SUB_PORTS)
/**
* tipc_nametbl_translate - perform service instance to socket translation
- *
- * On entry, 'dnode' is the search domain used during translation.
+ * @net: network namespace
+ * @type: message type
+ * @instance: message instance
+ * @dnode: the search domain used during translation
*
* On exit:
* - if translation is deferred to another node, leave 'dnode' unchanged and
- * return 0
+ * return 0
* - if translation is attempted and succeeds, set 'dnode' to the publishing
- * node and return the published (non-zero) port number
+ * node and return the published (non-zero) port number
* - if translation is attempted and fails, set 'dnode' to 0 and return 0
*
* Note that for legacy users (node configured with Z.C.N address format) the
/**
* tipc_nametbl_withdraw - withdraw a service binding
+ * @net: network namespace
+ * @type: service type
+ * @lower: service range lower bound
+ * @upper: service range upper bound
+ * @key: target publication key
*/
int tipc_nametbl_withdraw(struct net *net, u32 type, u32 lower,
u32 upper, u32 key)
/**
* tipc_nametbl_subscribe - add a subscription object to the name table
+ * @sub: subscription to add
*/
bool tipc_nametbl_subscribe(struct tipc_subscription *sub)
{
/**
* tipc_nametbl_unsubscribe - remove a subscription object from name table
+ * @sub: subscription to remove
*/
void tipc_nametbl_unsubscribe(struct tipc_subscription *sub)
{
}
/**
- * tipc_service_delete - purge all publications for a service and delete it
+ * tipc_service_delete - purge all publications for a service and delete it
+ * @net: the associated network namespace
+ * @sc: tipc_service to delete
*/
static void tipc_service_delete(struct net *net, struct tipc_service *sc)
{
* @key: publication key, unique across the cluster
* @id: publication id
* @binding_node: all publications from the same node which bound this one
- * - Remote publications: in node->publ_list
- * Used by node/name distr to withdraw publications when node is lost
+ * - Remote publications: in node->publ_list;
+ * Used by node/name distr to withdraw publications when node is lost
* - Local/node scope publications: in name_table->node_scope list
* - Local/cluster scope publications: in name_table->cluster_scope list
* @binding_sock: all publications from the same socket which bound this one
/**
* struct name_table - table containing all existing port name publications
- * @seq_hlist: name sequence hash lists
+ * @services: name sequence hash lists
* @node_scope: all local publications with node scope
* - used by name_distr during re-init of name table
* @cluster_scope: all local publications with cluster scope
* - used by name_distr to send bulk updates to new nodes
* - used by name_distr during re-init of name table
+ * @cluster_scope_lock: lock for accessing @cluster_scope
* @local_publ_count: number of publications issued by this node
+ * @rc_dests: destination node counter
+ * @snd_nxt: next sequence number to be used
*/
struct name_table {
struct hlist_head services[TIPC_NAMETBL_SIZE];
tipc_named_reinit(net);
tipc_sk_reinit(net);
tipc_mon_reinit_self(net);
- tipc_nametbl_publish(net, TIPC_CFG_SRV, addr, addr,
+ tipc_nametbl_publish(net, TIPC_NODE_STATE, addr, addr,
TIPC_CLUSTER_SCOPE, 0, addr);
}
link_info.dest = nla_get_flag(link[TIPC_NLA_LINK_DEST]);
link_info.up = htonl(nla_get_flag(link[TIPC_NLA_LINK_UP]));
- nla_strlcpy(link_info.str, link[TIPC_NLA_LINK_NAME],
+ nla_strscpy(link_info.str, link[TIPC_NLA_LINK_NAME],
TIPC_MAX_LINK_NAME);
return tipc_add_tlv(msg->rep, TIPC_TLV_LINK_INFO,
/**
* struct tipc_node - TIPC node structure
* @addr: network address of node
- * @ref: reference counter to node object
+ * @kref: reference counter to node object
* @lock: rwlock governing access to structure
* @net: the applicable net namespace
* @hash: links to adjacent nodes in unsorted hash chain
* @namedq: pointer to name table input queue with name table messages
* @active_links: bearer ids of active links, used as index into links[] array
* @links: array containing references to all links to node
+ * @bc_entry: broadcast link entry
* @action_flags: bit mask of different types of node actions
* @state: connectivity state vs peer node
* @preliminary: a preliminary node or not
+ * @failover_sent: failover sent or not
* @sync_point: sequence number where synch/failover is finished
* @list: links to adjacent nodes in sorted list of cluster's nodes
* @working_links: number of working links to node (both active and standby)
* @capabilities: bitmap, indicating peer node's functional capabilities
* @signature: node instance identifier
* @link_id: local and remote bearer ids of changing link, if any
+ * @peer_id: 128-bit ID of peer
+ * @peer_id_string: ID string of peer
* @publ_list: list of publications
+ * @conn_sks: list of connections (FIXME)
+ * @timer: node's keepalive timer
+ * @keepalive_intv: keepalive interval in milliseconds
* @rcu: rcu struct for tipc_node
* @delete_at: indicates the time for deleting a down node
+ * @peer_net: peer's net namespace
+ * @peer_hash_mix: hash for this peer (FIXME)
* @crypto_rx: RX crypto handler
*/
struct tipc_node {
#ifdef CONFIG_TIPC_CRYPTO
/**
* tipc_node_crypto_rx - Retrieve crypto RX handle from node
+ * @__n: target tipc_node
* Note: node ref counter must be held first!
*/
struct tipc_crypto *tipc_node_crypto_rx(struct tipc_node *__n)
/**
* __tipc_node_link_up - handle addition of link
+ * @n: target tipc_node
+ * @bearer_id: id of the bearer
+ * @xmitq: queue for messages to be xmited on
* Node lock must be held by caller
* Link becomes active (alone or shared) or standby, depending on its priority.
*/
/**
* tipc_node_link_up - handle addition of link
+ * @n: target tipc_node
+ * @bearer_id: id of the bearer
+ * @xmitq: queue for messages to be xmited on
*
* Link becomes active (alone or shared) or standby, depending on its priority.
*/
*
* This function is only called in a very special situation where link
* failover can be already started on peer node but not on this node.
- * This can happen when e.g.
+ * This can happen when e.g.::
+ *
* 1. Both links <1A-2A>, <1B-2B> down
* 2. Link endpoint 2A up, but 1A still down (e.g. due to network
- * disturbance, wrong session, etc.)
+ * disturbance, wrong session, etc.)
* 3. Link <1B-2B> up
* 4. Link endpoint 2A down (e.g. due to link tolerance timeout)
* 5. Node 2 starts failover onto link <1B-2B>
/**
* __tipc_node_link_down - handle loss of link
+ * @n: target tipc_node
+ * @bearer_id: id of the bearer
+ * @xmitq: queue for messages to be xmited on
+ * @maddr: output media address of the bearer
*/
static void __tipc_node_link_down(struct tipc_node *n, int *bearer_id,
struct sk_buff_head *xmitq,
/**
* tipc_node_get_linkname - get the name of a link
*
+ * @net: the applicable net namespace
* @bearer_id: id of the bearer
* @addr: peer node address
* @linkname: link name output buffer
+ * @len: size of @linkname output buffer
*
- * Returns 0 on success
+ * Return: 0 on success
*/
int tipc_node_get_linkname(struct net *net, u32 bearer_id, u32 addr,
char *linkname, size_t len)
* @dnode: address of destination node
* @selector: a number used for deterministic link selection
* Consumes the buffer chain.
- * Returns 0 if success, otherwise: -ELINKCONG,-EHOSTUNREACH,-EMSGSIZE,-ENOBUF
+ * Return: 0 if success, otherwise: -ELINKCONG,-EHOSTUNREACH,-EMSGSIZE,-ENOBUF
*/
int tipc_node_xmit(struct net *net, struct sk_buff_head *list,
u32 dnode, int selector)
/**
* tipc_node_check_state - check and if necessary update node state
+ * @n: target tipc_node
* @skb: TIPC packet
* @bearer_id: identity of bearer delivering the packet
- * Returns true if state and msg are ok, otherwise false
+ * @xmitq: queue for messages to be xmited on
+ * Return: true if state and msg are ok, otherwise false
*/
static bool tipc_node_check_state(struct tipc_node *n, struct sk_buff *skb,
int bearer_id, struct sk_buff_head *xmitq)
else if (prop == TIPC_NLA_PROP_MTU)
tipc_link_set_mtu(e->link, b->mtu);
}
+ /* Update MTU for node link entry */
+ e->mtu = tipc_link_mss(e->link);
tipc_node_write_unlock(n);
tipc_bearer_xmit(net, bearer_id, &xmitq, &e->maddr, NULL);
}
/*
* net/tipc/socket.c: TIPC socket API
*
- * Copyright (c) 2001-2007, 2012-2017, Ericsson AB
+ * Copyright (c) 2001-2007, 2012-2019, Ericsson AB
* Copyright (c) 2004-2008, 2010-2013, Wind River Systems
+ * Copyright (c) 2020, Red Hat Inc
* All rights reserved.
*
* Redistribution and use in source and binary forms, with or without
* @maxnagle: maximum size of msg which can be subject to nagle
* @portid: unique port identity in TIPC socket hash table
* @phdr: preformatted message header used when sending messages
- * #cong_links: list of congested links
+ * @cong_links: list of congested links
* @publications: list of publications for port
* @blocking_link: address of the congested link we are currently sleeping on
* @pub_count: total # of publications port has made during its lifetime
* @conn_timeout: the time we can wait for an unresponded setup request
+ * @probe_unacked: probe has not received ack yet
* @dupl_rcvcnt: number of bytes counted twice, in both backlog and rcv queue
* @cong_link_cnt: number of congested links
* @snt_unacked: # messages sent by socket, and not yet acked by peer
+ * @snd_win: send window size
+ * @peer_caps: peer capabilities mask
* @rcv_unacked: # messages read by user, but not yet acked back to peer
+ * @rcv_win: receive window size
* @peer: 'connected' peer for dgram/rdm
* @node: hash table node
* @mc_method: cookie for use between socket and broadcast layer
* @rcu: rcu struct for tipc_sock
+ * @group: TIPC communications group
+ * @oneway: message count in one direction (FIXME)
+ * @nagle_start: current nagle value
+ * @snd_backlog: send backlog count
+ * @msg_acc: messages accepted; used in managing backlog and nagle
+ * @pkt_cnt: TIPC socket packet count
+ * @expect_ack: whether this TIPC socket is expecting an ack
+ * @nodelay: setsockopt() TIPC_NODELAY setting
+ * @group_is_open: TIPC socket group is fully open (FIXME)
*/
struct tipc_sock {
struct sock sk;
bool kern);
static void tipc_sk_timeout(struct timer_list *t);
static int tipc_sk_publish(struct tipc_sock *tsk, uint scope,
- struct tipc_name_seq const *seq);
+ struct tipc_service_range const *seq);
static int tipc_sk_withdraw(struct tipc_sock *tsk, uint scope,
- struct tipc_name_seq const *seq);
+ struct tipc_service_range const *seq);
static int tipc_sk_leave(struct tipc_sock *tsk);
static struct tipc_sock *tipc_sk_lookup(struct net *net, u32 portid);
static int tipc_sk_insert(struct tipc_sock *tsk);
/**
* tsk_advance_rx_queue - discard first buffer in socket receive queue
+ * @sk: network socket
*
* Caller must hold socket lock
*/
/**
* tsk_rej_rx_queue - reject all buffers in socket receive queue
+ * @sk: network socket
+ * @error: response error code
*
* Caller must hold socket lock
*/
* This routine creates additional data structures used by the TIPC socket,
* initializes them, and links them together.
*
- * Returns 0 on success, errno otherwise
+ * Return: 0 on success, errno otherwise
*/
static int tipc_sk_create(struct net *net, struct socket *sock,
int protocol, int kern)
* are returned or discarded according to the "destination droppable" setting
* specified for the message by the sender.
*
- * Returns 0 on success, errno otherwise
+ * Return: 0 on success, errno otherwise
*/
static int tipc_release(struct socket *sock)
{
}
/**
- * tipc_bind - associate or disassocate TIPC name(s) with a socket
+ * __tipc_bind - associate or disassocate TIPC name(s) with a socket
* @sock: socket structure
- * @uaddr: socket address describing name(s) and desired operation
- * @uaddr_len: size of socket address data structure
+ * @skaddr: socket address describing name(s) and desired operation
+ * @alen: size of socket address data structure
*
* Name and name sequence binding is indicated using a positive scope value;
* a negative scope value unbinds the specified name. Specifying no name
* (i.e. a socket address length of 0) unbinds all names from the socket.
*
- * Returns 0 on success, errno otherwise
+ * Return: 0 on success, errno otherwise
*
* NOTE: This routine doesn't need to take the socket lock since it doesn't
* access any non-constant socket information.
*/
-
-int tipc_sk_bind(struct socket *sock, struct sockaddr *uaddr, int uaddr_len)
+static int __tipc_bind(struct socket *sock, struct sockaddr *skaddr, int alen)
{
- struct sock *sk = sock->sk;
- struct sockaddr_tipc *addr = (struct sockaddr_tipc *)uaddr;
- struct tipc_sock *tsk = tipc_sk(sk);
- int res = -EINVAL;
+ struct sockaddr_tipc *addr = (struct sockaddr_tipc *)skaddr;
+ struct tipc_sock *tsk = tipc_sk(sock->sk);
- lock_sock(sk);
- if (unlikely(!uaddr_len)) {
- res = tipc_sk_withdraw(tsk, 0, NULL);
- goto exit;
- }
- if (tsk->group) {
- res = -EACCES;
- goto exit;
- }
- if (uaddr_len < sizeof(struct sockaddr_tipc)) {
- res = -EINVAL;
- goto exit;
- }
- if (addr->family != AF_TIPC) {
- res = -EAFNOSUPPORT;
- goto exit;
- }
+ if (unlikely(!alen))
+ return tipc_sk_withdraw(tsk, 0, NULL);
- if (addr->addrtype == TIPC_ADDR_NAME)
+ if (addr->addrtype == TIPC_SERVICE_ADDR)
addr->addr.nameseq.upper = addr->addr.nameseq.lower;
- else if (addr->addrtype != TIPC_ADDR_NAMESEQ) {
- res = -EAFNOSUPPORT;
- goto exit;
- }
- res = (addr->scope >= 0) ?
- tipc_sk_publish(tsk, addr->scope, &addr->addr.nameseq) :
- tipc_sk_withdraw(tsk, -addr->scope, &addr->addr.nameseq);
-exit:
- release_sock(sk);
+ if (tsk->group)
+ return -EACCES;
+
+ if (addr->scope >= 0)
+ return tipc_sk_publish(tsk, addr->scope, &addr->addr.nameseq);
+ else
+ return tipc_sk_withdraw(tsk, -addr->scope, &addr->addr.nameseq);
+}
+
+int tipc_sk_bind(struct socket *sock, struct sockaddr *skaddr, int alen)
+{
+ int res;
+
+ lock_sock(sock->sk);
+ res = __tipc_bind(sock, skaddr, alen);
+ release_sock(sock->sk);
return res;
}
if (alen) {
if (alen < sizeof(struct sockaddr_tipc))
return -EINVAL;
+ if (addr->family != AF_TIPC)
+ return -EAFNOSUPPORT;
+ if (addr->addrtype > TIPC_SERVICE_ADDR)
+ return -EAFNOSUPPORT;
if (addr->addr.nameseq.type < TIPC_RESERVED_TYPES) {
pr_warn_once("Can't bind to reserved service type %u\n",
addr->addr.nameseq.type);
* @uaddr: area for returned socket address
* @peer: 0 = own ID, 1 = current peer ID, 2 = current/former peer ID
*
- * Returns 0 on success, errno otherwise
+ * Return: 0 on success, errno otherwise
*
* NOTE: This routine doesn't need to take the socket lock since it only
* accesses socket information that is unchanging (or which changes in
addr->addr.id.node = tipc_own_addr(sock_net(sk));
}
- addr->addrtype = TIPC_ADDR_ID;
+ addr->addrtype = TIPC_SOCKET_ADDR;
addr->family = AF_TIPC;
addr->scope = 0;
addr->addr.name.domain = 0;
* @sock: socket for which to calculate the poll bits
* @wait: ???
*
- * Returns pollmask value
+ * Return: pollmask value
*
* COMMENTARY:
* It appears that the usual socket locking mechanisms are not useful here
* @timeout: timeout to wait for wakeup
*
* Called from function tipc_sendmsg(), which has done all sanity checks
- * Returns the number of bytes sent on success, or errno
+ * Return: the number of bytes sent on success, or errno
*/
-static int tipc_sendmcast(struct socket *sock, struct tipc_name_seq *seq,
+static int tipc_sendmcast(struct socket *sock, struct tipc_service_range *seq,
struct msghdr *msg, size_t dlen, long timeout)
{
struct sock *sk = sock->sk;
/**
* tipc_send_group_msg - send a message to a member in the group
* @net: network namespace
+ * @tsk: tipc socket
* @m: message to send
* @mb: group member
* @dnode: destination node
* @timeout: timeout to wait for wakeup
*
* Called from function tipc_sendmsg(), which has done all sanity checks
- * Returns the number of bytes sent on success, or errno
+ * Return: the number of bytes sent on success, or errno
*/
static int tipc_send_group_unicast(struct socket *sock, struct msghdr *m,
int dlen, long timeout)
* @timeout: timeout to wait for wakeup
*
* Called from function tipc_sendmsg(), which has done all sanity checks
- * Returns the number of bytes sent on success, or errno
+ * Return: the number of bytes sent on success, or errno
*/
static int tipc_send_group_anycast(struct socket *sock, struct msghdr *m,
int dlen, long timeout)
* @timeout: timeout to wait for wakeup
*
* Called from function tipc_sendmsg(), which has done all sanity checks
- * Returns the number of bytes sent on success, or errno
+ * Return: the number of bytes sent on success, or errno
*/
static int tipc_send_group_bcast(struct socket *sock, struct msghdr *m,
int dlen, long timeout)
* @timeout: timeout to wait for wakeup
*
* Called from function tipc_sendmsg(), which has done all sanity checks
- * Returns the number of bytes sent on success, or errno
+ * Return: the number of bytes sent on success, or errno
*/
static int tipc_send_group_mcast(struct socket *sock, struct msghdr *m,
int dlen, long timeout)
/**
* tipc_sk_mcast_rcv - Deliver multicast messages to all destination sockets
+ * @net: the associated network namespace
* @arrvq: queue with arriving messages, to be cloned after destination lookup
* @inputq: queue with cloned messages, delivered to socket after dest lookup
*
* tipc_sk_conn_proto_rcv - receive a connection mng protocol message
* @tsk: receiving socket
* @skb: pointer to message buffer.
+ * @inputq: buffer list containing the buffers
+ * @xmitq: output message area
*/
static void tipc_sk_conn_proto_rcv(struct tipc_sock *tsk, struct sk_buff *skb,
struct sk_buff_head *inputq,
* and for 'SYN' messages on SOCK_SEQPACKET and SOCK_STREAM connections.
* (Note: 'SYN+' is prohibited on SOCK_STREAM.)
*
- * Returns the number of bytes sent on success, or errno otherwise
+ * Return: the number of bytes sent on success, or errno otherwise
*/
static int tipc_sendmsg(struct socket *sock,
struct msghdr *m, size_t dsz)
bool syn = !tipc_sk_type_connectionless(sk);
struct tipc_group *grp = tsk->group;
struct tipc_msg *hdr = &tsk->phdr;
- struct tipc_name_seq *seq;
+ struct tipc_service_range *seq;
struct sk_buff_head pkts;
u32 dport = 0, dnode = 0;
u32 type = 0, inst = 0;
if (grp) {
if (!dest)
return tipc_send_group_bcast(sock, m, dlen, timeout);
- if (dest->addrtype == TIPC_ADDR_NAME)
+ if (dest->addrtype == TIPC_SERVICE_ADDR)
return tipc_send_group_anycast(sock, m, dlen, timeout);
- if (dest->addrtype == TIPC_ADDR_ID)
+ if (dest->addrtype == TIPC_SOCKET_ADDR)
return tipc_send_group_unicast(sock, m, dlen, timeout);
if (dest->addrtype == TIPC_ADDR_MCAST)
return tipc_send_group_mcast(sock, m, dlen, timeout);
return -EISCONN;
if (tsk->published)
return -EOPNOTSUPP;
- if (dest->addrtype == TIPC_ADDR_NAME) {
+ if (dest->addrtype == TIPC_SERVICE_ADDR) {
tsk->conn_type = dest->addr.name.name.type;
tsk->conn_instance = dest->addr.name.name.instance;
}
if (dest->addrtype == TIPC_ADDR_MCAST)
return tipc_sendmcast(sock, seq, m, dlen, timeout);
- if (dest->addrtype == TIPC_ADDR_NAME) {
+ if (dest->addrtype == TIPC_SERVICE_ADDR) {
type = dest->addr.name.name.type;
inst = dest->addr.name.name.instance;
dnode = dest->addr.name.domain;
dport = tipc_nametbl_translate(net, type, inst, &dnode);
if (unlikely(!dport && !dnode))
return -EHOSTUNREACH;
- } else if (dest->addrtype == TIPC_ADDR_ID) {
+ } else if (dest->addrtype == TIPC_SOCKET_ADDR) {
dnode = dest->addr.id.node;
} else {
return -EINVAL;
if (unlikely(rc))
return rc;
- if (dest->addrtype == TIPC_ADDR_NAME) {
+ if (dest->addrtype == TIPC_SERVICE_ADDR) {
msg_set_type(hdr, TIPC_NAMED_MSG);
msg_set_hdr_sz(hdr, NAMED_H_SIZE);
msg_set_nametype(hdr, type);
msg_set_lookup_scope(hdr, tipc_node2scope(dnode));
msg_set_destnode(hdr, dnode);
msg_set_destport(hdr, dport);
- } else { /* TIPC_ADDR_ID */
+ } else { /* TIPC_SOCKET_ADDR */
msg_set_type(hdr, TIPC_DIRECT_MSG);
msg_set_lookup_scope(hdr, 0);
msg_set_destnode(hdr, dnode);
*
* Used for SOCK_STREAM data.
*
- * Returns the number of bytes sent on success (or partial success),
+ * Return: the number of bytes sent on success (or partial success),
* or errno if no data sent
*/
static int tipc_sendstream(struct socket *sock, struct msghdr *m, size_t dsz)
*
* Used for SOCK_SEQPACKET messages.
*
- * Returns the number of bytes sent on success, or errno otherwise
+ * Return: the number of bytes sent on success, or errno otherwise
*/
static int tipc_send_packet(struct socket *sock, struct msghdr *m, size_t dsz)
{
return;
srcaddr->sock.family = AF_TIPC;
- srcaddr->sock.addrtype = TIPC_ADDR_ID;
+ srcaddr->sock.addrtype = TIPC_SOCKET_ADDR;
srcaddr->sock.scope = 0;
srcaddr->sock.addr.id.ref = msg_origport(hdr);
srcaddr->sock.addr.id.node = msg_orignode(hdr);
/* Group message users may also want to know sending member's id */
srcaddr->member.family = AF_TIPC;
- srcaddr->member.addrtype = TIPC_ADDR_NAME;
+ srcaddr->member.addrtype = TIPC_SERVICE_ADDR;
srcaddr->member.scope = 0;
srcaddr->member.addr.name.name.type = msg_nametype(hdr);
srcaddr->member.addr.name.name.instance = TIPC_SKB_CB(skb)->orig_member;
*
* Note: Ancillary data is not captured if not requested by receiver.
*
- * Returns 0 if successful, otherwise errno
+ * Return: 0 if successful, otherwise errno
*/
static int tipc_sk_anc_data_recv(struct msghdr *m, struct sk_buff *skb,
struct tipc_sock *tsk)
/**
* tipc_recvmsg - receive packet-oriented message
+ * @sock: network socket
* @m: descriptor for message info
* @buflen: length of user buffer area
* @flags: receive flags
* Used for SOCK_DGRAM, SOCK_RDM, and SOCK_SEQPACKET messages.
* If the complete message doesn't fit in user area, truncate it.
*
- * Returns size of returned message data, errno otherwise
+ * Return: size of returned message data, errno otherwise
*/
static int tipc_recvmsg(struct socket *sock, struct msghdr *m,
size_t buflen, int flags)
/**
* tipc_recvstream - receive stream-oriented data
+ * @sock: network socket
* @m: descriptor for message info
* @buflen: total size of user buffer area
* @flags: receive flags
* Used for SOCK_STREAM messages only. If not enough data is available
* will optionally wait for more; never truncates data.
*
- * Returns size of returned message data, errno otherwise
+ * Return: size of returned message data, errno otherwise
*/
static int tipc_recvstream(struct socket *sock, struct msghdr *m,
size_t buflen, int flags)
* @tsk: TIPC socket
* @skb: pointer to message buffer.
* @xmitq: for Nagle ACK if any
- * Returns true if message should be added to receive queue, false otherwise
+ * Return: true if message should be added to receive queue, false otherwise
*/
static bool tipc_sk_filter_connect(struct tipc_sock *tsk, struct sk_buff *skb,
struct sk_buff_head *xmitq)
* TIPC_HIGH_IMPORTANCE (8 MB)
* TIPC_CRITICAL_IMPORTANCE (16 MB)
*
- * Returns overload limit according to corresponding message importance
+ * Return: overload limit according to corresponding message importance
*/
static unsigned int rcvbuf_limit(struct sock *sk, struct sk_buff *skb)
{
* tipc_sk_filter_rcv - validate incoming message
* @sk: socket
* @skb: pointer to message.
+ * @xmitq: output message area (FIXME)
*
* Enqueues message on receive queue if acceptable; optionally handles
* disconnect indication for a connected socket.
*
* Called with socket lock already taken
- *
*/
static void tipc_sk_filter_rcv(struct sock *sk, struct sk_buff *skb,
struct sk_buff_head *xmitq)
* @inputq: list of incoming buffers with potentially different destinations
* @sk: socket where the buffers should be enqueued
* @dport: port number for the socket
+ * @xmitq: output queue
*
* Caller must hold socket lock
*/
/**
* tipc_sk_rcv - handle a chain of incoming buffers
+ * @net: the associated network namespace
* @inputq: buffer list containing the buffers
* Consumes all buffers in list until inputq is empty
* Note: may be called in multiple threads referring to the same queue
* @destlen: size of socket address data structure
* @flags: file-related flags associated with socket
*
- * Returns 0 on success, errno otherwise
+ * Return: 0 on success, errno otherwise
*/
static int tipc_connect(struct socket *sock, struct sockaddr *dest,
int destlen, int flags)
* @sock: socket structure
* @len: (unused)
*
- * Returns 0 on success, errno otherwise
+ * Return: 0 on success, errno otherwise
*/
static int tipc_listen(struct socket *sock, int len)
{
* @sock: listening socket
* @new_sock: new socket that is to be connected
* @flags: file-related flags associated with socket
+ * @kern: caused by kernel or by userspace?
*
- * Returns 0 on success, errno otherwise
+ * Return: 0 on success, errno otherwise
*/
static int tipc_accept(struct socket *sock, struct socket *new_sock, int flags,
bool kern)
*
* Terminates connection (if necessary), then purges socket's receive queue.
*
- * Returns 0 on success, errno otherwise
+ * Return: 0 on success, errno otherwise
*/
static int tipc_shutdown(struct socket *sock, int how)
{
}
static int tipc_sk_publish(struct tipc_sock *tsk, uint scope,
- struct tipc_name_seq const *seq)
+ struct tipc_service_range const *seq)
{
struct sock *sk = &tsk->sk;
struct net *net = sock_net(sk);
}
static int tipc_sk_withdraw(struct tipc_sock *tsk, uint scope,
- struct tipc_name_seq const *seq)
+ struct tipc_service_range const *seq)
{
struct net *net = sock_net(&tsk->sk);
struct publication *publ;
struct net *net = sock_net(&tsk->sk);
struct tipc_group *grp = tsk->group;
struct tipc_msg *hdr = &tsk->phdr;
- struct tipc_name_seq seq;
+ struct tipc_service_range seq;
int rc;
if (mreq->type < TIPC_RESERVED_TYPES)
{
struct net *net = sock_net(&tsk->sk);
struct tipc_group *grp = tsk->group;
- struct tipc_name_seq seq;
+ struct tipc_service_range seq;
int scope;
if (!grp)
* For stream sockets only, accepts and ignores all IPPROTO_TCP options
* (to ease compatibility).
*
- * Returns 0 on success, errno otherwise
+ * Return: 0 on success, errno otherwise
*/
static int tipc_setsockopt(struct socket *sock, int lvl, int opt,
sockptr_t ov, unsigned int ol)
* For stream sockets only, returns 0 length result for all IPPROTO_TCP options
* (to ease compatibility).
*
- * Returns 0 on success, errno otherwise
+ * Return: 0 on success, errno otherwise
*/
static int tipc_getsockopt(struct socket *sock, int lvl, int opt,
char __user *ov, int __user *ol)
{
struct sock *sk = sock->sk;
struct tipc_sock *tsk = tipc_sk(sk);
- struct tipc_name_seq seq;
+ struct tipc_service_range seq;
int len, scope;
u32 value;
int res;
u32 onode = tipc_own_addr(sock_net(sock1->sk));
tsk1->peer.family = AF_TIPC;
- tsk1->peer.addrtype = TIPC_ADDR_ID;
+ tsk1->peer.addrtype = TIPC_SOCKET_ADDR;
tsk1->peer.scope = TIPC_NODE_SCOPE;
tsk1->peer.addr.id.ref = tsk2->portid;
tsk1->peer.addr.id.node = onode;
tsk2->peer.family = AF_TIPC;
- tsk2->peer.addrtype = TIPC_ADDR_ID;
+ tsk2->peer.addrtype = TIPC_SOCKET_ADDR;
tsk2->peer.scope = TIPC_NODE_SCOPE;
tsk2->peer.addr.id.ref = tsk1->portid;
tsk2->peer.addr.id.node = onode;
/**
* tipc_socket_init - initialize TIPC socket interface
*
- * Returns 0 on success, errno otherwise
+ * Return: 0 on success, errno otherwise
*/
int tipc_socket_init(void)
{
/**
* tipc_sk_filtering - check if a socket should be traced
* @sk: the socket to be examined
- * @sysctl_tipc_sk_filter[]: the socket tuple for filtering,
- * (portid, sock type, name type, name lower, name upper)
*
- * Returns true if the socket meets the socket tuple data
+ * @sysctl_tipc_sk_filter is used as the socket tuple for filtering:
+ * (portid, sock type, name type, name lower, name upper)
+ *
+ * Return: true if the socket meets the socket tuple data
* (value 0 = 'any') or when there is no tuple set (all = 0),
* otherwise false
*/
* @sk: tipc sk to be checked
* @skb: tipc msg to be checked
*
- * Returns true if the socket rx queue allocation is > 90%, otherwise false
+ * Return: true if the socket rx queue allocation is > 90%, otherwise false
*/
bool tipc_sk_overlimit1(struct sock *sk, struct sk_buff *skb)
* @sk: tipc sk to be checked
* @skb: tipc msg to be checked
*
- * Returns true if the socket rx queue allocation is > 90%, otherwise false
+ * Return: true if the socket rx queue allocation is > 90%, otherwise false
*/
bool tipc_sk_overlimit2(struct sock *sk, struct sk_buff *skb)
*
* Copyright (c) 2000-2017, Ericsson AB
* Copyright (c) 2005-2007, 2010-2013, Wind River Systems
+ * Copyright (c) 2020, Red Hat Inc
* All rights reserved.
*
* Redistribution and use in source and binary forms, with or without
}
/**
- * tipc_sub_check_overlap - test for subscription overlap with the
- * given values
+ * tipc_sub_check_overlap - test for subscription overlap with the given values
+ * @seq: tipc_name_seq to check
+ * @found_lower: lower value to test
+ * @found_upper: upper value to test
*
- * Returns 1 if there is overlap, otherwise 0.
+ * Return: 1 if there is overlap, otherwise 0.
*/
-int tipc_sub_check_overlap(struct tipc_name_seq *seq, u32 found_lower,
+int tipc_sub_check_overlap(struct tipc_service_range *seq, u32 found_lower,
u32 found_upper)
{
if (found_lower < seq->lower)
{
struct tipc_subscr *s = &sub->evt.s;
u32 filter = tipc_sub_read(s, filter);
- struct tipc_name_seq seq;
+ struct tipc_service_range seq;
seq.type = tipc_sub_read(s, seq.type);
seq.lower = tipc_sub_read(s, seq.lower);
*
* Copyright (c) 2003-2017, Ericsson AB
* Copyright (c) 2005-2007, 2012-2013, Wind River Systems
+ * Copyright (c) 2020, Red Hat Inc
* All rights reserved.
*
* Redistribution and use in source and binary forms, with or without
/**
* struct tipc_subscription - TIPC network topology subscription object
- * @subscriber: pointer to its subscriber
- * @seq: name sequence associated with subscription
+ * @kref: reference count for this subscription
+ * @net: network namespace associated with subscription
* @timer: timer governing subscription duration (optional)
- * @nameseq_list: adjacent subscriptions in name sequence's subscription list
+ * @service_list: adjacent subscriptions in name sequence's subscription list
* @sub_list: adjacent subscriptions in subscriber's subscription list
* @evt: template for events generated by subscription
+ * @conid: connection identifier of topology server
+ * @inactive: true if this subscription is inactive
+ * @lock: serialize up/down and timer events
*/
struct tipc_subscription {
struct kref kref;
struct tipc_event evt;
int conid;
bool inactive;
- spinlock_t lock; /* serialize up/down and timer events */
+ spinlock_t lock;
};
struct tipc_subscription *tipc_sub_subscribe(struct net *net,
int conid);
void tipc_sub_unsubscribe(struct tipc_subscription *sub);
-int tipc_sub_check_overlap(struct tipc_name_seq *seq, u32 found_lower,
- u32 found_upper);
+int tipc_sub_check_overlap(struct tipc_service_range *seq,
+ u32 found_lower, u32 found_upper);
void tipc_sub_report_overlap(struct tipc_subscription *sub,
u32 found_lower, u32 found_upper,
u32 event, u32 port, u32 node,
goto err;
saddr.family = AF_TIPC;
- saddr.addrtype = TIPC_ADDR_NAMESEQ;
- saddr.addr.nameseq.type = TIPC_TOP_SRV;
+ saddr.addrtype = TIPC_SERVICE_RANGE;
+ saddr.addr.nameseq.type = TIPC_TOP_SRV;
saddr.addr.nameseq.lower = TIPC_TOP_SRV;
saddr.addr.nameseq.upper = TIPC_TOP_SRV;
saddr.scope = TIPC_NODE_SCOPE;
#define CREATE_TRACE_POINTS
#include "trace.h"
-/**
+/*
* socket tuples for filtering in socket traces:
* (portid, sock type, name type, name lower, name upper)
*/
*
* This is the bearer level originating address used in neighbor discovery
* messages, and all fields should be in network byte order
+ *
+ * @proto: Ethernet protocol in use
+ * @port: port being used
+ * @ipv4: IPv4 address of neighbor
+ * @ipv6: IPv6 address of neighbor
*/
struct udp_media_addr {
__be16 proto;
* @ubsock: bearer associated socket
* @ifindex: local address scope
* @work: used to schedule deferred work on a bearer
+ * @rcast: associated udp_replicast container
*/
struct udp_bearer {
struct tipc_bearer __rcu *bearer;
if (err)
goto free;
- /**
+ /*
* The bcast media address port is used for all peers and the ip
* is used if it's a multicast address.
*/
/* fill prepend */
tls_fill_prepend(ctx, skb_frag_address(&record->frags[0]),
record->len - prot->overhead_size,
- record_type, prot->version);
+ record_type);
return ret;
}
static bool
tls_device_rx_resync_async(struct tls_offload_resync_async *resync_async,
- s64 resync_req, u32 *seq)
+ s64 resync_req, u32 *seq, u16 *rcd_delta)
{
u32 is_async = resync_req & RESYNC_REQ_ASYNC;
u32 req_seq = resync_req >> 32;
u32 req_end = req_seq + ((resync_req >> 16) & 0xffff);
+ u16 i;
+
+ *rcd_delta = 0;
if (is_async) {
+ /* shouldn't get to wraparound:
+ * too long in async stage, something bad happened
+ */
+ if (WARN_ON_ONCE(resync_async->rcd_delta == USHRT_MAX))
+ return false;
+
/* asynchronous stage: log all headers seq such that
* req_seq <= seq <= end_seq, and wait for real resync request
*/
- if (between(*seq, req_seq, req_end) &&
+ if (before(*seq, req_seq))
+ return false;
+ if (!after(*seq, req_end) &&
resync_async->loglen < TLS_DEVICE_RESYNC_ASYNC_LOGMAX)
resync_async->log[resync_async->loglen++] = *seq;
+ resync_async->rcd_delta++;
+
return false;
}
/* synchronous stage: check against the logged entries and
* proceed to check the next entries if no match was found
*/
- while (resync_async->loglen) {
- if (req_seq == resync_async->log[resync_async->loglen - 1] &&
- atomic64_try_cmpxchg(&resync_async->req,
- &resync_req, 0)) {
- resync_async->loglen = 0;
+ for (i = 0; i < resync_async->loglen; i++)
+ if (req_seq == resync_async->log[i] &&
+ atomic64_try_cmpxchg(&resync_async->req, &resync_req, 0)) {
+ *rcd_delta = resync_async->rcd_delta - i;
*seq = req_seq;
+ resync_async->loglen = 0;
+ resync_async->rcd_delta = 0;
return true;
}
- resync_async->loglen--;
- }
+
+ resync_async->loglen = 0;
+ resync_async->rcd_delta = 0;
if (req_seq == *seq &&
atomic64_try_cmpxchg(&resync_async->req,
u32 sock_data, is_req_pending;
struct tls_prot_info *prot;
s64 resync_req;
+ u16 rcd_delta;
u32 req_seq;
if (tls_ctx->rx_conf != TLS_HW)
return;
if (!tls_device_rx_resync_async(rx_ctx->resync_async,
- resync_req, &seq))
+ resync_req, &seq, &rcd_delta))
return;
+ tls_bigint_subtract(rcd_sn, rcd_delta);
break;
}
int tls_set_device_offload(struct sock *sk, struct tls_context *ctx)
{
- u16 nonce_size, tag_size, iv_size, rec_seq_size;
+ u16 nonce_size, tag_size, iv_size, rec_seq_size, salt_size;
struct tls_context *tls_ctx = tls_get_ctx(sk);
struct tls_prot_info *prot = &tls_ctx->prot_info;
struct tls_record_info *start_marker_record;
iv_size = TLS_CIPHER_AES_GCM_128_IV_SIZE;
iv = ((struct tls12_crypto_info_aes_gcm_128 *)crypto_info)->iv;
rec_seq_size = TLS_CIPHER_AES_GCM_128_REC_SEQ_SIZE;
+ salt_size = TLS_CIPHER_AES_GCM_128_SALT_SIZE;
rec_seq =
((struct tls12_crypto_info_aes_gcm_128 *)crypto_info)->rec_seq;
break;
prot->tag_size = tag_size;
prot->overhead_size = prot->prepend_size + prot->tag_size;
prot->iv_size = iv_size;
+ prot->salt_size = salt_size;
ctx->tx.iv = kmalloc(iv_size + TLS_CIPHER_AES_GCM_128_SALT_SIZE,
GFP_KERNEL);
if (!ctx->tx.iv) {
if (tls_ctx->tx_conf != TLS_HW) {
dev_put(netdev);
tls_ctx->netdev = NULL;
+ } else {
+ set_bit(TLS_RX_DEV_CLOSED, &tls_ctx->flags);
}
out:
up_read(&device_offload_lock);
if (ctx->tx_conf == TLS_HW)
netdev->tlsdev_ops->tls_dev_del(netdev, ctx,
TLS_OFFLOAD_CTX_DIR_TX);
- if (ctx->rx_conf == TLS_HW)
+ if (ctx->rx_conf == TLS_HW &&
+ !test_bit(TLS_RX_DEV_CLOSED, &ctx->flags))
netdev->tlsdev_ops->tls_dev_del(netdev, ctx,
TLS_OFFLOAD_CTX_DIR_RX);
WRITE_ONCE(ctx->netdev, NULL);
struct crypto_aead *aead, char *aad,
char *iv, __be64 rcd_sn,
struct scatter_walk *in,
- struct scatter_walk *out, int *in_len)
+ struct scatter_walk *out, int *in_len,
+ struct tls_prot_info *prot)
{
unsigned char buf[TLS_HEADER_SIZE + TLS_CIPHER_AES_GCM_128_IV_SIZE];
struct scatterlist sg_in[3];
len -= TLS_CIPHER_AES_GCM_128_IV_SIZE;
tls_make_aad(aad, len - TLS_CIPHER_AES_GCM_128_TAG_SIZE,
- (char *)&rcd_sn, sizeof(rcd_sn), buf[0],
- TLS_1_2_VERSION);
+ (char *)&rcd_sn, buf[0], prot);
memcpy(iv + TLS_CIPHER_AES_GCM_128_SALT_SIZE, buf + TLS_HEADER_SIZE,
TLS_CIPHER_AES_GCM_128_IV_SIZE);
static int tls_enc_records(struct aead_request *aead_req,
struct crypto_aead *aead, struct scatterlist *sg_in,
struct scatterlist *sg_out, char *aad, char *iv,
- u64 rcd_sn, int len)
+ u64 rcd_sn, int len, struct tls_prot_info *prot)
{
struct scatter_walk out, in;
int rc;
do {
rc = tls_enc_record(aead_req, aead, aad, iv,
- cpu_to_be64(rcd_sn), &in, &out, &len);
+ cpu_to_be64(rcd_sn), &in, &out, &len, prot);
rcd_sn++;
} while (rc == 0 && len);
payload_len, sync_size, dummy_buf);
if (tls_enc_records(aead_req, ctx->aead_send, sg_in, sg_out, aad, iv,
- rcd_sn, sync_size + payload_len) < 0)
+ rcd_sn, sync_size + payload_len,
+ &tls_ctx->prot_info) < 0)
goto free_nskb;
complete_skb(nskb, skb, tcp_payload_offset);
case TLS_CIPHER_AES_CCM_128:
optsize = sizeof(struct tls12_crypto_info_aes_ccm_128);
break;
+ case TLS_CIPHER_CHACHA20_POLY1305:
+ optsize = sizeof(struct tls12_crypto_info_chacha20_poly1305);
+ break;
default:
rc = -EINVAL;
goto err_crypto_info;
memcpy(&rec->iv_data[iv_offset], tls_ctx->tx.iv,
prot->iv_size + prot->salt_size);
- xor_iv_with_seq(prot->version, rec->iv_data, tls_ctx->tx.rec_seq);
+ xor_iv_with_seq(prot, rec->iv_data, tls_ctx->tx.rec_seq);
sge->offset += prot->prepend_size;
sge->length -= prot->prepend_size;
sg_chain(rec->sg_aead_out, 2, &msg_en->sg.data[i]);
tls_make_aad(rec->aad_space, msg_pl->sg.size + prot->tail_size,
- tls_ctx->tx.rec_seq, prot->rec_seq_size,
- record_type, prot->version);
+ tls_ctx->tx.rec_seq, record_type, prot);
tls_fill_prepend(tls_ctx,
page_address(sg_page(&msg_en->sg.data[i])) +
msg_en->sg.data[i].offset,
msg_pl->sg.size + prot->tail_size,
- record_type, prot->version);
+ record_type);
tls_ctx->pending_open_record_frags = false;
return NULL;
}
+ if (!skb_queue_empty(&sk->sk_receive_queue)) {
+ __strp_unpause(&ctx->strp);
+ if (ctx->recv_pkt)
+ return ctx->recv_pkt;
+ }
+
if (sk->sk_shutdown & RCV_SHUTDOWN)
return NULL;
kfree(mem);
return err;
}
- if (prot->version == TLS_1_3_VERSION)
+ if (prot->version == TLS_1_3_VERSION ||
+ prot->cipher_type == TLS_CIPHER_CHACHA20_POLY1305)
memcpy(iv + iv_offset, tls_ctx->rx.iv,
crypto_aead_ivsize(ctx->aead_recv));
else
memcpy(iv + iv_offset, tls_ctx->rx.iv, prot->salt_size);
- xor_iv_with_seq(prot->version, iv, tls_ctx->rx.rec_seq);
+ xor_iv_with_seq(prot, iv, tls_ctx->rx.rec_seq);
/* Prepare AAD */
tls_make_aad(aad, rxm->full_len - prot->overhead_size +
prot->tail_size,
- tls_ctx->rx.rec_seq, prot->rec_seq_size,
- ctx->control, prot->version);
+ tls_ctx->rx.rec_seq, ctx->control, prot);
/* Prepare sgin */
sg_init_table(sgin, n_sgin);
* another message type
*/
msg->msg_flags |= MSG_EOR;
- if (ctx->control != TLS_RECORD_TYPE_DATA)
+ if (control != TLS_RECORD_TYPE_DATA)
goto recv_end;
} else {
break;
data_len = ((header[4] & 0xFF) | (header[3] << 8));
cipher_overhead = prot->tag_size;
- if (prot->version != TLS_1_3_VERSION)
+ if (prot->version != TLS_1_3_VERSION &&
+ prot->cipher_type != TLS_CIPHER_CHACHA20_POLY1305)
cipher_overhead += prot->iv_size;
if (data_len > TLS_MAX_PAYLOAD_SIZE + cipher_overhead +
struct tls12_crypto_info_aes_gcm_128 *gcm_128_info;
struct tls12_crypto_info_aes_gcm_256 *gcm_256_info;
struct tls12_crypto_info_aes_ccm_128 *ccm_128_info;
+ struct tls12_crypto_info_chacha20_poly1305 *chacha20_poly1305_info;
struct tls_sw_context_tx *sw_ctx_tx = NULL;
struct tls_sw_context_rx *sw_ctx_rx = NULL;
struct cipher_context *cctx;
cipher_name = "ccm(aes)";
break;
}
+ case TLS_CIPHER_CHACHA20_POLY1305: {
+ chacha20_poly1305_info = (void *)crypto_info;
+ nonce_size = 0;
+ tag_size = TLS_CIPHER_CHACHA20_POLY1305_TAG_SIZE;
+ iv_size = TLS_CIPHER_CHACHA20_POLY1305_IV_SIZE;
+ iv = chacha20_poly1305_info->iv;
+ rec_seq_size = TLS_CIPHER_CHACHA20_POLY1305_REC_SEQ_SIZE;
+ rec_seq = chacha20_poly1305_info->rec_seq;
+ keysize = TLS_CIPHER_CHACHA20_POLY1305_KEY_SIZE;
+ key = chacha20_poly1305_info->key;
+ salt = chacha20_poly1305_info->salt;
+ salt_size = TLS_CIPHER_CHACHA20_POLY1305_SALT_SIZE;
+ cipher_name = "rfc7539(chacha20,poly1305)";
+ break;
+ }
default:
rc = -EINVAL;
goto free_priv;
case SOCK_STREAM:
if (vsock_use_local_transport(remote_cid))
new_transport = transport_local;
- else if (remote_cid <= VMADDR_CID_HOST)
+ else if (remote_cid <= VMADDR_CID_HOST || !transport_h2g)
new_transport = transport_g2h;
else
new_transport = transport_h2g;
virtio_transport_free_pkt(pkt);
}
- if (remove_sock)
+ if (remove_sock) {
+ sock_set_flag(sk, SOCK_DONE);
vsock_remove_sock(vsk);
+ }
}
EXPORT_SYMBOL_GPL(virtio_transport_release);
lock_sock(sk);
- /* Check if sk has been released before lock_sock */
- if (sk->sk_shutdown == SHUTDOWN_MASK) {
+ /* Check if sk has been closed before lock_sock */
+ if (sock_flag(sk, SOCK_DONE)) {
(void)virtio_transport_reset_no_sock(t, pkt);
release_sock(sk);
sock_put(sk);
write_unlock_bh(&x25_list_lock);
}
-/*
- * Kill all bound sockets on a dropped device.
- */
-static void x25_kill_by_device(struct net_device *dev)
-{
- struct sock *s;
-
- write_lock_bh(&x25_list_lock);
-
- sk_for_each(s, &x25_list)
- if (x25_sk(s)->neighbour && x25_sk(s)->neighbour->dev == dev)
- x25_disconnect(s, ENETUNREACH, 0, 0);
-
- write_unlock_bh(&x25_list_lock);
-}
-
/*
* Handle device status changes.
*/
#endif
) {
switch (event) {
- case NETDEV_UP:
+ case NETDEV_REGISTER:
+ case NETDEV_POST_TYPE_CHANGE:
x25_link_device_up(dev);
break;
- case NETDEV_GOING_DOWN:
+ case NETDEV_DOWN:
nb = x25_get_neigh(dev);
if (nb) {
- x25_terminate_link(nb);
+ x25_link_terminated(nb);
x25_neigh_put(nb);
}
- break;
- case NETDEV_DOWN:
- x25_kill_by_device(dev);
x25_route_device_down(dev);
+ break;
+ case NETDEV_PRE_TYPE_CHANGE:
+ case NETDEV_UNREGISTER:
x25_link_device_down(dev);
break;
+ case NETDEV_CHANGE:
+ if (!netif_carrier_ok(dev)) {
+ nb = x25_get_neigh(dev);
+ if (nb) {
+ x25_link_terminated(nb);
+ x25_neigh_put(nb);
+ }
+ }
+ break;
}
}
int len, i, rc = 0;
if (addr_len != sizeof(struct sockaddr_x25) ||
- addr->sx25_family != AF_X25) {
+ addr->sx25_family != AF_X25 ||
+ strnlen(addr->sx25_addr.x25_addr, X25_ADDR_LEN) == X25_ADDR_LEN) {
rc = -EINVAL;
goto out;
}
rc = -EINVAL;
if (addr_len != sizeof(struct sockaddr_x25) ||
- addr->sx25_family != AF_X25)
+ addr->sx25_family != AF_X25 ||
+ strnlen(addr->sx25_addr.x25_addr, X25_ADDR_LEN) == X25_ADDR_LEN)
goto out;
rc = -ENETUNREACH;
makex25->lci = lci;
makex25->dest_addr = dest_addr;
makex25->source_addr = source_addr;
+ x25_neigh_hold(nb);
makex25->neighbour = nb;
makex25->facilities = facilities;
makex25->dte_facilities= dte_facilities;
switch (frametype) {
case X25_RESTART_REQUEST:
- confirm = !x25_t20timer_pending(nb);
- x25_stop_t20timer(nb);
- nb->state = X25_LINK_STATE_3;
- if (confirm)
+ switch (nb->state) {
+ case X25_LINK_STATE_2:
+ confirm = !x25_t20timer_pending(nb);
+ x25_stop_t20timer(nb);
+ nb->state = X25_LINK_STATE_3;
+ if (confirm)
+ x25_transmit_restart_confirmation(nb);
+ break;
+ case X25_LINK_STATE_3:
+ /* clear existing virtual calls */
+ x25_kill_by_neigh(nb);
+
x25_transmit_restart_confirmation(nb);
+ break;
+ }
break;
case X25_RESTART_CONFIRMATION:
- x25_stop_t20timer(nb);
- nb->state = X25_LINK_STATE_3;
+ switch (nb->state) {
+ case X25_LINK_STATE_2:
+ if (x25_t20timer_pending(nb)) {
+ x25_stop_t20timer(nb);
+ nb->state = X25_LINK_STATE_3;
+ } else {
+ x25_transmit_restart_request(nb);
+ x25_start_t20timer(nb);
+ }
+ break;
+ case X25_LINK_STATE_3:
+ /* clear existing virtual calls */
+ x25_kill_by_neigh(nb);
+
+ x25_transmit_restart_request(nb);
+ nb->state = X25_LINK_STATE_2;
+ x25_start_t20timer(nb);
+ break;
+ }
break;
case X25_DIAGNOSTIC:
{
switch (nb->state) {
case X25_LINK_STATE_0:
- nb->state = X25_LINK_STATE_2;
- break;
case X25_LINK_STATE_1:
x25_transmit_restart_request(nb);
nb->state = X25_LINK_STATE_2;
void x25_link_terminated(struct x25_neigh *nb)
{
nb->state = X25_LINK_STATE_0;
+ skb_queue_purge(&nb->queue);
+ x25_stop_t20timer(nb);
+
/* Out of order: clear existing virtual calls (X.25 03/93 4.6.3) */
x25_kill_by_neigh(nb);
}
*/
static void __x25_remove_neigh(struct x25_neigh *nb)
{
- skb_queue_purge(&nb->queue);
- x25_stop_t20timer(nb);
-
if (nb->node.next) {
list_del(&nb->node);
x25_neigh_put(nb);
__x25_remove_route(rt);
}
write_unlock_bh(&x25_route_list_lock);
-
- /* Remove any related forwarding */
- x25_clear_forward_by_dev(dev);
}
/*
kfree(umem);
}
+static void xdp_umem_release_deferred(struct work_struct *work)
+{
+ struct xdp_umem *umem = container_of(work, struct xdp_umem, work);
+
+ xdp_umem_release(umem);
+}
+
void xdp_get_umem(struct xdp_umem *umem)
{
refcount_inc(&umem->users);
}
-void xdp_put_umem(struct xdp_umem *umem)
+void xdp_put_umem(struct xdp_umem *umem, bool defer_cleanup)
{
if (!umem)
return;
- if (refcount_dec_and_test(&umem->users))
- xdp_umem_release(umem);
+ if (refcount_dec_and_test(&umem->users)) {
+ if (defer_cleanup) {
+ INIT_WORK(&umem->work, xdp_umem_release_deferred);
+ schedule_work(&umem->work);
+ } else {
+ xdp_umem_release(umem);
+ }
+ }
}
static int xdp_umem_pin_pages(struct xdp_umem *umem, unsigned long address)
#include <net/xdp_sock_drv.h>
void xdp_get_umem(struct xdp_umem *umem);
-void xdp_put_umem(struct xdp_umem *umem);
+void xdp_put_umem(struct xdp_umem *umem, bool defer_cleanup);
struct xdp_umem *xdp_umem_create(struct xdp_umem_reg *mr);
#endif /* XDP_UMEM_H_ */
skb_shinfo(skb)->destructor_arg = (void *)(long)desc.addr;
skb->destructor = xsk_destruct_skb;
- /* Hinder dev_direct_xmit from freeing the packet and
- * therefore completing it in the destructor
- */
- refcount_inc(&skb->users);
- err = dev_direct_xmit(skb, xs->queue_id);
+ err = __dev_direct_xmit(skb, xs->queue_id);
if (err == NETDEV_TX_BUSY) {
/* Tell user-space to retry the send */
skb->destructor = sock_wfree;
/* Ignore NET_XMIT_CN as packet might have been sent */
if (err == NET_XMIT_DROP) {
/* SKB completed but not sent */
- kfree_skb(skb);
err = -EBUSY;
goto out;
}
- consume_skb(skb);
sent_frame = true;
}
return;
if (!xp_put_pool(xs->pool))
- xdp_put_umem(xs->umem);
+ xdp_put_umem(xs->umem, !xs->pool);
sk_refcnt_debug_dec(sk);
}
err_unreg_pool:
if (!force_zc)
err = 0; /* fallback to copy mode */
- if (err)
+ if (err) {
xsk_clear_pool_at_qid(netdev, queue_id);
+ dev_put(netdev);
+ }
return err;
}
pool->cq = NULL;
}
- xdp_put_umem(pool->umem);
+ xdp_put_umem(pool->umem, false);
xp_destroy(pool);
}
err = dst_output(xi->net, skb->sk, skb);
if (net_xmit_eval(err) == 0) {
- struct pcpu_sw_netstats *tstats = this_cpu_ptr(dev->tstats);
-
- u64_stats_update_begin(&tstats->syncp);
- tstats->tx_bytes += length;
- tstats->tx_packets++;
- u64_stats_update_end(&tstats->syncp);
+ dev_sw_netstats_tx_add(dev, 1, length);
} else {
stats->tx_errors++;
stats->tx_aborted_errors++;
return err;
}
-static void xfrmi_get_stats64(struct net_device *dev,
- struct rtnl_link_stats64 *s)
-{
- dev_fetch_sw_netstats(s, dev->tstats);
-
- s->rx_dropped = dev->stats.rx_dropped;
- s->tx_dropped = dev->stats.tx_dropped;
-}
-
static int xfrmi_get_iflink(const struct net_device *dev)
{
struct xfrm_if *xi = netdev_priv(dev);
return xi->p.link;
}
-
static const struct net_device_ops xfrmi_netdev_ops = {
.ndo_init = xfrmi_dev_init,
.ndo_uninit = xfrmi_dev_uninit,
.ndo_start_xmit = xfrmi_xmit,
- .ndo_get_stats64 = xfrmi_get_stats64,
+ .ndo_get_stats64 = dev_get_tstats64,
.ndo_get_iflink = xfrmi_get_iflink,
};
asm (
" .pushsection .text, \"ax\", @progbits\n"
" .type my_tramp1, @function\n"
+" .globl my_tramp1\n"
" my_tramp1:"
" pushq %rbp\n"
" movq %rsp, %rbp\n"
" .size my_tramp1, .-my_tramp1\n"
" ret\n"
" .type my_tramp2, @function\n"
+" .globl my_tramp2\n"
" my_tramp2:"
" pushq %rbp\n"
" movq %rsp, %rbp\n"
asm (
" .pushsection .text, \"ax\", @progbits\n"
" .type my_tramp, @function\n"
+" .globl my_tramp\n"
" my_tramp:"
" pushq %rbp\n"
" movq %rsp, %rbp\n"
asm (
" .pushsection .text, \"ax\", @progbits\n"
" .type my_tramp, @function\n"
+" .globl my_tramp\n"
" my_tramp:"
" pushq %rbp\n"
" movq %rsp, %rbp\n"
chmod -R go-w "$pdir"
# in case we are in a restrictive umask environment like 0077
chmod -R a+rX "$pdir"
+ # in case we build in a setuid/setgid directory
+ chmod -R ug-s "$pdir"
# Create the package
dpkg-gencontrol -p$pname -P"$pdir"
struct socket *sock);
int apparmor_secmark_check(struct aa_label *label, char *op, u32 request,
- u32 secid, struct sock *sk);
+ u32 secid, const struct sock *sk);
#endif /* __AA_NET_H */
}
#ifdef CONFIG_NETWORK_SECMARK
-static int apparmor_inet_conn_request(struct sock *sk, struct sk_buff *skb,
+static int apparmor_inet_conn_request(const struct sock *sk, struct sk_buff *skb,
struct request_sock *req)
{
struct aa_sk_ctx *ctx = SK_CTX(sk);
}
static int aa_secmark_perm(struct aa_profile *profile, u32 request, u32 secid,
- struct common_audit_data *sa, struct sock *sk)
+ struct common_audit_data *sa)
{
int i, ret;
struct aa_perms perms = { };
}
int apparmor_secmark_check(struct aa_label *label, char *op, u32 request,
- u32 secid, struct sock *sk)
+ u32 secid, const struct sock *sk)
{
struct aa_profile *profile;
DEFINE_AUDIT_SK(sa, op, sk);
return fn_for_each_confined(label, profile,
aa_secmark_perm(profile, request, secid,
- &sa, sk));
+ &sa));
}
#endif
int ret;
memset(&prep, 0, sizeof(prep));
+ prep.orig_description = key->description;
prep.data = data;
prep.datalen = datalen;
prep.quotalen = key->type->def_datalen;
goto error_put_type;
memset(&prep, 0, sizeof(prep));
+ prep.orig_description = description;
prep.data = payload;
prep.datalen = plen;
prep.quotalen = index_key.type->def_datalen;
static inline void print_ipv6_addr(struct audit_buffer *ab,
- struct in6_addr *addr, __be16 port,
+ const struct in6_addr *addr, __be16 port,
char *name1, char *name2)
{
if (!ipv6_addr_any(addr))
}
case LSM_AUDIT_DATA_NET:
if (a->u.net->sk) {
- struct sock *sk = a->u.net->sk;
+ const struct sock *sk = a->u.net->sk;
struct unix_sock *u;
struct unix_address *addr;
int len = 0;
}
EXPORT_SYMBOL(security_sock_graft);
-int security_inet_conn_request(struct sock *sk,
+int security_inet_conn_request(const struct sock *sk,
struct sk_buff *skb, struct request_sock *req)
{
return call_int_hook(inet_conn_request, 0, sk, skb, req);
selinux_netlbl_sctp_sk_clone(sk, newsk);
}
-static int selinux_inet_conn_request(struct sock *sk, struct sk_buff *skb,
+static int selinux_inet_conn_request(const struct sock *sk, struct sk_buff *skb,
struct request_sock *req)
{
struct sk_security_struct *sksec = sk->sk_security;
* is valid, it just won't be added to the cache.
*/
new = kzalloc(sizeof(*new), GFP_ATOMIC);
- if (!new)
+ if (!new) {
+ ret = -ENOMEM;
goto out;
+ }
new->psec.subnet_prefix = subnet_prefix;
new->psec.pkey = pkey_num;
*
* Returns smack_known of the IP options or NULL if that won't work.
*/
-static struct smack_known *smack_from_netlbl(struct sock *sk, u16 family,
+static struct smack_known *smack_from_netlbl(const struct sock *sk, u16 family,
struct sk_buff *skb)
{
struct netlbl_lsm_secattr secattr;
* Returns 0 if a task with the packet label could write to
* the socket, otherwise an error code
*/
-static int smack_inet_conn_request(struct sock *sk, struct sk_buff *skb,
+static int smack_inet_conn_request(const struct sock *sk, struct sk_buff *skb,
struct request_sock *req)
{
u16 family = sk->sk_family;
unlock:
up_write(&card->controls_rwsem);
- return 0;
+ return err;
}
static int snd_ctl_elem_add_user(struct snd_ctl_file *file,
t = (struct snd_efw_transaction *)data;
length = min_t(size_t, be32_to_cpu(t->length) * sizeof(u32), length);
- spin_lock_irq(&efw->lock);
+ spin_lock(&efw->lock);
if (efw->push_ptr < efw->pull_ptr)
capacity = (unsigned int)(efw->pull_ptr - efw->push_ptr);
copy_resp_to_buf(efw, data, length, rcode);
end:
- spin_unlock_irq(&instances_lock);
+ spin_unlock(&instances_lock);
}
static void
struct nid_path *path;
hda_nid_t pin = pins[i];
- path = snd_hda_get_path_from_idx(codec, path_idx[i]);
- if (path) {
- badness += assign_out_path_ctls(codec, path);
- continue;
+ if (!spec->obey_preferred_dacs) {
+ path = snd_hda_get_path_from_idx(codec, path_idx[i]);
+ if (path) {
+ badness += assign_out_path_ctls(codec, path);
+ continue;
+ }
}
dacs[i] = get_preferred_dac(codec, pin);
if (dacs[i]) {
if (is_dac_already_used(codec, dacs[i]))
badness += bad->shared_primary;
+ } else if (spec->obey_preferred_dacs) {
+ badness += BAD_NO_PRIMARY_DAC;
}
if (!dacs[i])
unsigned int power_down_unused:1; /* power down unused widgets */
unsigned int dac_min_mute:1; /* minimal = mute for DACs */
unsigned int suppress_vmaster:1; /* don't create vmaster kctls */
+ unsigned int obey_preferred_dacs:1; /* obey preferred_dacs assignment */
/* other internal flags */
unsigned int no_analog:1; /* digital I/O only */
/* DG1 */
{ PCI_DEVICE(0x8086, 0x490d),
.driver_data = AZX_DRIVER_SKL | AZX_DCAPS_INTEL_SKYLAKE},
+ /* Alderlake-S */
+ { PCI_DEVICE(0x8086, 0x7ad0),
+ .driver_data = AZX_DRIVER_SKL | AZX_DCAPS_INTEL_SKYLAKE},
/* Elkhart Lake */
{ PCI_DEVICE(0x8086, 0x4b55),
.driver_data = AZX_DRIVER_SKL | AZX_DCAPS_INTEL_SKYLAKE},
case QUIRK_AE5:
ca0132_mmio_init_ae5(codec);
break;
+ default:
+ break;
}
}
HDA_CODEC_ENTRY(0x8086280f, "Icelake HDMI", patch_i915_icl_hdmi),
HDA_CODEC_ENTRY(0x80862812, "Tigerlake HDMI", patch_i915_tgl_hdmi),
HDA_CODEC_ENTRY(0x80862814, "DG1 HDMI", patch_i915_tgl_hdmi),
+HDA_CODEC_ENTRY(0x80862815, "Alderlake HDMI", patch_i915_tgl_hdmi),
HDA_CODEC_ENTRY(0x80862816, "Rocketlake HDMI", patch_i915_tgl_hdmi),
HDA_CODEC_ENTRY(0x8086281a, "Jasperlake HDMI", patch_i915_icl_hdmi),
HDA_CODEC_ENTRY(0x8086281b, "Elkhartlake HDMI", patch_i915_icl_hdmi),
unsigned int no_shutup_pins:1;
unsigned int ultra_low_power:1;
unsigned int has_hs_key:1;
+ unsigned int no_internal_mic_pin:1;
/* for PLL fix */
hda_nid_t pll_nid;
alc_update_coef_idx(codec, 0x7, 1<<5, 0);
break;
case 0x10ec0892:
+ case 0x10ec0897:
alc_update_coef_idx(codec, 0x7, 1<<5, 0);
break;
case 0x10ec0899:
SND_PCI_QUIRK_VENDOR(0x1462, "MSI", ALC882_FIXUP_GPIO3),
SND_PCI_QUIRK(0x147b, 0x107a, "Abit AW9D-MAX", ALC882_FIXUP_ABIT_AW9D_MAX),
SND_PCI_QUIRK(0x1558, 0x9501, "Clevo P950HR", ALC1220_FIXUP_CLEVO_P950),
+ SND_PCI_QUIRK(0x1558, 0x9506, "Clevo P955HQ", ALC1220_FIXUP_CLEVO_P950),
+ SND_PCI_QUIRK(0x1558, 0x950A, "Clevo P955H[PR]", ALC1220_FIXUP_CLEVO_P950),
SND_PCI_QUIRK(0x1558, 0x95e1, "Clevo P95xER", ALC1220_FIXUP_CLEVO_P950),
SND_PCI_QUIRK(0x1558, 0x95e2, "Clevo P950ER", ALC1220_FIXUP_CLEVO_P950),
+ SND_PCI_QUIRK(0x1558, 0x95e3, "Clevo P955[ER]T", ALC1220_FIXUP_CLEVO_P950),
+ SND_PCI_QUIRK(0x1558, 0x95e4, "Clevo P955ER", ALC1220_FIXUP_CLEVO_P950),
+ SND_PCI_QUIRK(0x1558, 0x95e5, "Clevo P955EE6", ALC1220_FIXUP_CLEVO_P950),
+ SND_PCI_QUIRK(0x1558, 0x95e6, "Clevo P950R[CDF]", ALC1220_FIXUP_CLEVO_P950),
SND_PCI_QUIRK(0x1558, 0x96e1, "Clevo P960[ER][CDFN]-K", ALC1220_FIXUP_CLEVO_P950),
SND_PCI_QUIRK(0x1558, 0x97e1, "Clevo P970[ER][CDFN]", ALC1220_FIXUP_CLEVO_P950),
+ SND_PCI_QUIRK(0x1558, 0x97e2, "Clevo P970RC-M", ALC1220_FIXUP_CLEVO_P950),
+ SND_PCI_QUIRK(0x1558, 0x50d3, "Clevo PC50[ER][CDF]", ALC1220_FIXUP_CLEVO_PB51ED_PINS),
SND_PCI_QUIRK(0x1558, 0x65d1, "Clevo PB51[ER][CDF]", ALC1220_FIXUP_CLEVO_PB51ED_PINS),
+ SND_PCI_QUIRK(0x1558, 0x65d2, "Clevo PB51R[CDF]", ALC1220_FIXUP_CLEVO_PB51ED_PINS),
+ SND_PCI_QUIRK(0x1558, 0x65e1, "Clevo PB51[ED][DF]", ALC1220_FIXUP_CLEVO_PB51ED_PINS),
SND_PCI_QUIRK(0x1558, 0x67d1, "Clevo PB71[ER][CDF]", ALC1220_FIXUP_CLEVO_PB51ED_PINS),
- SND_PCI_QUIRK(0x1558, 0x50d3, "Clevo PC50[ER][CDF]", ALC1220_FIXUP_CLEVO_PB51ED_PINS),
+ SND_PCI_QUIRK(0x1558, 0x67e1, "Clevo PB71[DE][CDF]", ALC1220_FIXUP_CLEVO_PB51ED_PINS),
SND_PCI_QUIRK(0x1558, 0x70d1, "Clevo PC70[ER][CDF]", ALC1220_FIXUP_CLEVO_PB51ED_PINS),
SND_PCI_QUIRK(0x1558, 0x7714, "Clevo X170", ALC1220_FIXUP_CLEVO_PB51ED_PINS),
SND_PCI_QUIRK_VENDOR(0x1558, "Clevo laptop", ALC882_FIXUP_EAPD),
alc_fixup_hp_gpio_led(codec, action, 0x02, 0x20);
}
+static void alc287_fixup_hp_gpio_led(struct hda_codec *codec,
+ const struct hda_fixup *fix, int action)
+{
+ alc_fixup_hp_gpio_led(codec, action, 0x10, 0);
+}
+
/* turn on/off mic-mute LED per capture hook via VREF change */
static int vref_micmute_led_set(struct led_classdev *led_cdev,
enum led_brightness brightness)
static void alc_headset_mode_unplugged(struct hda_codec *codec)
{
+ struct alc_spec *spec = codec->spec;
static const struct coef_fw coef0255[] = {
WRITE_COEF(0x1b, 0x0c0b), /* LDO and MISC control */
WRITE_COEF(0x45, 0xd089), /* UAJ function set to menual mode */
{}
};
+ if (spec->no_internal_mic_pin) {
+ alc_update_coef_idx(codec, 0x45, 0xf<<12 | 1<<10, 5<<12);
+ return;
+ }
+
switch (codec->core.vendor_id) {
case 0x10ec0255:
alc_process_coef_fw(codec, coef0255);
{}
};
+ if (spec->no_internal_mic_pin) {
+ alc_update_coef_idx(codec, 0x45, 0xf<<12 | 1<<10, 5<<12);
+ return;
+ }
+
switch (codec->core.vendor_id) {
case 0x10ec0255:
alc_process_coef_fw(codec, coef0255);
codec->power_save_node = 0;
}
+/* avoid DAC 0x06 for bass speaker 0x17; it has no volume control */
+static void alc289_fixup_asus_ga401(struct hda_codec *codec,
+ const struct hda_fixup *fix, int action)
+{
+ static const hda_nid_t preferred_pairs[] = {
+ 0x14, 0x02, 0x17, 0x02, 0x21, 0x03, 0
+ };
+ struct alc_spec *spec = codec->spec;
+
+ if (action == HDA_FIXUP_ACT_PRE_PROBE) {
+ spec->gen.preferred_dacs = preferred_pairs;
+ spec->gen.obey_preferred_dacs = 1;
+ }
+}
+
/* The DAC of NID 0x3 will introduce click/pop noise on headphones, so invalidate it */
static void alc285_fixup_invalidate_dacs(struct hda_codec *codec,
const struct hda_fixup *fix, int action)
}
}
+static void alc_fixup_no_int_mic(struct hda_codec *codec,
+ const struct hda_fixup *fix, int action)
+{
+ struct alc_spec *spec = codec->spec;
+
+ switch (action) {
+ case HDA_FIXUP_ACT_PRE_PROBE:
+ /* Mic RING SLEEVE swap for combo jack */
+ alc_update_coef_idx(codec, 0x45, 0xf<<12 | 1<<10, 5<<12);
+ spec->no_internal_mic_pin = true;
+ break;
+ case HDA_FIXUP_ACT_INIT:
+ alc_combo_jack_hp_jd_restart(codec);
+ break;
+ }
+}
+
/* for hda_fixup_thinkpad_acpi() */
#include "thinkpad_helper.c"
ALC274_FIXUP_HP_MIC,
ALC274_FIXUP_HP_HEADSET_MIC,
ALC256_FIXUP_ASUS_HPE,
+ ALC285_FIXUP_THINKPAD_NO_BASS_SPK_HEADSET_JACK,
+ ALC287_FIXUP_HP_GPIO_LED,
+ ALC256_FIXUP_HP_HEADSET_MIC,
+ ALC236_FIXUP_DELL_AIO_HEADSET_MIC,
};
static const struct hda_fixup alc269_fixups[] = {
.chain_id = ALC269_FIXUP_HEADSET_MIC
},
[ALC289_FIXUP_ASUS_GA401] = {
- .type = HDA_FIXUP_PINS,
- .v.pins = (const struct hda_pintbl[]) {
- { 0x19, 0x03a11020 }, /* headset mic with jack detect */
- { }
- },
+ .type = HDA_FIXUP_FUNC,
+ .v.func = alc289_fixup_asus_ga401,
+ .chained = true,
+ .chain_id = ALC289_FIXUP_ASUS_GA502,
},
[ALC289_FIXUP_ASUS_GA502] = {
.type = HDA_FIXUP_PINS,
{ }
},
.chained = true,
- .chain_id = ALC289_FIXUP_ASUS_GA401
+ .chain_id = ALC289_FIXUP_ASUS_GA502
},
[ALC274_FIXUP_HP_MIC] = {
.type = HDA_FIXUP_VERBS,
.chained = true,
.chain_id = ALC294_FIXUP_ASUS_HEADSET_MIC
},
+ [ALC285_FIXUP_THINKPAD_NO_BASS_SPK_HEADSET_JACK] = {
+ .type = HDA_FIXUP_FUNC,
+ .v.func = alc_fixup_headset_jack,
+ .chained = true,
+ .chain_id = ALC269_FIXUP_THINKPAD_ACPI
+ },
+ [ALC287_FIXUP_HP_GPIO_LED] = {
+ .type = HDA_FIXUP_FUNC,
+ .v.func = alc287_fixup_hp_gpio_led,
+ },
+ [ALC256_FIXUP_HP_HEADSET_MIC] = {
+ .type = HDA_FIXUP_FUNC,
+ .v.func = alc274_fixup_hp_headset_mic,
+ },
+ [ALC236_FIXUP_DELL_AIO_HEADSET_MIC] = {
+ .type = HDA_FIXUP_FUNC,
+ .v.func = alc_fixup_no_int_mic,
+ .chained = true,
+ .chain_id = ALC255_FIXUP_DELL1_MIC_NO_PRESENCE
+ },
};
static const struct snd_pci_quirk alc269_fixup_tbl[] = {
SND_PCI_QUIRK(0x1028, 0x097d, "Dell Precision", ALC289_FIXUP_DUAL_SPK),
SND_PCI_QUIRK(0x1028, 0x098d, "Dell Precision", ALC233_FIXUP_ASUS_MIC_NO_PRESENCE),
SND_PCI_QUIRK(0x1028, 0x09bf, "Dell Precision", ALC233_FIXUP_ASUS_MIC_NO_PRESENCE),
+ SND_PCI_QUIRK(0x1028, 0x0a2e, "Dell", ALC236_FIXUP_DELL_AIO_HEADSET_MIC),
+ SND_PCI_QUIRK(0x1028, 0x0a30, "Dell", ALC236_FIXUP_DELL_AIO_HEADSET_MIC),
SND_PCI_QUIRK(0x1028, 0x164a, "Dell", ALC293_FIXUP_DELL1_MIC_NO_PRESENCE),
SND_PCI_QUIRK(0x1028, 0x164b, "Dell", ALC293_FIXUP_DELL1_MIC_NO_PRESENCE),
SND_PCI_QUIRK(0x103c, 0x1586, "HP", ALC269_FIXUP_HP_MUTE_LED_MIC2),
SND_PCI_QUIRK(0x103c, 0x820d, "HP Pavilion 15", ALC269_FIXUP_HP_MUTE_LED_MIC3),
SND_PCI_QUIRK(0x103c, 0x8256, "HP", ALC221_FIXUP_HP_FRONT_MIC),
SND_PCI_QUIRK(0x103c, 0x827e, "HP x360", ALC295_FIXUP_HP_X360),
+ SND_PCI_QUIRK(0x103c, 0x827f, "HP x360", ALC269_FIXUP_HP_MUTE_LED_MIC3),
SND_PCI_QUIRK(0x103c, 0x82bf, "HP G3 mini", ALC221_FIXUP_HP_MIC_NO_PRESENCE),
SND_PCI_QUIRK(0x103c, 0x82c0, "HP G3 mini premium", ALC221_FIXUP_HP_MIC_NO_PRESENCE),
SND_PCI_QUIRK(0x103c, 0x83b9, "HP Spectre x360", ALC269_FIXUP_HP_MUTE_LED_MIC3),
SND_PCI_QUIRK(0x103c, 0x8760, "HP", ALC285_FIXUP_HP_MUTE_LED),
SND_PCI_QUIRK(0x103c, 0x877a, "HP", ALC285_FIXUP_HP_MUTE_LED),
SND_PCI_QUIRK(0x103c, 0x877d, "HP", ALC236_FIXUP_HP_MUTE_LED),
+ SND_PCI_QUIRK(0x103c, 0x87f4, "HP", ALC287_FIXUP_HP_GPIO_LED),
+ SND_PCI_QUIRK(0x103c, 0x87f5, "HP", ALC287_FIXUP_HP_GPIO_LED),
SND_PCI_QUIRK(0x1043, 0x103e, "ASUS X540SA", ALC256_FIXUP_ASUS_MIC),
SND_PCI_QUIRK(0x1043, 0x103f, "ASUS TX300", ALC282_FIXUP_ASUS_TX300),
SND_PCI_QUIRK(0x1043, 0x106d, "Asus K53BE", ALC269_FIXUP_LIMIT_INT_MIC_BOOST),
SND_PCI_QUIRK(0x1458, 0xfa53, "Gigabyte BXBT-2807", ALC283_FIXUP_HEADSET_MIC),
SND_PCI_QUIRK(0x1462, 0xb120, "MSI Cubi MS-B120", ALC283_FIXUP_HEADSET_MIC),
SND_PCI_QUIRK(0x1462, 0xb171, "Cubi N 8GL (MS-B171)", ALC283_FIXUP_HEADSET_MIC),
+ SND_PCI_QUIRK(0x1558, 0x1323, "Clevo N130ZU", ALC293_FIXUP_SYSTEM76_MIC_NO_PRESENCE),
SND_PCI_QUIRK(0x1558, 0x1325, "System76 Darter Pro (darp5)", ALC293_FIXUP_SYSTEM76_MIC_NO_PRESENCE),
+ SND_PCI_QUIRK(0x1558, 0x1401, "Clevo L140[CZ]U", ALC293_FIXUP_SYSTEM76_MIC_NO_PRESENCE),
+ SND_PCI_QUIRK(0x1558, 0x1403, "Clevo N140CU", ALC293_FIXUP_SYSTEM76_MIC_NO_PRESENCE),
+ SND_PCI_QUIRK(0x1558, 0x1404, "Clevo N150CU", ALC293_FIXUP_SYSTEM76_MIC_NO_PRESENCE),
+ SND_PCI_QUIRK(0x1558, 0x14a1, "Clevo L141MU", ALC293_FIXUP_SYSTEM76_MIC_NO_PRESENCE),
+ SND_PCI_QUIRK(0x1558, 0x4018, "Clevo NV40M[BE]", ALC293_FIXUP_SYSTEM76_MIC_NO_PRESENCE),
+ SND_PCI_QUIRK(0x1558, 0x4019, "Clevo NV40MZ", ALC293_FIXUP_SYSTEM76_MIC_NO_PRESENCE),
+ SND_PCI_QUIRK(0x1558, 0x4020, "Clevo NV40MB", ALC293_FIXUP_SYSTEM76_MIC_NO_PRESENCE),
+ SND_PCI_QUIRK(0x1558, 0x40a1, "Clevo NL40GU", ALC293_FIXUP_SYSTEM76_MIC_NO_PRESENCE),
+ SND_PCI_QUIRK(0x1558, 0x40c1, "Clevo NL40[CZ]U", ALC293_FIXUP_SYSTEM76_MIC_NO_PRESENCE),
+ SND_PCI_QUIRK(0x1558, 0x40d1, "Clevo NL41DU", ALC293_FIXUP_SYSTEM76_MIC_NO_PRESENCE),
+ SND_PCI_QUIRK(0x1558, 0x50a3, "Clevo NJ51GU", ALC293_FIXUP_SYSTEM76_MIC_NO_PRESENCE),
+ SND_PCI_QUIRK(0x1558, 0x50b3, "Clevo NK50S[BEZ]", ALC293_FIXUP_SYSTEM76_MIC_NO_PRESENCE),
+ SND_PCI_QUIRK(0x1558, 0x50b6, "Clevo NK50S5", ALC293_FIXUP_SYSTEM76_MIC_NO_PRESENCE),
+ SND_PCI_QUIRK(0x1558, 0x50b8, "Clevo NK50SZ", ALC293_FIXUP_SYSTEM76_MIC_NO_PRESENCE),
+ SND_PCI_QUIRK(0x1558, 0x50d5, "Clevo NP50D5", ALC293_FIXUP_SYSTEM76_MIC_NO_PRESENCE),
+ SND_PCI_QUIRK(0x1558, 0x50f0, "Clevo NH50A[CDF]", ALC293_FIXUP_SYSTEM76_MIC_NO_PRESENCE),
+ SND_PCI_QUIRK(0x1558, 0x50f3, "Clevo NH58DPQ", ALC293_FIXUP_SYSTEM76_MIC_NO_PRESENCE),
+ SND_PCI_QUIRK(0x1558, 0x5101, "Clevo S510WU", ALC293_FIXUP_SYSTEM76_MIC_NO_PRESENCE),
+ SND_PCI_QUIRK(0x1558, 0x5157, "Clevo W517GU1", ALC293_FIXUP_SYSTEM76_MIC_NO_PRESENCE),
+ SND_PCI_QUIRK(0x1558, 0x51a1, "Clevo NS50MU", ALC293_FIXUP_SYSTEM76_MIC_NO_PRESENCE),
+ SND_PCI_QUIRK(0x1558, 0x70a1, "Clevo NB70T[HJK]", ALC293_FIXUP_SYSTEM76_MIC_NO_PRESENCE),
+ SND_PCI_QUIRK(0x1558, 0x70b3, "Clevo NK70SB", ALC293_FIXUP_SYSTEM76_MIC_NO_PRESENCE),
+ SND_PCI_QUIRK(0x1558, 0x8228, "Clevo NR40BU", ALC293_FIXUP_SYSTEM76_MIC_NO_PRESENCE),
+ SND_PCI_QUIRK(0x1558, 0x8520, "Clevo NH50D[CD]", ALC293_FIXUP_SYSTEM76_MIC_NO_PRESENCE),
+ SND_PCI_QUIRK(0x1558, 0x8521, "Clevo NH77D[CD]", ALC293_FIXUP_SYSTEM76_MIC_NO_PRESENCE),
+ SND_PCI_QUIRK(0x1558, 0x8535, "Clevo NH50D[BE]", ALC293_FIXUP_SYSTEM76_MIC_NO_PRESENCE),
+ SND_PCI_QUIRK(0x1558, 0x8536, "Clevo NH79D[BE]", ALC293_FIXUP_SYSTEM76_MIC_NO_PRESENCE),
SND_PCI_QUIRK(0x1558, 0x8550, "System76 Gazelle (gaze14)", ALC293_FIXUP_SYSTEM76_MIC_NO_PRESENCE),
SND_PCI_QUIRK(0x1558, 0x8551, "System76 Gazelle (gaze14)", ALC293_FIXUP_SYSTEM76_MIC_NO_PRESENCE),
SND_PCI_QUIRK(0x1558, 0x8560, "System76 Gazelle (gaze14)", ALC269_FIXUP_HEADSET_MIC),
SND_PCI_QUIRK(0x1558, 0x8561, "System76 Gazelle (gaze14)", ALC269_FIXUP_HEADSET_MIC),
+ SND_PCI_QUIRK(0x1558, 0x8668, "Clevo NP50B[BE]", ALC293_FIXUP_SYSTEM76_MIC_NO_PRESENCE),
+ SND_PCI_QUIRK(0x1558, 0x8680, "Clevo NJ50LU", ALC293_FIXUP_SYSTEM76_MIC_NO_PRESENCE),
+ SND_PCI_QUIRK(0x1558, 0x8686, "Clevo NH50[CZ]U", ALC293_FIXUP_SYSTEM76_MIC_NO_PRESENCE),
+ SND_PCI_QUIRK(0x1558, 0x8a20, "Clevo NH55DCQ-Y", ALC293_FIXUP_SYSTEM76_MIC_NO_PRESENCE),
+ SND_PCI_QUIRK(0x1558, 0x8a51, "Clevo NH70RCQ-Y", ALC293_FIXUP_SYSTEM76_MIC_NO_PRESENCE),
+ SND_PCI_QUIRK(0x1558, 0x8d50, "Clevo NH55RCQ-M", ALC293_FIXUP_SYSTEM76_MIC_NO_PRESENCE),
+ SND_PCI_QUIRK(0x1558, 0x951d, "Clevo N950T[CDF]", ALC293_FIXUP_SYSTEM76_MIC_NO_PRESENCE),
+ SND_PCI_QUIRK(0x1558, 0x961d, "Clevo N960S[CDF]", ALC293_FIXUP_SYSTEM76_MIC_NO_PRESENCE),
+ SND_PCI_QUIRK(0x1558, 0x971d, "Clevo N970T[CDF]", ALC293_FIXUP_SYSTEM76_MIC_NO_PRESENCE),
+ SND_PCI_QUIRK(0x1558, 0xa500, "Clevo NL53RU", ALC293_FIXUP_SYSTEM76_MIC_NO_PRESENCE),
SND_PCI_QUIRK(0x17aa, 0x1036, "Lenovo P520", ALC233_FIXUP_LENOVO_MULTI_CODECS),
SND_PCI_QUIRK(0x17aa, 0x1048, "ThinkCentre Station", ALC283_FIXUP_HEADSET_MIC),
SND_PCI_QUIRK(0x17aa, 0x20f2, "Thinkpad SL410/510", ALC269_FIXUP_SKU_IGNORE),
SND_PCI_QUIRK(0x17aa, 0x225d, "Thinkpad T480", ALC269_FIXUP_LIMIT_INT_MIC_BOOST),
SND_PCI_QUIRK(0x17aa, 0x2292, "Thinkpad X1 Carbon 7th", ALC285_FIXUP_THINKPAD_HEADSET_JACK),
SND_PCI_QUIRK(0x17aa, 0x22be, "Thinkpad X1 Carbon 8th", ALC285_FIXUP_THINKPAD_HEADSET_JACK),
+ SND_PCI_QUIRK(0x17aa, 0x22c1, "Thinkpad P1 Gen 3", ALC285_FIXUP_THINKPAD_NO_BASS_SPK_HEADSET_JACK),
+ SND_PCI_QUIRK(0x17aa, 0x22c2, "Thinkpad X1 Extreme Gen 3", ALC285_FIXUP_THINKPAD_NO_BASS_SPK_HEADSET_JACK),
SND_PCI_QUIRK(0x17aa, 0x30bb, "ThinkCentre AIO", ALC233_FIXUP_LENOVO_LINE2_MIC_HOTKEY),
SND_PCI_QUIRK(0x17aa, 0x30e2, "ThinkCentre AIO", ALC233_FIXUP_LENOVO_LINE2_MIC_HOTKEY),
SND_PCI_QUIRK(0x17aa, 0x310c, "ThinkCentre Station", ALC294_FIXUP_LENOVO_MIC_LOCATION),
{0x19, 0x02a11020},
{0x1a, 0x02a11030},
{0x21, 0x0221101f}),
+ SND_HDA_PIN_QUIRK(0x10ec0236, 0x1028, "Dell", ALC236_FIXUP_DELL_AIO_HEADSET_MIC,
+ {0x21, 0x02211010}),
+ SND_HDA_PIN_QUIRK(0x10ec0236, 0x103c, "HP", ALC256_FIXUP_HP_HEADSET_MIC,
+ {0x14, 0x90170110},
+ {0x19, 0x02a11020},
+ {0x21, 0x02211030}),
SND_HDA_PIN_QUIRK(0x10ec0255, 0x1028, "Dell", ALC255_FIXUP_DELL2_MIC_NO_PRESENCE,
{0x14, 0x90170110},
{0x21, 0x02211020}),
{0x1a, 0x90a70130},
{0x1b, 0x90170110},
{0x21, 0x03211020}),
+ SND_HDA_PIN_QUIRK(0x10ec0256, 0x103c, "HP", ALC256_FIXUP_HP_HEADSET_MIC,
+ {0x14, 0x90170110},
+ {0x19, 0x02a11020},
+ {0x21, 0x0221101f}),
SND_HDA_PIN_QUIRK(0x10ec0274, 0x103c, "HP", ALC274_FIXUP_HP_HEADSET_MIC,
{0x17, 0x90170110},
{0x19, 0x03a11030},
SND_HDA_PIN_QUIRK(0x10ec0293, 0x1028, "Dell", ALC293_FIXUP_DELL1_MIC_NO_PRESENCE,
ALC292_STANDARD_PINS,
{0x13, 0x90a60140}),
+ SND_HDA_PIN_QUIRK(0x10ec0294, 0x1043, "ASUS", ALC294_FIXUP_ASUS_HPE,
+ {0x17, 0x90170110},
+ {0x21, 0x04211020}),
SND_HDA_PIN_QUIRK(0x10ec0294, 0x1043, "ASUS", ALC294_FIXUP_ASUS_MIC,
{0x14, 0x90170110},
{0x1b, 0x90a70130},
HDA_CODEC_ENTRY(0x10ec0888, "ALC888", patch_alc882),
HDA_CODEC_ENTRY(0x10ec0889, "ALC889", patch_alc882),
HDA_CODEC_ENTRY(0x10ec0892, "ALC892", patch_alc662),
+ HDA_CODEC_ENTRY(0x10ec0897, "ALC897", patch_alc662),
HDA_CODEC_ENTRY(0x10ec0899, "ALC898", patch_alc882),
HDA_CODEC_ENTRY(0x10ec0900, "ALC1150", patch_alc882),
HDA_CODEC_ENTRY(0x10ec0b00, "ALCS1200A", patch_alc882),
unsigned int i;
#endif
- mutex_lock(&mgr->msg_lock);
err = 0;
/* copy message descriptor from miXart to driver */
writel_be(headptr, MIXART_MEM(mgr, MSG_OUTBOUND_FREE_HEAD));
_clean_exit:
- mutex_unlock(&mgr->msg_lock);
-
return err;
}
resp.data = resp_data;
resp.size = max_resp_size;
+ mutex_lock(&mgr->msg_lock);
err = get_msg(mgr, &resp, msg_frame);
+ mutex_unlock(&mgr->msg_lock);
if( request->message_id != resp.message_id )
dev_err(&mgr->pci->dev, "RESPONSE ERROR!\n");
#include <sound/soc-dapm.h>
#include <sound/soc.h>
#include <sound/tlv.h>
+#include <sound/rt1015.h>
#include "rl6231.h"
#include "rt1015.h"
+static const struct rt1015_platform_data i2s_default_platform_data = {
+ .power_up_delay_ms = 50,
+};
+
static const struct reg_default rt1015_reg[] = {
{ 0x0000, 0x0000 },
{ 0x0004, 0xa000 },
struct rt1015_priv *rt1015 = container_of(work, struct rt1015_priv,
flush_work.work);
struct snd_soc_component *component = rt1015->component;
- unsigned int val, i = 0, count = 20;
+ unsigned int val, i = 0, count = 200;
while (i < count) {
usleep_range(1000, 1500);
case SND_SOC_DAPM_POST_PMU:
if (rt1015->hw_config == RT1015_HW_28)
schedule_delayed_work(&rt1015->flush_work, msecs_to_jiffies(10));
+ msleep(rt1015->pdata.power_up_delay_ms);
break;
default:
break;
MODULE_DEVICE_TABLE(acpi, rt1015_acpi_match);
#endif
+static void rt1015_parse_dt(struct rt1015_priv *rt1015, struct device *dev)
+{
+ device_property_read_u32(dev, "realtek,power-up-delay-ms",
+ &rt1015->pdata.power_up_delay_ms);
+}
+
static int rt1015_i2c_probe(struct i2c_client *i2c,
const struct i2c_device_id *id)
{
+ struct rt1015_platform_data *pdata = dev_get_platdata(&i2c->dev);
struct rt1015_priv *rt1015;
int ret;
unsigned int val;
i2c_set_clientdata(i2c, rt1015);
+ rt1015->pdata = i2s_default_platform_data;
+
+ if (pdata)
+ rt1015->pdata = *pdata;
+ else
+ rt1015_parse_dt(rt1015, &i2c->dev);
+
rt1015->regmap = devm_regmap_init_i2c(i2c, &rt1015_regmap);
if (IS_ERR(rt1015->regmap)) {
ret = PTR_ERR(rt1015->regmap);
#ifndef __RT1015_H__
#define __RT1015_H__
+#include <sound/rt1015.h>
#define RT1015_DEVICE_ID_VAL 0x1011
#define RT1015_DEVICE_ID_VAL2 0x1015
struct rt1015_priv {
struct snd_soc_component *component;
+ struct rt1015_platform_data pdata;
struct regmap *regmap;
int sysclk;
int sysclk_src;
{RT5682_DAC_ADC_DIG_VOL1, 0xa020},
{RT5682_I2C_CTRL, 0x000f},
{RT5682_PLL2_INTERNAL, 0x8266},
+ {RT5682_SAR_IL_CMD_3, 0x8365},
};
void rt5682_apply_patch_list(struct rt5682_priv *rt5682, struct device *dev)
mem = wm_adsp_find_region(dsp, type);
if (!mem) {
adsp_err(dsp, "No region of type: %x\n", type);
+ ret = -EINVAL;
goto out_fw;
}
.driver_data = (void *)(BYT_RT5640_IN1_MAP |
BYT_RT5640_MCLK_EN),
},
- { /* HP Pavilion x2 10-n000nd */
+ { /* HP Pavilion x2 10-k0XX, 10-n0XX */
.matches = {
- DMI_EXACT_MATCH(DMI_SYS_VENDOR, "Hewlett-Packard"),
- DMI_EXACT_MATCH(DMI_PRODUCT_NAME, "HP Pavilion x2 Detachable"),
+ DMI_MATCH(DMI_SYS_VENDOR, "Hewlett-Packard"),
+ DMI_MATCH(DMI_PRODUCT_NAME, "HP Pavilion x2 Detachable"),
},
.driver_data = (void *)(BYT_RT5640_DMIC1_MAP |
BYT_RT5640_JD_SRC_JD2_IN4N |
BYT_RT5640_SSP0_AIF1 |
BYT_RT5640_MCLK_EN),
},
+ { /* HP Pavilion x2 10-p0XX */
+ .matches = {
+ DMI_MATCH(DMI_SYS_VENDOR, "HP"),
+ DMI_MATCH(DMI_PRODUCT_NAME, "HP x2 Detachable 10-p0XX"),
+ },
+ .driver_data = (void *)(BYT_RT5640_DMIC1_MAP |
+ BYT_RT5640_JD_SRC_JD1_IN4P |
+ BYT_RT5640_OVCD_TH_1500UA |
+ BYT_RT5640_OVCD_SF_0P75 |
+ BYT_RT5640_MCLK_EN),
+ },
{ /* HP Stream 7 */
.matches = {
DMI_EXACT_MATCH(DMI_SYS_VENDOR, "Hewlett-Packard"),
switch (level) {
case SND_SOC_BIAS_PREPARE:
if (dapm->bias_level == SND_SOC_BIAS_ON) {
+ if (!__clk_is_enabled(priv->mclk))
+ return 0;
dev_dbg(card->dev, "Disable mclk");
clk_disable_unprepare(priv->mclk);
} else {
if (ret)
return CATPT_IPC_ERROR(ret);
- ret = catpt_dsp_update_lpclock(cdev);
- if (ret)
- return ret;
-
ret = catpt_dai_apply_usettings(dai, stream);
if (ret)
return ret;
case SNDRV_PCM_TRIGGER_RESUME:
case SNDRV_PCM_TRIGGER_PAUSE_RELEASE:
resume_stream:
+ catpt_dsp_update_lpclock(cdev);
ret = catpt_ipc_resume_stream(cdev, stream->info.stream_hw_id);
if (ret)
return CATPT_IPC_ERROR(ret);
case SNDRV_PCM_TRIGGER_STOP:
stream->prepared = false;
- catpt_dsp_update_lpclock(cdev);
fallthrough;
case SNDRV_PCM_TRIGGER_SUSPEND:
case SNDRV_PCM_TRIGGER_PAUSE_PUSH:
ret = catpt_ipc_pause_stream(cdev, stream->info.stream_hw_id);
+ catpt_dsp_update_lpclock(cdev);
if (ret)
return CATPT_IPC_ERROR(ret);
break;
dsppos = bytes_to_frames(r, pos->stream_position);
+ if (!stream->prepared)
+ goto exit;
/* only offload is set_write_pos driven */
if (stream->template->type != CATPT_STRM_TYPE_RENDER)
goto exit;
kmb_i2s->xfer_resolution = 0x02;
break;
case SNDRV_PCM_FORMAT_S24_LE:
- config->data_width = 24;
- kmb_i2s->ccr = 0x08;
- kmb_i2s->xfer_resolution = 0x04;
+ config->data_width = 32;
+ kmb_i2s->ccr = 0x14;
+ kmb_i2s->xfer_resolution = 0x05;
break;
case SNDRV_PCM_FORMAT_S32_LE:
config->data_width = 32;
return 0;
}
-static int lpass_cpu_daiops_prepare(struct snd_pcm_substream *substream,
- struct snd_soc_dai *dai)
-{
- struct lpass_data *drvdata = snd_soc_dai_get_drvdata(dai);
- struct lpaif_i2sctl *i2sctl = drvdata->i2sctl;
- unsigned int id = dai->driver->id;
- int ret;
-
- if (substream->stream == SNDRV_PCM_STREAM_PLAYBACK) {
- ret = regmap_fields_write(i2sctl->spken, id,
- LPAIF_I2SCTL_SPKEN_ENABLE);
- } else {
- ret = regmap_fields_write(i2sctl->micen, id,
- LPAIF_I2SCTL_MICEN_ENABLE);
- }
-
- if (ret)
- dev_err(dai->dev, "error writing to i2sctl enable: %d\n", ret);
-
- return ret;
-}
-
static int lpass_cpu_daiops_trigger(struct snd_pcm_substream *substream,
int cmd, struct snd_soc_dai *dai)
{
struct lpaif_i2sctl *i2sctl = drvdata->i2sctl;
unsigned int id = dai->driver->id;
int ret = -EINVAL;
+ unsigned int val = 0;
+
+ ret = regmap_read(drvdata->lpaif_map,
+ LPAIF_I2SCTL_REG(drvdata->variant, dai->driver->id), &val);
+ if (ret) {
+ dev_err(dai->dev, "error reading from i2sctl reg: %d\n", ret);
+ return ret;
+ }
+ if (val == LPAIF_I2SCTL_RESET_STATE) {
+ dev_err(dai->dev, "error in i2sctl register state\n");
+ return -ENOTRECOVERABLE;
+ }
switch (cmd) {
case SNDRV_PCM_TRIGGER_START:
dev_err(dai->dev, "error writing to i2sctl reg: %d\n",
ret);
- ret = clk_enable(drvdata->mi2s_bit_clk[id]);
- if (ret) {
- dev_err(dai->dev, "error in enabling mi2s bit clk: %d\n", ret);
- clk_disable(drvdata->mi2s_osr_clk[id]);
- return ret;
+ if (drvdata->bit_clk_state[id] == LPAIF_BIT_CLK_DISABLE) {
+ ret = clk_enable(drvdata->mi2s_bit_clk[id]);
+ if (ret) {
+ dev_err(dai->dev, "error in enabling mi2s bit clk: %d\n", ret);
+ clk_disable(drvdata->mi2s_osr_clk[id]);
+ return ret;
+ }
+ drvdata->bit_clk_state[id] = LPAIF_BIT_CLK_ENABLE;
}
break;
if (ret)
dev_err(dai->dev, "error writing to i2sctl reg: %d\n",
ret);
- clk_disable(drvdata->mi2s_bit_clk[dai->driver->id]);
+ if (drvdata->bit_clk_state[id] == LPAIF_BIT_CLK_ENABLE) {
+ clk_disable(drvdata->mi2s_bit_clk[dai->driver->id]);
+ drvdata->bit_clk_state[id] = LPAIF_BIT_CLK_DISABLE;
+ }
break;
}
.startup = lpass_cpu_daiops_startup,
.shutdown = lpass_cpu_daiops_shutdown,
.hw_params = lpass_cpu_daiops_hw_params,
- .prepare = lpass_cpu_daiops_prepare,
.trigger = lpass_cpu_daiops_trigger,
};
EXPORT_SYMBOL_GPL(asoc_qcom_lpass_cpu_dai_ops);
struct lpass_variant *v = drvdata->variant;
int i;
+ for (i = 0; i < v->i2s_ports; ++i)
+ if (reg == LPAIF_I2SCTL_REG(v, i))
+ return true;
for (i = 0; i < v->irq_ports; ++i)
if (reg == LPAIF_IRQSTAT_REG(v, i))
return true;
for (i = 0; i < v->rdma_channels; ++i)
- if (reg == LPAIF_RDMACURR_REG(v, i))
+ if (reg == LPAIF_RDMACURR_REG(v, i) || reg == LPAIF_RDMACTL_REG(v, i))
return true;
for (i = 0; i < v->wrdma_channels; ++i)
- if (reg == LPAIF_WRDMACURR_REG(v, i + v->wrdma_channel_start))
+ if (reg == LPAIF_WRDMACURR_REG(v, i + v->wrdma_channel_start) ||
+ reg == LPAIF_WRDMACTL_REG(v, i + v->wrdma_channel_start))
return true;
return false;
PTR_ERR(drvdata->mi2s_bit_clk[dai_id]));
return PTR_ERR(drvdata->mi2s_bit_clk[dai_id]);
}
+ drvdata->bit_clk_state[dai_id] = LPAIF_BIT_CLK_DISABLE;
}
/* Allocation for i2sctl regmap fields */
#define LPAIF_I2SCTL_BITWIDTH_24 1
#define LPAIF_I2SCTL_BITWIDTH_32 2
+#define LPAIF_BIT_CLK_DISABLE 0
+#define LPAIF_BIT_CLK_ENABLE 1
+
+#define LPAIF_I2SCTL_RESET_STATE 0x003C0004
+#define LPAIF_DMACTL_RESET_STATE 0x00200000
+
+
/* LPAIF IRQ */
#define LPAIF_IRQ_REG_ADDR(v, addr, port) \
(v->irq_reg_base + (addr) + v->irq_reg_stride * (port))
struct regmap *map;
unsigned int dai_id = cpu_dai->driver->id;
+ component->id = dai_id;
data = kzalloc(sizeof(*data), GFP_KERNEL);
if (!data)
return -ENOMEM;
else
dma_ch = 0;
- if (dma_ch < 0)
+ if (dma_ch < 0) {
+ kfree(data);
return dma_ch;
+ }
if (cpu_dai->driver->id == LPASS_DP_RX) {
map = drvdata->hdmiif_map;
ret = snd_pcm_hw_constraint_integer(runtime,
SNDRV_PCM_HW_PARAM_PERIODS);
if (ret < 0) {
+ kfree(data);
dev_err(soc_runtime->dev, "setting constraints failed: %d\n",
ret);
return -EINVAL;
unsigned int reg_irqclr = 0, val_irqclr = 0;
unsigned int reg_irqen = 0, val_irqen = 0, val_mask = 0;
unsigned int dai_id = cpu_dai->driver->id;
+ unsigned int dma_ctrl_reg = 0;
ch = pcm_data->dma_ch;
if (dir == SNDRV_PCM_STREAM_PLAYBACK) {
id = pcm_data->dma_ch;
- if (dai_id == LPASS_DP_RX)
+ if (dai_id == LPASS_DP_RX) {
dmactl = drvdata->hdmi_rd_dmactl;
- else
+ map = drvdata->hdmiif_map;
+ } else {
dmactl = drvdata->rd_dmactl;
+ map = drvdata->lpaif_map;
+ }
} else {
dmactl = drvdata->wr_dmactl;
id = pcm_data->dma_ch - v->wrdma_channel_start;
+ map = drvdata->lpaif_map;
+ }
+ ret = regmap_read(map, LPAIF_DMACTL_REG(v, ch, dir, dai_id), &dma_ctrl_reg);
+ if (ret) {
+ dev_err(soc_runtime->dev, "error reading from rdmactl reg: %d\n", ret);
+ return ret;
}
+ if (dma_ctrl_reg == LPAIF_DMACTL_RESET_STATE ||
+ dma_ctrl_reg == LPAIF_DMACTL_RESET_STATE + 1) {
+ dev_err(soc_runtime->dev, "error in rdmactl register state\n");
+ return -ENOTRECOVERABLE;
+ }
switch (cmd) {
case SNDRV_PCM_TRIGGER_START:
case SNDRV_PCM_TRIGGER_RESUME:
unsigned int mi2s_playback_sd_mode[LPASS_MAX_MI2S_PORTS];
unsigned int mi2s_capture_sd_mode[LPASS_MAX_MI2S_PORTS];
int hdmi_port_enable;
+ int bit_clk_state[LPASS_MAX_MI2S_PORTS];
/* low-power audio interface (LPAIF) registers */
void __iomem *lpaif;
DEVICE_NAME(0x046d, 0x0990, "Logitech, Inc.", "QuickCam Pro 9000"),
+ /* ASUS ROG Strix */
+ PROFILE_NAME(0x0b05, 0x1917,
+ "Realtek", "ALC1220-VB-DT", "Realtek-ALC1220-VB-Desktop"),
+
/* Dell WD15 Dock */
PROFILE_NAME(0x0bda, 0x4014, "Dell", "WD15 Dock", "Dell-WD15-Dock"),
/* Dell WD19 Dock */
},
{ /* ASUS ROG Strix */
.id = USB_ID(0x0b05, 0x1917),
- .map = asus_rog_map,
+ .map = trx40_mobo_map,
+ .connector_map = trx40_mobo_connector_map,
},
{ /* MSI TRX40 Creator */
.id = USB_ID(0x0db0, 0x0d64),
static int snd_us16x08_meter_info(struct snd_kcontrol *kcontrol,
struct snd_ctl_elem_info *uinfo)
{
- uinfo->count = 1;
+ uinfo->count = 34;
uinfo->type = SNDRV_CTL_ELEM_TYPE_INTEGER;
uinfo->value.integer.max = 0x7FFF;
uinfo->value.integer.min = 0;
&& (requesttype & USB_TYPE_MASK) == USB_TYPE_CLASS)
msleep(20);
- /* Zoom R16/24, Logitech H650e/H570e, Jabra 550a, Kingston HyperX
- * needs a tiny delay here, otherwise requests like get/set
- * frequency return as failed despite actually succeeding.
+ /* Zoom R16/24, many Logitech(at least H650e/H570e/BCC950),
+ * Jabra 550a, Kingston HyperX needs a tiny delay here,
+ * otherwise requests like get/set frequency return
+ * as failed despite actually succeeding.
*/
if ((chip->usb_id == USB_ID(0x1686, 0x00dd) ||
- chip->usb_id == USB_ID(0x046d, 0x0a46) ||
- chip->usb_id == USB_ID(0x046d, 0x0a56) ||
+ USB_ID_VENDOR(chip->usb_id) == 0x046d || /* Logitech */
chip->usb_id == USB_ID(0x0b0e, 0x0349) ||
chip->usb_id == USB_ID(0x0951, 0x16ad)) &&
(requesttype & USB_TYPE_MASK) == USB_TYPE_CLASS)
* to a jmp to memcpy_erms which does the REP; MOVSB mem copy.
*/
-.weak memcpy
-
/*
* memcpy - Copy a memory block.
*
* rax original destination
*/
SYM_FUNC_START_ALIAS(__memcpy)
-SYM_FUNC_START_LOCAL(memcpy)
+SYM_FUNC_START_WEAK(memcpy)
ALTERNATIVE_2 "jmp memcpy_orig", "", X86_FEATURE_REP_GOOD, \
"jmp memcpy_erms", X86_FEATURE_ERMS
* memcpy_erms() - enhanced fast string memcpy. This is faster and
* simpler than memcpy. Use memcpy_erms when possible.
*/
-SYM_FUNC_START(memcpy_erms)
+SYM_FUNC_START_LOCAL(memcpy_erms)
movq %rdi, %rax
movq %rdx, %rcx
rep movsb
ret
SYM_FUNC_END(memcpy_erms)
-SYM_FUNC_START(memcpy_orig)
+SYM_FUNC_START_LOCAL(memcpy_orig)
movq %rdi, %rax
cmpq $0x20, %rdx
#include <linux/linkage.h>
#include <asm/cpufeatures.h>
#include <asm/alternative-asm.h>
-
-.weak memset
+#include <asm/export.h>
/*
* ISO C memset - set a memory block to a byte value. This function uses fast
*
* rax original destination
*/
-SYM_FUNC_START_ALIAS(memset)
+SYM_FUNC_START_WEAK(memset)
SYM_FUNC_START(__memset)
/*
* Some CPUs support enhanced REP MOVSB/STOSB feature. It is recommended
ret
SYM_FUNC_END(__memset)
SYM_FUNC_END_ALIAS(memset)
+EXPORT_SYMBOL(memset)
+EXPORT_SYMBOL(__memset)
/*
* ISO C memset - set a memory block to a byte value. This function uses
*
* rax original destination
*/
-SYM_FUNC_START(memset_erms)
+SYM_FUNC_START_LOCAL(memset_erms)
movq %rdi,%r9
movb %sil,%al
movq %rdx,%rcx
ret
SYM_FUNC_END(memset_erms)
-SYM_FUNC_START(memset_orig)
+SYM_FUNC_START_LOCAL(memset_orig)
movq %rdi,%r10
/* expand byte value */
#include <unistd.h>
#include <string.h>
#include <errno.h>
+#include <endian.h>
#include <linux/kernel.h>
#include <linux/bootconfig.h>
return ret;
}
+static int pr_errno(const char *msg, int err)
+{
+ pr_err("%s: %d\n", msg, err);
+ return err;
+}
+
static int load_xbc_from_initrd(int fd, char **buf)
{
struct stat stat;
if (stat.st_size < 8 + BOOTCONFIG_MAGIC_LEN)
return 0;
- if (lseek(fd, -BOOTCONFIG_MAGIC_LEN, SEEK_END) < 0) {
- pr_err("Failed to lseek: %d\n", -errno);
- return -errno;
- }
+ if (lseek(fd, -BOOTCONFIG_MAGIC_LEN, SEEK_END) < 0)
+ return pr_errno("Failed to lseek for magic", -errno);
+
if (read(fd, magic, BOOTCONFIG_MAGIC_LEN) < 0)
- return -errno;
+ return pr_errno("Failed to read", -errno);
+
/* Check the bootconfig magic bytes */
if (memcmp(magic, BOOTCONFIG_MAGIC, BOOTCONFIG_MAGIC_LEN) != 0)
return 0;
- if (lseek(fd, -(8 + BOOTCONFIG_MAGIC_LEN), SEEK_END) < 0) {
- pr_err("Failed to lseek: %d\n", -errno);
- return -errno;
- }
+ if (lseek(fd, -(8 + BOOTCONFIG_MAGIC_LEN), SEEK_END) < 0)
+ return pr_errno("Failed to lseek for size", -errno);
if (read(fd, &size, sizeof(u32)) < 0)
- return -errno;
+ return pr_errno("Failed to read size", -errno);
+ size = le32toh(size);
if (read(fd, &csum, sizeof(u32)) < 0)
- return -errno;
+ return pr_errno("Failed to read checksum", -errno);
+ csum = le32toh(csum);
/* Wrong size error */
if (stat.st_size < size + 8 + BOOTCONFIG_MAGIC_LEN) {
}
if (lseek(fd, stat.st_size - (size + 8 + BOOTCONFIG_MAGIC_LEN),
- SEEK_SET) < 0) {
- pr_err("Failed to lseek: %d\n", -errno);
- return -errno;
- }
+ SEEK_SET) < 0)
+ return pr_errno("Failed to lseek", -errno);
ret = load_xbc_fd(fd, buf, size);
if (ret < 0)
ret = stat(path, &st);
if (ret < 0) {
- pr_err("Failed to stat %s: %d\n", path, -errno);
- return -errno;
+ ret = -errno;
+ pr_err("Failed to stat %s: %d\n", path, ret);
+ return ret;
}
fd = open(path, O_RDONLY);
if (fd < 0) {
- pr_err("Failed to open initrd %s: %d\n", path, fd);
- return -errno;
+ ret = -errno;
+ pr_err("Failed to open initrd %s: %d\n", path, ret);
+ return ret;
}
ret = load_xbc_from_initrd(fd, &buf);
fd = open(path, O_RDWR);
if (fd < 0) {
- pr_err("Failed to open initrd %s: %d\n", path, fd);
- return -errno;
+ ret = -errno;
+ pr_err("Failed to open initrd %s: %d\n", path, ret);
+ return ret;
}
size = load_xbc_from_initrd(fd, &buf);
static int apply_xbc(const char *path, const char *xbc_path)
{
+ char *buf, *data, *p;
+ size_t total_size;
+ struct stat stat;
+ const char *msg;
u32 size, csum;
- char *buf, *data;
+ int pos, pad;
int ret, fd;
- const char *msg;
- int pos;
ret = load_xbc_file(xbc_path, &buf);
if (ret < 0) {
size = strlen(buf) + 1;
csum = checksum((unsigned char *)buf, size);
- /* Prepare xbc_path data */
- data = malloc(size + 8);
+ /* Backup the bootconfig data */
+ data = calloc(size + BOOTCONFIG_ALIGN +
+ sizeof(u32) + sizeof(u32) + BOOTCONFIG_MAGIC_LEN, 1);
if (!data)
return -ENOMEM;
- strcpy(data, buf);
- *(u32 *)(data + size) = size;
- *(u32 *)(data + size + 4) = csum;
+ memcpy(data, buf, size);
/* Check the data format */
ret = xbc_init(buf, &msg, &pos);
/* Apply new one */
fd = open(path, O_RDWR | O_APPEND);
if (fd < 0) {
- pr_err("Failed to open %s: %d\n", path, fd);
+ ret = -errno;
+ pr_err("Failed to open %s: %d\n", path, ret);
free(data);
- return fd;
+ return ret;
}
/* TODO: Ensure the @path is initramfs/initrd image */
- ret = write(fd, data, size + 8);
- if (ret < 0) {
- pr_err("Failed to apply a boot config: %d\n", ret);
+ if (fstat(fd, &stat) < 0) {
+ pr_err("Failed to get the size of %s\n", path);
goto out;
}
- /* Write a magic word of the bootconfig */
- ret = write(fd, BOOTCONFIG_MAGIC, BOOTCONFIG_MAGIC_LEN);
- if (ret < 0) {
- pr_err("Failed to apply a boot config magic: %d\n", ret);
- goto out;
- }
- ret = 0;
+
+ /* To align up the total size to BOOTCONFIG_ALIGN, get padding size */
+ total_size = stat.st_size + size + sizeof(u32) * 2 + BOOTCONFIG_MAGIC_LEN;
+ pad = ((total_size + BOOTCONFIG_ALIGN - 1) & (~BOOTCONFIG_ALIGN_MASK)) - total_size;
+ size += pad;
+
+ /* Add a footer */
+ p = data + size;
+ *(u32 *)p = htole32(size);
+ p += sizeof(u32);
+
+ *(u32 *)p = htole32(csum);
+ p += sizeof(u32);
+
+ memcpy(p, BOOTCONFIG_MAGIC, BOOTCONFIG_MAGIC_LEN);
+ p += BOOTCONFIG_MAGIC_LEN;
+
+ total_size = p - data;
+
+ ret = write(fd, data, total_size);
+ if (ret < total_size) {
+ if (ret < 0)
+ ret = -errno;
+ pr_err("Failed to apply a boot config: %d\n", ret);
+ if (ret >= 0)
+ goto out_rollback;
+ } else
+ ret = 0;
+
out:
close(fd);
free(data);
return ret;
+
+out_rollback:
+ /* Map the partial write to -ENOSPC */
+ if (ret >= 0)
+ ret = -ENOSPC;
+ if (ftruncate(fd, stat.st_size) < 0) {
+ ret = -errno;
+ pr_err("Failed to rollback the write error: %d\n", ret);
+ pr_err("The initrd %s may be corrupted. Recommend to rebuild.\n", path);
+ }
+ goto out;
}
static int usage(void)
TESTDIR=.
fi
BOOTCONF=${TESTDIR}/bootconfig
+ALIGN=4
INITRD=`mktemp ${TESTDIR}/initrd-XXXX`
TEMPCONF=`mktemp ${TESTDIR}/temp-XXXX.bconf`
xpass $BOOTCONF $INITRD
echo "File size check"
-xpass test $new_size -eq $(expr $bconf_size + $initrd_size + 9 + 12)
+total_size=$(expr $bconf_size + $initrd_size + 9 + 12 + $ALIGN - 1 )
+total_size=$(expr $total_size / $ALIGN)
+total_size=$(expr $total_size \* $ALIGN)
+xpass test $new_size -eq $total_size
echo "Apply command repeat test"
xpass $BOOTCONF -a $TEMPCONF $INITRD
obj_node = calloc(1, sizeof(*obj_node));
if (!obj_node) {
p_err("failed to allocate memory: %s", strerror(errno));
+ err = -ENOMEM;
goto err_free;
}
ifindex = net_parse_dev(&argc, &argv);
if (ifindex < 1) {
- close(progfd);
- return -EINVAL;
+ err = -EINVAL;
+ goto cleanup;
}
if (argc) {
overwrite = true;
} else {
p_err("expected 'overwrite', got: '%s'?", *argv);
- close(progfd);
- return -EINVAL;
+ err = -EINVAL;
+ goto cleanup;
}
}
if (is_prefix("xdp", attach_type_strings[attach_type]))
err = do_attach_detach_xdp(progfd, attach_type, ifindex,
overwrite);
-
- if (err < 0) {
+ if (err) {
p_err("interface %s attach failed: %s",
attach_type_strings[attach_type], strerror(-err));
- return err;
+ goto cleanup;
}
if (json_output)
jsonw_null(json_wtr);
-
- return 0;
+cleanup:
+ close(progfd);
+ return err;
}
static int do_detach(int argc, char **argv)
IFLA_MACVLAN_MACADDR,
IFLA_MACVLAN_MACADDR_DATA,
IFLA_MACVLAN_MACADDR_COUNT,
+ IFLA_MACVLAN_BC_QUEUE_LEN,
+ IFLA_MACVLAN_BC_QUEUE_LEN_USED,
__IFLA_MACVLAN_MAX,
};
awk '/GLOBAL/ && /DEFAULT/ && !/UND/ {print $$NF}' | \
sort -u | wc -l)
VERSIONED_SYM_COUNT = $(shell readelf --dyn-syms --wide $(OUTPUT)libbpf.so | \
+ sed 's/\[.*\]//' | \
awk '/GLOBAL/ && /DEFAULT/ && !/UND/ {print $$NF}' | \
grep -Eo '[^ ]+@LIBBPF_' | cut -d@ -f1 | sort -u | wc -l)
awk '/GLOBAL/ && /DEFAULT/ && !/UND/ {print $$NF}'| \
sort -u > $(OUTPUT)libbpf_global_syms.tmp; \
readelf --dyn-syms --wide $(OUTPUT)libbpf.so | \
+ sed 's/\[.*\]//' | \
awk '/GLOBAL/ && /DEFAULT/ && !/UND/ {print $$NF}'| \
grep -Eo '[^ ]+@LIBBPF_' | cut -d@ -f1 | \
sort -u > $(OUTPUT)libbpf_versioned_syms.tmp; \
const char *name, size_t sec_idx, const char *sec_name,
size_t sec_off, void *insn_data, size_t insn_data_sz)
{
- int i;
-
if (insn_data_sz == 0 || insn_data_sz % BPF_INSN_SZ || sec_off % BPF_INSN_SZ) {
pr_warn("sec '%s': corrupted program '%s', offset %zu, size %zu\n",
sec_name, name, sec_off, insn_data_sz);
goto errout;
memcpy(prog->insns, insn_data, insn_data_sz);
- for (i = 0; i < prog->insns_cnt; i++) {
- if (insn_is_subprog_call(&prog->insns[i])) {
- obj->has_subcalls = true;
- break;
- }
- }
-
return 0;
errout:
pr_warn("sec '%s': failed to allocate memory for prog '%s'\n", sec_name, name);
static bool prog_is_subprog(const struct bpf_object *obj,
const struct bpf_program *prog)
{
- return prog->sec_idx == obj->efile.text_shndx && obj->has_subcalls;
+ /* For legacy reasons, libbpf supports an entry-point BPF programs
+ * without SEC() attribute, i.e., those in the .text section. But if
+ * there are 2 or more such programs in the .text section, they all
+ * must be subprograms called from entry-point BPF programs in
+ * designated SEC()'tions, otherwise there is no way to distinguish
+ * which of those programs should be loaded vs which are a subprogram.
+ * Similarly, if there is a function/program in .text and at least one
+ * other BPF program with custom SEC() attribute, then we just assume
+ * .text programs are subprograms (even if they are not called from
+ * other programs), because libbpf never explicitly supported mixing
+ * SEC()-designated BPF programs and .text entry-point BPF programs.
+ */
+ return prog->sec_idx == obj->efile.text_shndx && obj->nr_programs > 1;
}
struct bpf_program *
stack_size = stack_size > STACK_SIZE ? STACK_SIZE : stack_size;
memcpy(buf, (void *) sp, stack_size);
+#ifdef MEMORY_SANITIZER
+ /*
+ * Copying the stack may copy msan poison, avoid false positives in the
+ * unwinder by removing the poison here.
+ */
+ __msan_unpoison(buf, stack_size);
+#endif
stack->data = (char *) buf;
stack->size = stack_size;
return 0;
/* Various wrappers to make the kernel .S file build in user-space: */
+// memcpy_orig and memcpy_erms are being defined as SYM_L_LOCAL but we need it
+#define SYM_FUNC_START_LOCAL(name) \
+ SYM_START(name, SYM_L_GLOBAL, SYM_A_ALIGN)
#define memcpy MEMCPY /* don't hide glibc's memcpy() */
#define altinstr_replacement text
#define globl p2align 4; .globl
/* SPDX-License-Identifier: GPL-2.0 */
+// memset_orig and memset_erms are being defined as SYM_L_LOCAL but we need it
+#define SYM_FUNC_START_LOCAL(name) \
+ SYM_START(name, SYM_L_GLOBAL, SYM_A_ALIGN)
#define memset MEMSET /* don't hide glibc's memset() */
#define altinstr_replacement text
#define globl p2align 4; .globl
if (compute == COMPUTE_STREAM) {
d->evlist_streams = evlist__create_streams(
d->session->evlist, 5);
- if (!d->evlist_streams)
+ if (!d->evlist_streams) {
+ ret = -ENOMEM;
goto out_delete;
+ }
}
}
dsos__hit_all(session);
/*
* The AUX areas have been removed and replaced with
- * synthesized hardware events, so clear the feature flag and
- * remove the evsel.
+ * synthesized hardware events, so clear the feature flag.
*/
if (inject->itrace_synth_opts.set) {
- struct evsel *evsel;
-
perf_header__clear_feat(&session->header,
HEADER_AUXTRACE);
if (inject->itrace_synth_opts.last_branch ||
inject->itrace_synth_opts.add_last_branch)
perf_header__set_feat(&session->header,
HEADER_BRANCH_STACK);
- evsel = perf_evlist__id2evsel_strict(session->evlist,
- inject->aux_id);
- if (evsel) {
- pr_debug("Deleting %s\n", evsel__name(evsel));
- evlist__remove(session->evlist, evsel);
- evsel__delete(evsel);
- }
}
session->header.data_offset = output_data_offset;
session->header.data_size = inject->bytes_written;
struct lock_seq_stat *seq;
const char *name = evsel__strval(evsel, sample, "name");
u64 tmp = evsel__intval(evsel, sample, "lockdep_addr");
- int flag = evsel__intval(evsel, sample, "flag");
+ int flag = evsel__intval(evsel, sample, "flags");
memcpy(&addr, &tmp, sizeof(void *));
case SEQ_STATE_READ_ACQUIRED:
seq->read_count--;
BUG_ON(seq->read_count < 0);
- if (!seq->read_count) {
+ if (seq->read_count) {
ls->nr_release++;
goto end;
}
# touch 6512 1 branches:u: ffffb22082e0 strcmp+0xa0 (/lib/aarch64-linux-gnu/ld-2.27.so)
# touch 6512 1 branches:u: ffffb2208320 strcmp+0xe0 (/lib/aarch64-linux-gnu/ld-2.27.so)
perf script -F,-time -i ${perfdata} | \
- egrep " +$1 +[0-9]+ .* +branches:([u|k]:)? +"
+ egrep " +$1 +[0-9]+ .* +branches:(.*:)? +"
}
perf_report_branch_samples() {
# `> device_name = 'tmc_etf0'
device_name=$(basename $path)
- if is_device_sink $path $devce_name; then
+ if is_device_sink $path $device_name; then
record_touch_file $device_name $2 &&
perf_script_branch_samples touch &&
bool die_is_func_def(Dwarf_Die *dw_die)
{
Dwarf_Attribute attr;
+ Dwarf_Addr addr = 0;
+
+ if (dwarf_tag(dw_die) != DW_TAG_subprogram)
+ return false;
+
+ if (dwarf_attr(dw_die, DW_AT_declaration, &attr))
+ return false;
- return (dwarf_tag(dw_die) == DW_TAG_subprogram &&
- dwarf_attr(dw_die, DW_AT_declaration, &attr) == NULL);
+ /*
+ * DW_AT_declaration can be lost from function declaration
+ * by gcc's bug #97060.
+ * So we need to check this subprogram DIE has DW_AT_inline
+ * or an entry address.
+ */
+ if (!dwarf_attr(dw_die, DW_AT_inline, &attr) &&
+ die_entrypc(dw_die, &addr) < 0)
+ return false;
+
+ return true;
}
/**
int die_entrypc(Dwarf_Die *dw_die, Dwarf_Addr *addr)
{
Dwarf_Addr base, end;
+ Dwarf_Attribute attr;
if (!addr)
return -EINVAL;
if (dwarf_entrypc(dw_die, addr) == 0)
return 0;
+ /*
+ * Since the dwarf_ranges() will return 0 if there is no
+ * DW_AT_ranges attribute, we should check it first.
+ */
+ if (!dwarf_attr(dw_die, DW_AT_ranges, &attr))
+ return -ENOENT;
+
return dwarf_ranges(dw_die, 0, &base, addr, &end) < 0 ? -ENOENT : 0;
}
static inline size_t hash_bits(size_t h, int bits)
{
/* shuffle bits and return requested number of upper bits */
+ if (bits == 0)
+ return 0;
+
#if (__SIZEOF_SIZE_T__ == __SIZEOF_LONG_LONG__)
/* LP64 case */
return (h * 11400714819323198485llu) >> (__SIZEOF_LONG_LONG__ * 8 - bits);
* @key: key to iterate entries for
*/
#define hashmap__for_each_key_entry(map, cur, _key) \
- for (cur = ({ size_t bkt = hash_bits(map->hash_fn((_key), map->ctx),\
- map->cap_bits); \
- map->buckets ? map->buckets[bkt] : NULL; }); \
+ for (cur = map->buckets \
+ ? map->buckets[hash_bits(map->hash_fn((_key), map->ctx), map->cap_bits)] \
+ : NULL; \
cur; \
cur = cur->next) \
if (map->equal_fn(cur->key, (_key), map->ctx))
#define hashmap__for_each_key_entry_safe(map, cur, tmp, _key) \
- for (cur = ({ size_t bkt = hash_bits(map->hash_fn((_key), map->ctx),\
- map->cap_bits); \
- cur = map->buckets ? map->buckets[bkt] : NULL; }); \
+ for (cur = map->buckets \
+ ? map->buckets[hash_bits(map->hash_fn((_key), map->ctx), map->cap_bits)] \
+ : NULL; \
cur && ({ tmp = cur->next; true; }); \
cur = tmp) \
if (map->equal_fn(cur->key, (_key), map->ctx))
/* SYM_L_* -- linkage of symbols */
#define SYM_L_GLOBAL(name) .globl name
+#define SYM_L_WEAK(name) .weak name
#define SYM_L_LOCAL(name) /* nothing */
#define ALIGN __ALIGN
SYM_END(name, SYM_T_FUNC)
#endif
+/* SYM_FUNC_START_WEAK -- use for weak functions */
+#ifndef SYM_FUNC_START_WEAK
+#define SYM_FUNC_START_WEAK(name) \
+ SYM_START(name, SYM_L_WEAK, SYM_A_ALIGN)
+#endif
+
/*
* SYM_FUNC_END -- the end of SYM_FUNC_START_LOCAL, SYM_FUNC_START,
* SYM_FUNC_START_WEAK, ...
if (lr->file && strtailcmp(lr->file, dwarf_decl_file(sp_die)))
return DWARF_CB_OK;
- if (die_is_func_def(sp_die) &&
- die_match_name(sp_die, lr->function)) {
+ if (die_match_name(sp_die, lr->function) && die_is_func_def(sp_die)) {
lf->fname = dwarf_decl_file(sp_die);
dwarf_decl_line(sp_die, &lr->offset);
pr_debug("fname: %s, lineno:%d\n", lf->fname, lr->offset);
struct evlist *evlist = evsel->evlist;
int i;
- if (!config->aggr_get_id)
- return 0;
-
if (config->aggr_mode == AGGR_NONE)
return id;
- if (config->aggr_mode == AGGR_GLOBAL)
+ if (!config->aggr_get_id)
return 0;
for (i = 0; i < evsel__nr_cpus(evsel); i++) {
char cgrp_root[PATH_MAX];
size_t mount_len; /* length of mount point in the path */
+ if (!tool || !tool->cgroup_events)
+ return 0;
+
if (cgroupfs_find_mountpoint(cgrp_root, PATH_MAX, "perf_event") < 0) {
pr_debug("cannot find cgroup mount point\n");
return -1;
+++ /dev/null
-test_data/* binary
import sys
import os
import time
-import shutil
from collections import namedtuple
from enum import Enum, auto
BUILD_FAILURE = auto()
TEST_FAILURE = auto()
-def create_default_kunitconfig():
- if not os.path.exists(kunit_kernel.kunitconfig_path):
- shutil.copyfile('arch/um/configs/kunit_defconfig',
- kunit_kernel.kunitconfig_path)
-
def get_kernel_root_path():
parts = sys.argv[0] if not __file__ else __file__
parts = os.path.realpath(parts).split('tools/testing/kunit')
kunit_parser.print_with_timestamp('Configuring KUnit Kernel ...')
config_start = time.time()
- create_default_kunitconfig()
success = linux.build_reconfig(request.build_dir, request.make_options)
config_end = time.time()
if not success:
if not os.path.exists(cli_args.build_dir):
os.mkdir(cli_args.build_dir)
- if not os.path.exists(kunit_kernel.kunitconfig_path):
- create_default_kunitconfig()
-
if not linux:
linux = kunit_kernel.LinuxSourceTree()
+ linux.create_kunitconfig(cli_args.build_dir)
+ linux.read_kunitconfig(cli_args.build_dir)
+
request = KunitRequest(cli_args.raw_output,
cli_args.timeout,
cli_args.jobs,
not os.path.exists(cli_args.build_dir)):
os.mkdir(cli_args.build_dir)
- if not os.path.exists(kunit_kernel.kunitconfig_path):
- create_default_kunitconfig()
-
if not linux:
linux = kunit_kernel.LinuxSourceTree()
+ linux.create_kunitconfig(cli_args.build_dir)
+ linux.read_kunitconfig(cli_args.build_dir)
+
request = KunitConfigRequest(cli_args.build_dir,
cli_args.make_options)
result = config_tests(linux, request)
if not linux:
linux = kunit_kernel.LinuxSourceTree()
+ linux.create_kunitconfig(cli_args.build_dir)
+ linux.read_kunitconfig(cli_args.build_dir)
+
request = KunitBuildRequest(cli_args.jobs,
cli_args.build_dir,
cli_args.alltests,
if not linux:
linux = kunit_kernel.LinuxSourceTree()
+ linux.create_kunitconfig(cli_args.build_dir)
+ linux.read_kunitconfig(cli_args.build_dir)
+
exec_request = KunitExecRequest(cli_args.timeout,
cli_args.build_dir,
cli_args.alltests)
kunit_output = f.read().splitlines()
request = KunitParseRequest(cli_args.raw_output,
kunit_output,
- cli_args.build_dir,
+ None,
cli_args.json)
result = parse_tests(request)
if result.status != KunitStatus.SUCCESS:
# Author: Felix Guo <felixguoxiuping@gmail.com>
# Author: Brendan Higgins <brendanhiggins@google.com>
-
import logging
import subprocess
import os
+import shutil
import signal
from contextlib import ExitStack
import kunit_parser
KCONFIG_PATH = '.config'
-kunitconfig_path = '.kunitconfig'
+KUNITCONFIG_PATH = '.kunitconfig'
+DEFAULT_KUNITCONFIG_PATH = 'arch/um/configs/kunit_defconfig'
BROKEN_ALLCONFIG_PATH = 'tools/testing/kunit/configs/broken_on_uml.config'
+OUTFILE_PATH = 'test.log'
class ConfigError(Exception):
"""Represents an error trying to configure the Linux kernel."""
if build_dir:
command += ['O=' + build_dir]
try:
- subprocess.check_output(command, stderr=subprocess.STDOUT)
+ proc = subprocess.Popen(command,
+ stderr=subprocess.PIPE,
+ stdout=subprocess.DEVNULL)
except OSError as e:
- raise BuildError('Could not call execute make: ' + str(e))
- except subprocess.CalledProcessError as e:
- raise BuildError(e.output.decode())
-
- def linux_bin(self, params, timeout, build_dir, outfile):
+ raise BuildError('Could not call make command: ' + str(e))
+ _, stderr = proc.communicate()
+ if proc.returncode != 0:
+ raise BuildError(stderr.decode())
+ if stderr: # likely only due to build warnings
+ print(stderr.decode())
+
+ def linux_bin(self, params, timeout, build_dir):
"""Runs the Linux UML binary. Must be named 'linux'."""
linux_bin = './linux'
if build_dir:
linux_bin = os.path.join(build_dir, 'linux')
+ outfile = get_outfile_path(build_dir)
with open(outfile, 'w') as output:
process = subprocess.Popen([linux_bin] + params,
stdout=output,
stderr=subprocess.STDOUT)
process.wait(timeout)
-
def get_kconfig_path(build_dir):
kconfig_path = KCONFIG_PATH
if build_dir:
kconfig_path = os.path.join(build_dir, KCONFIG_PATH)
return kconfig_path
+def get_kunitconfig_path(build_dir):
+ kunitconfig_path = KUNITCONFIG_PATH
+ if build_dir:
+ kunitconfig_path = os.path.join(build_dir, KUNITCONFIG_PATH)
+ return kunitconfig_path
+
+def get_outfile_path(build_dir):
+ outfile_path = OUTFILE_PATH
+ if build_dir:
+ outfile_path = os.path.join(build_dir, OUTFILE_PATH)
+ return outfile_path
+
class LinuxSourceTree(object):
"""Represents a Linux kernel source tree with KUnit tests."""
def __init__(self):
- self._kconfig = kunit_config.Kconfig()
- self._kconfig.read_from_file(kunitconfig_path)
self._ops = LinuxSourceTreeOperations()
signal.signal(signal.SIGINT, self.signal_handler)
return False
return True
+ def create_kunitconfig(self, build_dir, defconfig=DEFAULT_KUNITCONFIG_PATH):
+ kunitconfig_path = get_kunitconfig_path(build_dir)
+ if not os.path.exists(kunitconfig_path):
+ shutil.copyfile(defconfig, kunitconfig_path)
+
+ def read_kunitconfig(self, build_dir):
+ kunitconfig_path = get_kunitconfig_path(build_dir)
+ self._kconfig = kunit_config.Kconfig()
+ self._kconfig.read_from_file(kunitconfig_path)
+
def validate_config(self, build_dir):
kconfig_path = get_kconfig_path(build_dir)
validated_kconfig = kunit_config.Kconfig()
def run_kernel(self, args=[], build_dir='', timeout=None):
args.extend(['mem=1G'])
- outfile = 'test.log'
- self._ops.linux_bin(args, timeout, build_dir, outfile)
+ self._ops.linux_bin(args, timeout, build_dir)
+ outfile = get_outfile_path(build_dir)
subprocess.call(['stty', 'sane'])
with open(outfile, 'r') as file:
for line in file:
from datetime import datetime
from enum import Enum, auto
from functools import reduce
-from typing import List
+from typing import List, Optional, Tuple
TestResult = namedtuple('TestResult', ['status','suites','log'])
def isolate_kunit_output(kernel_output):
started = False
for line in kernel_output:
+ line = line.rstrip() # line always has a trailing \n
if kunit_start_re.search(line):
prefix_len = len(line.split('TAP version')[0])
started = True
def raw_output(kernel_output):
for line in kernel_output:
- print(line)
+ print(line.rstrip())
DIVIDER = '=' * 60
else:
return False
-def parse_test_case(lines: List[str]) -> TestCase:
+def parse_test_case(lines: List[str]) -> Optional[TestCase]:
test_case = TestCase()
save_non_diagnositic(lines, test_case)
while parse_diagnostic(lines, test_case):
SUBTEST_HEADER = re.compile(r'^[\s]+# Subtest: (.*)$')
-def parse_subtest_header(lines: List[str]) -> str:
+def parse_subtest_header(lines: List[str]) -> Optional[str]:
consume_non_diagnositic(lines)
if not lines:
return None
SUBTEST_PLAN = re.compile(r'[\s]+[0-9]+\.\.([0-9]+)')
-def parse_subtest_plan(lines: List[str]) -> int:
+def parse_subtest_plan(lines: List[str]) -> Optional[int]:
consume_non_diagnositic(lines)
match = SUBTEST_PLAN.match(lines[0])
if match:
max_test_case_status = bubble_up_errors(lambda x: x.status, test_suite.cases)
return max_status(max_test_case_status, test_suite.status)
-def parse_test_suite(lines: List[str], expected_suite_index: int) -> TestSuite:
+def parse_test_suite(lines: List[str], expected_suite_index: int) -> Optional[TestSuite]:
if not lines:
return None
consume_non_diagnositic(lines)
TEST_PLAN = re.compile(r'[0-9]+\.\.([0-9]+)')
-def parse_test_plan(lines: List[str]) -> int:
+def parse_test_plan(lines: List[str]) -> Optional[int]:
consume_non_diagnositic(lines)
match = TEST_PLAN.match(lines[0])
if match:
else:
return TestResult(TestStatus.NO_TESTS, [], lines)
-def print_and_count_results(test_result: TestResult) -> None:
+def print_and_count_results(test_result: TestResult) -> Tuple[int, int, int]:
total_tests = 0
failed_tests = 0
crashed_tests = 0
'test_data/test_output_isolated_correctly.log')
file = open(log_path)
result = kunit_parser.isolate_kunit_output(file.readlines())
- self.assertContains('TAP version 14\n', result)
+ self.assertContains('TAP version 14', result)
self.assertContains(' # Subtest: example', result)
self.assertContains(' 1..2', result)
self.assertContains(' ok 1 - example_simple_test', result)
'test_data/test_pound_sign.log')
with open(log_path) as file:
result = kunit_parser.isolate_kunit_output(file.readlines())
- self.assertContains('TAP version 14\n', result)
+ self.assertContains('TAP version 14', result)
self.assertContains(' # Subtest: kunit-resource-test', result)
self.assertContains(' 1..5', result)
self.assertContains(' ok 1 - kunit_resource_test_init_resources', result)
#define __ALIGN_KERNEL(x, a) __ALIGN_KERNEL_MASK(x, (typeof(x))(a) - 1)
#define __ALIGN_KERNEL_MASK(x, mask) (((x) + (mask)) & ~(mask))
#define ALIGN(x, a) __ALIGN_KERNEL((x), (a))
+#define ALIGN_DOWN(x, a) __ALIGN_KERNEL((x) - ((a) - 1), (a))
#define PAGE_ALIGN(addr) ALIGN(addr, PAGE_SIZE)
{
const unsigned int sgmax = SCATTERLIST_MAX_SEGMENT;
struct test *test, tests[] = {
- { -EINVAL, 1, pfn(0), PAGE_SIZE, PAGE_SIZE + 1, 1 },
{ -EINVAL, 1, pfn(0), PAGE_SIZE, 0, 1 },
- { -EINVAL, 1, pfn(0), PAGE_SIZE, sgmax + 1, 1 },
+ { 0, 1, pfn(0), PAGE_SIZE, PAGE_SIZE + 1, 1 },
+ { 0, 1, pfn(0), PAGE_SIZE, sgmax + 1, 1 },
{ 0, 1, pfn(0), PAGE_SIZE, sgmax, 1 },
{ 0, 1, pfn(0), 1, sgmax, 1 },
{ 0, 2, pfn(0, 1), 2 * PAGE_SIZE, sgmax, 1 },
--- /dev/null
+// SPDX-License-Identifier: GPL-2.0
+#include <test_progs.h>
+#include "test_probe_read_user_str.skel.h"
+
+static const char str1[] = "mestring";
+static const char str2[] = "mestringalittlebigger";
+static const char str3[] = "mestringblubblubblubblubblub";
+
+static int test_one_str(struct test_probe_read_user_str *skel, const char *str,
+ size_t len)
+{
+ int err, duration = 0;
+ char buf[256];
+
+ /* Ensure bytes after string are ones */
+ memset(buf, 1, sizeof(buf));
+ memcpy(buf, str, len);
+
+ /* Give prog our userspace pointer */
+ skel->bss->user_ptr = buf;
+
+ /* Trigger tracepoint */
+ usleep(1);
+
+ /* Did helper fail? */
+ if (CHECK(skel->bss->ret < 0, "prog_ret", "prog returned: %ld\n",
+ skel->bss->ret))
+ return 1;
+
+ /* Check that string was copied correctly */
+ err = memcmp(skel->bss->buf, str, len);
+ if (CHECK(err, "memcmp", "prog copied wrong string"))
+ return 1;
+
+ /* Now check that no extra trailing bytes were copied */
+ memset(buf, 0, sizeof(buf));
+ err = memcmp(skel->bss->buf + len, buf, sizeof(buf) - len);
+ if (CHECK(err, "memcmp", "trailing bytes were not stripped"))
+ return 1;
+
+ return 0;
+}
+
+void test_probe_read_user_str(void)
+{
+ struct test_probe_read_user_str *skel;
+ int err, duration = 0;
+
+ skel = test_probe_read_user_str__open_and_load();
+ if (CHECK(!skel, "test_probe_read_user_str__open_and_load",
+ "skeleton open and load failed\n"))
+ return;
+
+ /* Give pid to bpf prog so it doesn't read from anyone else */
+ skel->bss->pid = getpid();
+
+ err = test_probe_read_user_str__attach(skel);
+ if (CHECK(err, "test_probe_read_user_str__attach",
+ "skeleton attach failed: %d\n", err))
+ goto out;
+
+ if (test_one_str(skel, str1, sizeof(str1)))
+ goto out;
+ if (test_one_str(skel, str2, sizeof(str2)))
+ goto out;
+ if (test_one_str(skel, str3, sizeof(str3)))
+ goto out;
+
+out:
+ test_probe_read_user_str__destroy(skel);
+}
*/
buf = 0x40;
- if (setsockopt(sock_fd, SOL_IP, IP_TOS, &buf, 1) < 0) {
+ err = setsockopt(sock_fd, SOL_IP, IP_TOS, &buf, 1);
+ if (err < 0) {
log_err("Failed to call setsockopt(IP_TOS)");
goto detach;
}
#include <test_progs.h>
#include <time.h>
#include "test_subprogs.skel.h"
+#include "test_subprogs_unused.skel.h"
static int duration;
void test_subprogs(void)
{
struct test_subprogs *skel;
+ struct test_subprogs_unused *skel2;
int err;
skel = test_subprogs__open_and_load();
CHECK(skel->bss->res3 != 19, "res3", "got %d, exp %d\n", skel->bss->res3, 19);
CHECK(skel->bss->res4 != 36, "res4", "got %d, exp %d\n", skel->bss->res4, 36);
+ skel2 = test_subprogs_unused__open_and_load();
+ ASSERT_OK_PTR(skel2, "unused_progs_skel");
+ test_subprogs_unused__destroy(skel2);
+
cleanup:
test_subprogs__destroy(skel);
}
{ "test_global_func5.o" , "expected pointer to ctx, but got PTR" },
{ "test_global_func6.o" , "modified ctx ptr R2" },
{ "test_global_func7.o" , "foo() doesn't return scalar" },
+ { "test_global_func8.o" },
};
libbpf_print_fn_t old_print_fn = NULL;
int err, i, duration = 0;
--- /dev/null
+// SPDX-License-Identifier: GPL-2.0-only
+/* Copyright (c) 2020 Facebook */
+#include <stddef.h>
+#include <linux/bpf.h>
+#include <bpf/bpf_helpers.h>
+
+__noinline int foo(struct __sk_buff *skb)
+{
+ return bpf_get_prandom_u32();
+}
+
+SEC("cgroup_skb/ingress")
+int test_cls(struct __sk_buff *skb)
+{
+ if (!foo(skb))
+ return 0;
+
+ return 1;
+}
--- /dev/null
+// SPDX-License-Identifier: GPL-2.0
+
+#include <linux/bpf.h>
+#include <bpf/bpf_helpers.h>
+#include <bpf/bpf_tracing.h>
+
+#include <sys/types.h>
+
+pid_t pid = 0;
+long ret = 0;
+void *user_ptr = 0;
+char buf[256] = {};
+
+SEC("tracepoint/syscalls/sys_enter_nanosleep")
+int on_write(void *ctx)
+{
+ if (pid != (bpf_get_current_pid_tgid() >> 32))
+ return 0;
+
+ ret = bpf_probe_read_user_str(buf, sizeof(buf), user_ptr);
+
+ return 0;
+}
+
+char _license[] SEC("license") = "GPL";
--- /dev/null
+#include "vmlinux.h"
+#include <bpf/bpf_helpers.h>
+#include <bpf/bpf_core_read.h>
+
+const char LICENSE[] SEC("license") = "GPL";
+
+__attribute__((unused)) __noinline int unused1(int x)
+{
+ return x + 1;
+}
+
+static __attribute__((unused)) __noinline int unused2(int x)
+{
+ return x + 2;
+}
+
+SEC("raw_tp/sys_enter")
+int main_prog(void *ctx)
+{
+ return 0;
+}
blackhole_route_test
irif_disabled_test
erif_disabled_test
+ blackhole_nexthop_test
"
NUM_NETIFS=4
devlink_trap_action_set $trap_name "drop"
}
+__blackhole_nexthop_test()
+{
+ local flags=$1; shift
+ local subnet=$1; shift
+ local proto=$1; shift
+ local dip=$1; shift
+ local trap_name="blackhole_nexthop"
+ local mz_pid
+
+ RET=0
+
+ ip -$flags nexthop add id 1 blackhole
+ ip -$flags route add $subnet nhid 1
+ tc filter add dev $rp2 egress protocol $proto pref 1 handle 101 \
+ flower skip_hw dst_ip $dip ip_proto udp action drop
+
+ # Generate packets to the blackhole nexthop
+ $MZ $h1 -$flags -t udp "sp=54321,dp=12345" -c 0 -p 100 -b $rp1mac \
+ -B $dip -d 1msec -q &
+ mz_pid=$!
+
+ devlink_trap_drop_test $trap_name $rp2 101
+ log_test "Blackhole nexthop: IPv$flags"
+
+ devlink_trap_drop_cleanup $mz_pid $rp2 $proto 1 101
+ ip -$flags route del $subnet
+ ip -$flags nexthop del id 1
+}
+
+blackhole_nexthop_test()
+{
+ __blackhole_nexthop_test "4" "198.51.100.0/30" "ip" $h2_ipv4
+ __blackhole_nexthop_test "6" "2001:db8:2::/120" "ipv6" $h2_ipv6
+}
+
trap cleanup EXIT
setup_prepare
--- /dev/null
+#!/bin/bash
+# SPDX-License-Identifier: GPL-2.0
+
+lib_dir=$(dirname $0)/../../../net/forwarding
+
+ALL_TESTS="
+ create_8021ad_vlan_upper_on_top_front_panel_port
+ create_8021ad_vlan_upper_on_top_bridge_port
+ create_8021ad_vlan_upper_on_top_lag
+ create_8021ad_vlan_upper_on_top_bridge
+ create_8021ad_vlan_upper_on_top_8021ad_bridge
+ create_vlan_upper_on_top_8021ad_bridge
+ create_vlan_upper_on_top_front_panel_enslaved_to_8021ad_bridge
+ create_vlan_upper_on_top_lag_enslaved_to_8021ad_bridge
+ enslave_front_panel_with_vlan_upper_to_8021ad_bridge
+ enslave_lag_with_vlan_upper_to_8021ad_bridge
+ add_ip_address_to_8021ad_bridge
+ switch_bridge_protocol_from_8021q_to_8021ad
+"
+NUM_NETIFS=2
+source $lib_dir/lib.sh
+
+setup_prepare()
+{
+ swp1=${NETIFS[p1]}
+ swp2=${NETIFS[p2]}
+
+ ip link set dev $swp1 up
+ ip link set dev $swp2 up
+
+ sleep 10
+}
+
+cleanup()
+{
+ pre_cleanup
+
+ ip link set dev $swp2 down
+ ip link set dev $swp1 down
+}
+
+create_vlan_upper_on_top_of_bridge()
+{
+ RET=0
+
+ local bridge_proto=$1; shift
+ local netdev_proto=$1; shift
+
+ ip link add dev br0 type bridge vlan_filtering 1 \
+ vlan_protocol $bridge_proto vlan_default_pvid 0 mcast_snooping 0
+
+ ip link set dev br0 up
+ ip link set dev $swp1 master br0
+
+ ip link add name br0.100 link br0 type vlan \
+ protocol $netdev_proto id 100 2>/dev/null
+ check_fail $? "$netdev_proto vlan upper creation on top of an $bridge_proto bridge not rejected"
+
+ ip link add name br0.100 link br0 type vlan \
+ protocol $netdev_proto id 100 2>&1 >/dev/null \
+ | grep -q mlxsw_spectrum
+ check_err $? "$netdev_proto vlan upper creation on top of an $bridge_proto bridge rejected without extack"
+
+ log_test "create $netdev_proto vlan upper on top $bridge_proto bridge"
+
+ ip link del dev br0
+}
+
+create_8021ad_vlan_upper_on_top_front_panel_port()
+{
+ RET=0
+
+ ip link add name $swp1.100 link $swp1 type vlan \
+ protocol 802.1ad id 100 2>/dev/null
+ check_fail $? "802.1ad vlan upper creation on top of a front panel not rejected"
+
+ ip link add name $swp1.100 link $swp1 type vlan \
+ protocol 802.1ad id 100 2>&1 >/dev/null \
+ | grep -q mlxsw_spectrum
+ check_err $? "802.1ad vlan upper creation on top of a front panel rejected without extack"
+
+ log_test "create 802.1ad vlan upper on top of a front panel"
+}
+
+create_8021ad_vlan_upper_on_top_bridge_port()
+{
+ RET=0
+
+ ip link add dev br0 type bridge vlan_filtering 1 \
+ vlan_default_pvid 0 mcast_snooping 0
+
+ ip link set dev $swp1 master br0
+ ip link set dev br0 up
+
+ ip link add name $swp1.100 link $swp1 type vlan \
+ protocol 802.1ad id 100 2>/dev/null
+ check_fail $? "802.1ad vlan upper creation on top of a bridge port not rejected"
+
+ ip link add name $swp1.100 link $swp1 type vlan \
+ protocol 802.1ad id 100 2>&1 >/dev/null \
+ | grep -q mlxsw_spectrum
+ check_err $? "802.1ad vlan upper creation on top of a bridge port rejected without extack"
+
+ log_test "create 802.1ad vlan upper on top of a bridge port"
+
+ ip link del dev br0
+}
+
+create_8021ad_vlan_upper_on_top_lag()
+{
+ RET=0
+
+ ip link add name bond1 type bond mode 802.3ad
+ ip link set dev $swp1 down
+ ip link set dev $swp1 master bond1
+
+ ip link add name bond1.100 link bond1 type vlan \
+ protocol 802.1ad id 100 2>/dev/null
+ check_fail $? "802.1ad vlan upper creation on top of a lag not rejected"
+
+ ip link add name bond1.100 link bond1 type vlan \
+ protocol 802.1ad id 100 2>&1 >/dev/null \
+ | grep -q mlxsw_spectrum
+ check_err $? "802.1ad vlan upper creation on top of a lag rejected without extack"
+
+ log_test "create 802.1ad vlan upper on top of a lag"
+
+ ip link del dev bond1
+}
+
+create_8021ad_vlan_upper_on_top_bridge()
+{
+ RET=0
+
+ create_vlan_upper_on_top_of_bridge "802.1q" "802.1ad"
+}
+
+create_8021ad_vlan_upper_on_top_8021ad_bridge()
+{
+ RET=0
+
+ create_vlan_upper_on_top_of_bridge "802.1ad" "802.1ad"
+}
+
+create_vlan_upper_on_top_8021ad_bridge()
+{
+ RET=0
+
+ create_vlan_upper_on_top_of_bridge "802.1ad" "802.1q"
+}
+
+create_vlan_upper_on_top_front_panel_enslaved_to_8021ad_bridge()
+{
+ RET=0
+
+ ip link add dev br0 type bridge vlan_filtering 1 \
+ vlan_protocol 802.1ad vlan_default_pvid 0 mcast_snooping 0
+ ip link set dev br0 up
+
+ ip link set dev $swp1 master br0
+
+ ip link add name $swp1.100 link $swp1 type vlan id 100 2>/dev/null
+ check_fail $? "vlan upper creation on top of front panel enslaved to 802.1ad bridge not rejected"
+
+ ip link add name $swp1.100 link $swp1 type vlan id 100 2>&1 >/dev/null \
+ | grep -q mlxsw_spectrum
+ check_err $? "vlan upper creation on top of front panel enslaved to 802.1ad bridge rejected without extack"
+
+ log_test "create vlan upper on top of front panel enslaved to 802.1ad bridge"
+
+ ip link del dev br0
+}
+
+create_vlan_upper_on_top_lag_enslaved_to_8021ad_bridge()
+{
+ RET=0
+
+ ip link add dev br0 type bridge vlan_filtering 1 \
+ vlan_protocol 802.1ad vlan_default_pvid 0 mcast_snooping 0
+ ip link set dev br0 up
+
+ ip link add name bond1 type bond mode 802.3ad
+ ip link set dev $swp1 down
+ ip link set dev $swp1 master bond1
+ ip link set dev bond1 master br0
+
+ ip link add name bond1.100 link bond1 type vlan id 100 2>/dev/null
+ check_fail $? "vlan upper creation on top of lag enslaved to 802.1ad bridge not rejected"
+
+ ip link add name bond1.100 link bond1 type vlan id 100 2>&1 >/dev/null \
+ | grep -q mlxsw_spectrum
+ check_err $? "vlan upper creation on top of lag enslaved to 802.1ad bridge rejected without extack"
+
+ log_test "create vlan upper on top of lag enslaved to 802.1ad bridge"
+
+ ip link del dev bond1
+ ip link del dev br0
+}
+
+enslave_front_panel_with_vlan_upper_to_8021ad_bridge()
+{
+ RET=0
+
+ ip link add dev br0 type bridge vlan_filtering 1 \
+ vlan_protocol 802.1ad vlan_default_pvid 0 mcast_snooping 0
+ ip link set dev br0 up
+
+ ip link add name $swp1.100 link $swp1 type vlan id 100
+
+ ip link set dev $swp1 master br0 2>/dev/null
+ check_fail $? "front panel with vlan upper enslavemnt to 802.1ad bridge not rejected"
+
+ ip link set dev $swp1 master br0 2>&1 >/dev/null | grep -q mlxsw_spectrum
+ check_err $? "front panel with vlan upper enslavemnt to 802.1ad bridge rejected without extack"
+
+ log_test "enslave front panel with vlan upper to 802.1ad bridge"
+
+ ip link del dev $swp1.100
+ ip link del dev br0
+}
+
+enslave_lag_with_vlan_upper_to_8021ad_bridge()
+{
+ RET=0
+
+ ip link add dev br0 type bridge vlan_filtering 1 \
+ vlan_protocol 802.1ad vlan_default_pvid 0 mcast_snooping 0
+ ip link set dev br0 up
+
+ ip link add name bond1 type bond mode 802.3ad
+ ip link set dev $swp1 down
+ ip link set dev $swp1 master bond1
+ ip link add name bond1.100 link bond1 type vlan id 100
+
+ ip link set dev bond1 master br0 2>/dev/null
+ check_fail $? "lag with vlan upper enslavemnt to 802.1ad bridge not rejected"
+
+ ip link set dev bond1 master br0 2>&1 >/dev/null \
+ | grep -q mlxsw_spectrum
+ check_err $? "lag with vlan upper enslavemnt to 802.1ad bridge rejected without extack"
+
+ log_test "enslave lag with vlan upper to 802.1ad bridge"
+
+ ip link del dev bond1
+ ip link del dev br0
+}
+
+
+add_ip_address_to_8021ad_bridge()
+{
+ RET=0
+
+ ip link add dev br0 type bridge vlan_filtering 1 \
+ vlan_protocol 802.1ad vlan_default_pvid 0 mcast_snooping 0
+
+ ip link set dev br0 up
+ ip link set dev $swp1 master br0
+
+ ip addr add dev br0 192.0.2.17/28 2>/dev/null
+ check_fail $? "IP address addition to 802.1ad bridge not rejected"
+
+ ip addr add dev br0 192.0.2.17/28 2>&1 >/dev/null | grep -q mlxsw_spectrum
+ check_err $? "IP address addition to 802.1ad bridge rejected without extack"
+
+ log_test "IP address addition to 802.1ad bridge"
+
+ ip link del dev br0
+}
+
+switch_bridge_protocol_from_8021q_to_8021ad()
+{
+ RET=0
+
+ ip link add dev br0 type bridge vlan_filtering 1 \
+ vlan_protocol 802.1ad vlan_default_pvid 0 mcast_snooping 0
+
+ ip link set dev br0 up
+ ip link set dev $swp1 master br0
+
+ ip link set dev br0 type bridge vlan_protocol 802.1q 2>/dev/null
+ check_fail $? "switching bridge protocol from 802.1q to 802.1ad not rejected"
+
+ log_test "switch bridge protocol"
+
+ ip link del dev br0
+}
+
+
+trap cleanup EXIT
+
+setup_prepare
+setup_wait
+
+tests_run
+
+exit $EXIT_STATUS
bridge_extern_learn_test
neigh_offload_test
nexthop_offload_test
+ nexthop_obj_invalid_test
+ nexthop_obj_offload_test
+ nexthop_obj_group_offload_test
+ nexthop_obj_blackhole_offload_test
+ nexthop_obj_route_offload_test
devlink_reload_test
"
NUM_NETIFS=2
sysctl_restore net.ipv6.conf.$swp2.keep_addr_on_down
}
+nexthop_obj_invalid_test()
+{
+ # Test that invalid nexthop object configurations are rejected
+ RET=0
+
+ simple_if_init $swp1 192.0.2.1/24 2001:db8:1::1/64
+ simple_if_init $swp2 192.0.2.2/24 2001:db8:1::2/64
+ setup_wait
+
+ ip nexthop add id 1 via 192.0.2.3 fdb
+ check_fail $? "managed to configure an FDB nexthop when should not"
+
+ ip nexthop add id 1 encap mpls 200/300 via 192.0.2.3 dev $swp1
+ check_fail $? "managed to configure a nexthop with MPLS encap when should not"
+
+ ip nexthop add id 1 dev $swp1
+ ip nexthop add id 2 dev $swp1
+ ip nexthop add id 10 group 1/2
+ check_fail $? "managed to configure a nexthop group with device-only nexthops when should not"
+
+ log_test "nexthop objects - invalid configurations"
+
+ ip nexthop del id 2
+ ip nexthop del id 1
+
+ simple_if_fini $swp2 192.0.2.2/24 2001:db8:1::2/64
+ simple_if_fini $swp1 192.0.2.1/24 2001:db8:1::1/64
+}
+
+nexthop_obj_offload_test()
+{
+ # Test offload indication of nexthop objects
+ RET=0
+
+ simple_if_init $swp1 192.0.2.1/24 2001:db8:1::1/64
+ simple_if_init $swp2
+ setup_wait
+
+ ip nexthop add id 1 via 192.0.2.2 dev $swp1
+ ip neigh replace 192.0.2.2 lladdr 00:11:22:33:44:55 nud reachable \
+ dev $swp1
+
+ busywait "$TIMEOUT" wait_for_offload \
+ ip nexthop show id 1
+ check_err $? "nexthop not marked as offloaded when should"
+
+ ip neigh replace 192.0.2.2 nud failed dev $swp1
+ busywait "$TIMEOUT" not wait_for_offload \
+ ip nexthop show id 1
+ check_err $? "nexthop marked as offloaded after setting neigh to failed state"
+
+ ip neigh replace 192.0.2.2 lladdr 00:11:22:33:44:55 nud reachable \
+ dev $swp1
+ busywait "$TIMEOUT" wait_for_offload \
+ ip nexthop show id 1
+ check_err $? "nexthop not marked as offloaded after neigh replace"
+
+ ip nexthop replace id 1 via 192.0.2.3 dev $swp1
+ busywait "$TIMEOUT" not wait_for_offload \
+ ip nexthop show id 1
+ check_err $? "nexthop marked as offloaded after replacing to use an invalid address"
+
+ ip nexthop replace id 1 via 192.0.2.2 dev $swp1
+ busywait "$TIMEOUT" wait_for_offload \
+ ip nexthop show id 1
+ check_err $? "nexthop not marked as offloaded after replacing to use a valid address"
+
+ log_test "nexthop objects offload indication"
+
+ ip neigh del 192.0.2.2 dev $swp1
+ ip nexthop del id 1
+
+ simple_if_fini $swp2
+ simple_if_fini $swp1 192.0.2.1/24 2001:db8:1::1/64
+}
+
+nexthop_obj_group_offload_test()
+{
+ # Test offload indication of nexthop group objects
+ RET=0
+
+ simple_if_init $swp1 192.0.2.1/24 2001:db8:1::1/64
+ simple_if_init $swp2
+ setup_wait
+
+ ip nexthop add id 1 via 192.0.2.2 dev $swp1
+ ip nexthop add id 2 via 2001:db8:1::2 dev $swp1
+ ip nexthop add id 10 group 1/2
+ ip neigh replace 192.0.2.2 lladdr 00:11:22:33:44:55 nud reachable \
+ dev $swp1
+ ip neigh replace 192.0.2.3 lladdr 00:11:22:33:44:55 nud reachable \
+ dev $swp1
+ ip neigh replace 2001:db8:1::2 lladdr 00:11:22:33:44:55 nud reachable \
+ dev $swp1
+
+ busywait "$TIMEOUT" wait_for_offload \
+ ip nexthop show id 1
+ check_err $? "IPv4 nexthop not marked as offloaded when should"
+ busywait "$TIMEOUT" wait_for_offload \
+ ip nexthop show id 2
+ check_err $? "IPv6 nexthop not marked as offloaded when should"
+ busywait "$TIMEOUT" wait_for_offload \
+ ip nexthop show id 10
+ check_err $? "nexthop group not marked as offloaded when should"
+
+ # Invalidate nexthop id 1
+ ip neigh replace 192.0.2.2 nud failed dev $swp1
+ busywait "$TIMEOUT" not wait_for_offload \
+ ip nexthop show id 10
+ check_fail $? "nexthop group not marked as offloaded with one valid nexthop"
+
+ # Invalidate nexthop id 2
+ ip neigh replace 2001:db8:1::2 nud failed dev $swp1
+ busywait "$TIMEOUT" not wait_for_offload \
+ ip nexthop show id 10
+ check_err $? "nexthop group marked as offloaded when should not"
+
+ # Revalidate nexthop id 1
+ ip nexthop replace id 1 via 192.0.2.3 dev $swp1
+ busywait "$TIMEOUT" wait_for_offload \
+ ip nexthop show id 10
+ check_err $? "nexthop group not marked as offloaded after revalidating nexthop"
+
+ log_test "nexthop group objects offload indication"
+
+ ip neigh del 2001:db8:1::2 dev $swp1
+ ip neigh del 192.0.2.3 dev $swp1
+ ip neigh del 192.0.2.2 dev $swp1
+ ip nexthop del id 10
+ ip nexthop del id 2
+ ip nexthop del id 1
+
+ simple_if_fini $swp2
+ simple_if_fini $swp1 192.0.2.1/24 2001:db8:1::1/64
+}
+
+nexthop_obj_blackhole_offload_test()
+{
+ # Test offload indication of blackhole nexthop objects
+ RET=0
+
+ ip nexthop add id 1 blackhole
+ busywait "$TIMEOUT" wait_for_offload \
+ ip nexthop show id 1
+ check_err $? "Blackhole nexthop not marked as offloaded when should"
+
+ ip nexthop add id 10 group 1
+ busywait "$TIMEOUT" wait_for_offload \
+ ip nexthop show id 10
+ check_err $? "Nexthop group not marked as offloaded when should"
+
+ log_test "blackhole nexthop objects offload indication"
+
+ ip nexthop del id 10
+ ip nexthop del id 1
+}
+
+nexthop_obj_route_offload_test()
+{
+ # Test offload indication of routes using nexthop objects
+ RET=0
+
+ simple_if_init $swp1 192.0.2.1/24 2001:db8:1::1/64
+ simple_if_init $swp2
+ setup_wait
+
+ ip nexthop add id 1 via 192.0.2.2 dev $swp1
+ ip neigh replace 192.0.2.2 lladdr 00:11:22:33:44:55 nud reachable \
+ dev $swp1
+ ip neigh replace 192.0.2.3 lladdr 00:11:22:33:44:55 nud reachable \
+ dev $swp1
+
+ ip route replace 198.51.100.0/24 nhid 1
+ busywait "$TIMEOUT" wait_for_offload \
+ ip route show 198.51.100.0/24
+ check_err $? "route not marked as offloaded when using valid nexthop"
+
+ ip nexthop replace id 1 via 192.0.2.3 dev $swp1
+ busywait "$TIMEOUT" wait_for_offload \
+ ip route show 198.51.100.0/24
+ check_err $? "route not marked as offloaded after replacing valid nexthop with a valid one"
+
+ ip nexthop replace id 1 via 192.0.2.4 dev $swp1
+ busywait "$TIMEOUT" not wait_for_offload \
+ ip route show 198.51.100.0/24
+ check_err $? "route marked as offloaded after replacing valid nexthop with an invalid one"
+
+ ip nexthop replace id 1 via 192.0.2.2 dev $swp1
+ busywait "$TIMEOUT" wait_for_offload \
+ ip route show 198.51.100.0/24
+ check_err $? "route not marked as offloaded after replacing invalid nexthop with a valid one"
+
+ log_test "routes using nexthop objects offload indication"
+
+ ip route del 198.51.100.0/24
+ ip neigh del 192.0.2.3 dev $swp1
+ ip neigh del 192.0.2.2 dev $swp1
+ ip nexthop del id 1
+
+ simple_if_fini $swp2
+ simple_if_fini $swp1 192.0.2.1/24 2001:db8:1::1/64
+}
+
devlink_reload_test()
{
# Test that after executing all the above configuration tests, a
--- /dev/null
+#!/bin/bash
+# SPDX-License-Identifier: GPL-2.0-only
+
+source ethtool-common.sh
+
+function get_value {
+ local query="${SETTINGS_MAP[$1]}"
+
+ echo $(ethtool -c $NSIM_NETDEV | \
+ awk -F':' -v pattern="$query:" '$0 ~ pattern {gsub(/[ \t]/, "", $2); print $2}')
+}
+
+function update_current_settings {
+ for key in ${!SETTINGS_MAP[@]}; do
+ CURRENT_SETTINGS[$key]=$(get_value $key)
+ done
+ echo ${CURRENT_SETTINGS[@]}
+}
+
+if ! ethtool -h | grep -q coalesce; then
+ echo "SKIP: No --coalesce support in ethtool"
+ exit 4
+fi
+
+NSIM_NETDEV=$(make_netdev)
+
+set -o pipefail
+
+declare -A SETTINGS_MAP=(
+ ["rx-frames-low"]="rx-frame-low"
+ ["tx-frames-low"]="tx-frame-low"
+ ["rx-frames-high"]="rx-frame-high"
+ ["tx-frames-high"]="tx-frame-high"
+ ["rx-usecs"]="rx-usecs"
+ ["rx-frames"]="rx-frames"
+ ["rx-usecs-irq"]="rx-usecs-irq"
+ ["rx-frames-irq"]="rx-frames-irq"
+ ["tx-usecs"]="tx-usecs"
+ ["tx-frames"]="tx-frames"
+ ["tx-usecs-irq"]="tx-usecs-irq"
+ ["tx-frames-irq"]="tx-frames-irq"
+ ["stats-block-usecs"]="stats-block-usecs"
+ ["pkt-rate-low"]="pkt-rate-low"
+ ["rx-usecs-low"]="rx-usecs-low"
+ ["tx-usecs-low"]="tx-usecs-low"
+ ["pkt-rate-high"]="pkt-rate-high"
+ ["rx-usecs-high"]="rx-usecs-high"
+ ["tx-usecs-high"]="tx-usecs-high"
+ ["sample-interval"]="sample-interval"
+)
+
+declare -A CURRENT_SETTINGS=(
+ ["rx-frames-low"]=""
+ ["tx-frames-low"]=""
+ ["rx-frames-high"]=""
+ ["tx-frames-high"]=""
+ ["rx-usecs"]=""
+ ["rx-frames"]=""
+ ["rx-usecs-irq"]=""
+ ["rx-frames-irq"]=""
+ ["tx-usecs"]=""
+ ["tx-frames"]=""
+ ["tx-usecs-irq"]=""
+ ["tx-frames-irq"]=""
+ ["stats-block-usecs"]=""
+ ["pkt-rate-low"]=""
+ ["rx-usecs-low"]=""
+ ["tx-usecs-low"]=""
+ ["pkt-rate-high"]=""
+ ["rx-usecs-high"]=""
+ ["tx-usecs-high"]=""
+ ["sample-interval"]=""
+)
+
+declare -A EXPECTED_SETTINGS=(
+ ["rx-frames-low"]=""
+ ["tx-frames-low"]=""
+ ["rx-frames-high"]=""
+ ["tx-frames-high"]=""
+ ["rx-usecs"]=""
+ ["rx-frames"]=""
+ ["rx-usecs-irq"]=""
+ ["rx-frames-irq"]=""
+ ["tx-usecs"]=""
+ ["tx-frames"]=""
+ ["tx-usecs-irq"]=""
+ ["tx-frames-irq"]=""
+ ["stats-block-usecs"]=""
+ ["pkt-rate-low"]=""
+ ["rx-usecs-low"]=""
+ ["tx-usecs-low"]=""
+ ["pkt-rate-high"]=""
+ ["rx-usecs-high"]=""
+ ["tx-usecs-high"]=""
+ ["sample-interval"]=""
+)
+
+# populate the expected settings map
+for key in ${!SETTINGS_MAP[@]}; do
+ EXPECTED_SETTINGS[$key]=$(get_value $key)
+done
+
+# test
+for key in ${!SETTINGS_MAP[@]}; do
+ value=$((RANDOM % $((2**32-1))))
+
+ ethtool -C $NSIM_NETDEV "$key" "$value"
+
+ EXPECTED_SETTINGS[$key]="$value"
+ expected=${EXPECTED_SETTINGS[@]}
+ current=$(update_current_settings)
+
+ check $? "$current" "$expected"
+ set +x
+done
+
+# bool settings which ethtool displays on the same line
+ethtool -C $NSIM_NETDEV adaptive-rx on
+s=$(ethtool -c $NSIM_NETDEV | grep -q "Adaptive RX: on TX: off")
+check $? "$s" ""
+
+ethtool -C $NSIM_NETDEV adaptive-tx on
+s=$(ethtool -c $NSIM_NETDEV | grep -q "Adaptive RX: on TX: on")
+check $? "$s" ""
+
+if [ $num_errors -eq 0 ]; then
+ echo "PASSED all $((num_passes)) checks"
+ exit 0
+else
+ echo "FAILED $num_errors/$((num_errors+num_passes)) checks"
+ exit 1
+fi
--- /dev/null
+#!/bin/bash
+# SPDX-License-Identifier: GPL-2.0-only
+
+NSIM_ID=$((RANDOM % 1024))
+NSIM_DEV_SYS=/sys/bus/netdevsim/devices/netdevsim$NSIM_ID
+NSIM_DEV_DFS=/sys/kernel/debug/netdevsim/netdevsim$NSIM_ID/ports/0
+NSIM_NETDEV=
+num_passes=0
+num_errors=0
+
+function cleanup_nsim {
+ if [ -e $NSIM_DEV_SYS ]; then
+ echo $NSIM_ID > /sys/bus/netdevsim/del_device
+ fi
+}
+
+function cleanup {
+ cleanup_nsim
+}
+
+trap cleanup EXIT
+
+function check {
+ local code=$1
+ local str=$2
+ local exp_str=$3
+
+ if [ $code -ne 0 ]; then
+ ((num_errors++))
+ return
+ fi
+
+ if [ "$str" != "$exp_str" ]; then
+ echo -e "Expected: '$exp_str', got '$str'"
+ ((num_errors++))
+ return
+ fi
+
+ ((num_passes++))
+}
+
+function make_netdev {
+ # Make a netdevsim
+ old_netdevs=$(ls /sys/class/net)
+
+ if ! $(lsmod | grep -q netdevsim); then
+ modprobe netdevsim
+ fi
+
+ echo $NSIM_ID > /sys/bus/netdevsim/new_device
+ # get new device name
+ ls /sys/bus/netdevsim/devices/netdevsim${NSIM_ID}/net/
+}
#!/bin/bash
# SPDX-License-Identifier: GPL-2.0-only
-NSIM_ID=$((RANDOM % 1024))
-NSIM_DEV_SYS=/sys/bus/netdevsim/devices/netdevsim$NSIM_ID
-NSIM_DEV_DFS=/sys/kernel/debug/netdevsim/netdevsim$NSIM_ID/ports/0
-NSIM_NETDEV=
-num_passes=0
-num_errors=0
-
-function cleanup_nsim {
- if [ -e $NSIM_DEV_SYS ]; then
- echo $NSIM_ID > /sys/bus/netdevsim/del_device
- fi
-}
-
-function cleanup {
- cleanup_nsim
-}
-
-trap cleanup EXIT
-
-function get_netdev_name {
- local -n old=$1
-
- new=$(ls /sys/class/net)
-
- for netdev in $new; do
- for check in $old; do
- [ $netdev == $check ] && break
- done
-
- if [ $netdev != $check ]; then
- echo $netdev
- break
- fi
- done
-}
-
-function check {
- local code=$1
- local str=$2
- local exp_str=$3
-
- if [ $code -ne 0 ]; then
- ((num_errors++))
- return
- fi
-
- if [ "$str" != "$exp_str" ]; then
- echo -e "Expected: '$exp_str', got '$str'"
- ((num_errors++))
- return
- fi
-
- ((num_passes++))
-}
+source ethtool-common.sh
# Bail if ethtool is too old
if ! ethtool -h | grep include-stat 2>&1 >/dev/null; then
exit 4
fi
-# Make a netdevsim
-old_netdevs=$(ls /sys/class/net)
-
-modprobe netdevsim
-echo $NSIM_ID > /sys/bus/netdevsim/new_device
-
-NSIM_NETDEV=`get_netdev_name old_netdevs`
+NSIM_NETDEV=$(make_netdev)
set -o pipefail
--- /dev/null
+#!/bin/bash
+# SPDX-License-Identifier: GPL-2.0-only
+
+source ethtool-common.sh
+
+function get_value {
+ local query="${SETTINGS_MAP[$1]}"
+
+ echo $(ethtool -g $NSIM_NETDEV | \
+ tail -n +$CURR_SETT_LINE | \
+ awk -F':' -v pattern="$query:" '$0 ~ pattern {gsub(/[\t ]/, "", $2); print $2}')
+}
+
+function update_current_settings {
+ for key in ${!SETTINGS_MAP[@]}; do
+ CURRENT_SETTINGS[$key]=$(get_value $key)
+ done
+ echo ${CURRENT_SETTINGS[@]}
+}
+
+if ! ethtool -h | grep -q set-ring >/dev/null; then
+ echo "SKIP: No --set-ring support in ethtool"
+ exit 4
+fi
+
+NSIM_NETDEV=$(make_netdev)
+
+set -o pipefail
+
+declare -A SETTINGS_MAP=(
+ ["rx"]="RX"
+ ["rx-mini"]="RX Mini"
+ ["rx-jumbo"]="RX Jumbo"
+ ["tx"]="TX"
+)
+
+declare -A EXPECTED_SETTINGS=(
+ ["rx"]=""
+ ["rx-mini"]=""
+ ["rx-jumbo"]=""
+ ["tx"]=""
+)
+
+declare -A CURRENT_SETTINGS=(
+ ["rx"]=""
+ ["rx-mini"]=""
+ ["rx-jumbo"]=""
+ ["tx"]=""
+)
+
+MAX_VALUE=$((RANDOM % $((2**32-1))))
+RING_MAX_LIST=$(ls $NSIM_DEV_DFS/ethtool/ring/)
+
+for ring_max_entry in $RING_MAX_LIST; do
+ echo $MAX_VALUE > $NSIM_DEV_DFS/ethtool/ring/$ring_max_entry
+done
+
+CURR_SETT_LINE=$(ethtool -g $NSIM_NETDEV | grep -i -m1 -n 'Current hardware settings' | cut -f1 -d:)
+
+# populate the expected settings map
+for key in ${!SETTINGS_MAP[@]}; do
+ EXPECTED_SETTINGS[$key]=$(get_value $key)
+done
+
+# test
+for key in ${!SETTINGS_MAP[@]}; do
+ value=$((RANDOM % $MAX_VALUE))
+
+ ethtool -G $NSIM_NETDEV "$key" "$value"
+
+ EXPECTED_SETTINGS[$key]="$value"
+ expected=${EXPECTED_SETTINGS[@]}
+ current=$(update_current_settings)
+
+ check $? "$current" "$expected"
+ set +x
+done
+
+if [ $num_errors -eq 0 ]; then
+ echo "PASSED all $((num_passes)) checks"
+ exit 0
+else
+ echo "FAILED $num_errors/$((num_errors+num_passes)) checks"
+ exit 1
+fi
--- /dev/null
+#!/bin/bash
+# SPDX-License-Identifier: GPL-2.0
+
+# Test traffic distribution when a wECMP route forwards traffic to two GRE
+# tunnels.
+#
+# +-------------------------+
+# | H1 |
+# | $h1 + |
+# | 192.0.2.1/28 | |
+# | 2001:db8:1::1/64 | |
+# +-------------------|-----+
+# |
+# +-------------------|------------------------+
+# | SW1 | |
+# | $ol1 + |
+# | 192.0.2.2/28 |
+# | 2001:db8:1::2/64 |
+# | |
+# | + g1a (gre) + g1b (gre) |
+# | loc=192.0.2.65 loc=192.0.2.81 |
+# | rem=192.0.2.66 --. rem=192.0.2.82 --. |
+# | tos=inherit | tos=inherit | |
+# | .------------------' | |
+# | | .------------------' |
+# | v v |
+# | + $ul1.111 (vlan) + $ul1.222 (vlan) |
+# | | 192.0.2.129/28 | 192.0.2.145/28 |
+# | \ / |
+# | \________________/ |
+# | | |
+# | + $ul1 |
+# +------------|-------------------------------+
+# |
+# +------------|-------------------------------+
+# | SW2 + $ul2 |
+# | _______|________ |
+# | / \ |
+# | / \ |
+# | + $ul2.111 (vlan) + $ul2.222 (vlan) |
+# | ^ 192.0.2.130/28 ^ 192.0.2.146/28 |
+# | | | |
+# | | '------------------. |
+# | '------------------. | |
+# | + g2a (gre) | + g2b (gre) | |
+# | loc=192.0.2.66 | loc=192.0.2.82 | |
+# | rem=192.0.2.65 --' rem=192.0.2.81 --' |
+# | tos=inherit tos=inherit |
+# | |
+# | $ol2 + |
+# | 192.0.2.17/28 | |
+# | 2001:db8:2::1/64 | |
+# +-------------------|------------------------+
+# |
+# +-------------------|-----+
+# | H2 | |
+# | $h2 + |
+# | 192.0.2.18/28 |
+# | 2001:db8:2::2/64 |
+# +-------------------------+
+
+ALL_TESTS="
+ ping_ipv4
+ ping_ipv6
+ multipath_ipv4
+ multipath_ipv6
+ multipath_ipv6_l4
+"
+
+NUM_NETIFS=6
+source lib.sh
+
+h1_create()
+{
+ simple_if_init $h1 192.0.2.1/28 2001:db8:1::1/64
+ ip route add vrf v$h1 192.0.2.16/28 via 192.0.2.2
+ ip route add vrf v$h1 2001:db8:2::/64 via 2001:db8:1::2
+}
+
+h1_destroy()
+{
+ ip route del vrf v$h1 2001:db8:2::/64 via 2001:db8:1::2
+ ip route del vrf v$h1 192.0.2.16/28 via 192.0.2.2
+ simple_if_fini $h1 192.0.2.1/28
+}
+
+sw1_create()
+{
+ simple_if_init $ol1 192.0.2.2/28 2001:db8:1::2/64
+ __simple_if_init $ul1 v$ol1
+ vlan_create $ul1 111 v$ol1 192.0.2.129/28
+ vlan_create $ul1 222 v$ol1 192.0.2.145/28
+
+ tunnel_create g1a gre 192.0.2.65 192.0.2.66 tos inherit dev v$ol1
+ __simple_if_init g1a v$ol1 192.0.2.65/32
+ ip route add vrf v$ol1 192.0.2.66/32 via 192.0.2.130
+
+ tunnel_create g1b gre 192.0.2.81 192.0.2.82 tos inherit dev v$ol1
+ __simple_if_init g1b v$ol1 192.0.2.81/32
+ ip route add vrf v$ol1 192.0.2.82/32 via 192.0.2.146
+
+ ip -6 nexthop add id 101 dev g1a
+ ip -6 nexthop add id 102 dev g1b
+ ip nexthop add id 103 group 101/102
+
+ ip route add vrf v$ol1 192.0.2.16/28 nhid 103
+ ip route add vrf v$ol1 2001:db8:2::/64 nhid 103
+}
+
+sw1_destroy()
+{
+ ip route del vrf v$ol1 2001:db8:2::/64
+ ip route del vrf v$ol1 192.0.2.16/28
+
+ ip nexthop del id 103
+ ip -6 nexthop del id 102
+ ip -6 nexthop del id 101
+
+ ip route del vrf v$ol1 192.0.2.82/32 via 192.0.2.146
+ __simple_if_fini g1b 192.0.2.81/32
+ tunnel_destroy g1b
+
+ ip route del vrf v$ol1 192.0.2.66/32 via 192.0.2.130
+ __simple_if_fini g1a 192.0.2.65/32
+ tunnel_destroy g1a
+
+ vlan_destroy $ul1 222
+ vlan_destroy $ul1 111
+ __simple_if_fini $ul1
+ simple_if_fini $ol1 192.0.2.2/28 2001:db8:1::2/64
+}
+
+sw2_create()
+{
+ simple_if_init $ol2 192.0.2.17/28 2001:db8:2::1/64
+ __simple_if_init $ul2 v$ol2
+ vlan_create $ul2 111 v$ol2 192.0.2.130/28
+ vlan_create $ul2 222 v$ol2 192.0.2.146/28
+
+ tunnel_create g2a gre 192.0.2.66 192.0.2.65 tos inherit dev v$ol2
+ __simple_if_init g2a v$ol2 192.0.2.66/32
+ ip route add vrf v$ol2 192.0.2.65/32 via 192.0.2.129
+
+ tunnel_create g2b gre 192.0.2.82 192.0.2.81 tos inherit dev v$ol2
+ __simple_if_init g2b v$ol2 192.0.2.82/32
+ ip route add vrf v$ol2 192.0.2.81/32 via 192.0.2.145
+
+ ip -6 nexthop add id 201 dev g2a
+ ip -6 nexthop add id 202 dev g2b
+ ip nexthop add id 203 group 201/202
+
+ ip route add vrf v$ol2 192.0.2.0/28 nhid 203
+ ip route add vrf v$ol2 2001:db8:1::/64 nhid 203
+
+ tc qdisc add dev $ul2 clsact
+ tc filter add dev $ul2 ingress pref 111 prot 802.1Q \
+ flower vlan_id 111 action pass
+ tc filter add dev $ul2 ingress pref 222 prot 802.1Q \
+ flower vlan_id 222 action pass
+}
+
+sw2_destroy()
+{
+ tc qdisc del dev $ul2 clsact
+
+ ip route del vrf v$ol2 2001:db8:1::/64
+ ip route del vrf v$ol2 192.0.2.0/28
+
+ ip nexthop del id 203
+ ip -6 nexthop del id 202
+ ip -6 nexthop del id 201
+
+ ip route del vrf v$ol2 192.0.2.81/32 via 192.0.2.145
+ __simple_if_fini g2b 192.0.2.82/32
+ tunnel_destroy g2b
+
+ ip route del vrf v$ol2 192.0.2.65/32 via 192.0.2.129
+ __simple_if_fini g2a 192.0.2.66/32
+ tunnel_destroy g2a
+
+ vlan_destroy $ul2 222
+ vlan_destroy $ul2 111
+ __simple_if_fini $ul2
+ simple_if_fini $ol2 192.0.2.17/28 2001:db8:2::1/64
+}
+
+h2_create()
+{
+ simple_if_init $h2 192.0.2.18/28 2001:db8:2::2/64
+ ip route add vrf v$h2 192.0.2.0/28 via 192.0.2.17
+ ip route add vrf v$h2 2001:db8:1::/64 via 2001:db8:2::1
+}
+
+h2_destroy()
+{
+ ip route del vrf v$h2 2001:db8:1::/64 via 2001:db8:2::1
+ ip route del vrf v$h2 192.0.2.0/28 via 192.0.2.17
+ simple_if_fini $h2 192.0.2.18/28 2001:db8:2::2/64
+}
+
+setup_prepare()
+{
+ h1=${NETIFS[p1]}
+ ol1=${NETIFS[p2]}
+
+ ul1=${NETIFS[p3]}
+ ul2=${NETIFS[p4]}
+
+ ol2=${NETIFS[p5]}
+ h2=${NETIFS[p6]}
+
+ vrf_prepare
+ h1_create
+ sw1_create
+ sw2_create
+ h2_create
+
+ forwarding_enable
+}
+
+cleanup()
+{
+ pre_cleanup
+
+ forwarding_restore
+
+ h2_destroy
+ sw2_destroy
+ sw1_destroy
+ h1_destroy
+ vrf_cleanup
+}
+
+multipath4_test()
+{
+ local what=$1; shift
+ local weight1=$1; shift
+ local weight2=$1; shift
+
+ sysctl_set net.ipv4.fib_multipath_hash_policy 1
+ ip nexthop replace id 103 group 101,$weight1/102,$weight2
+
+ local t0_111=$(tc_rule_stats_get $ul2 111 ingress)
+ local t0_222=$(tc_rule_stats_get $ul2 222 ingress)
+
+ ip vrf exec v$h1 \
+ $MZ $h1 -q -p 64 -A 192.0.2.1 -B 192.0.2.18 \
+ -d 1msec -t udp "sp=1024,dp=0-32768"
+
+ local t1_111=$(tc_rule_stats_get $ul2 111 ingress)
+ local t1_222=$(tc_rule_stats_get $ul2 222 ingress)
+
+ local d111=$((t1_111 - t0_111))
+ local d222=$((t1_222 - t0_222))
+ multipath_eval "$what" $weight1 $weight2 $d111 $d222
+
+ ip nexthop replace id 103 group 101/102
+ sysctl_restore net.ipv4.fib_multipath_hash_policy
+}
+
+multipath6_test()
+{
+ local what=$1; shift
+ local weight1=$1; shift
+ local weight2=$1; shift
+
+ sysctl_set net.ipv6.fib_multipath_hash_policy 0
+ ip nexthop replace id 103 group 101,$weight1/102,$weight2
+
+ local t0_111=$(tc_rule_stats_get $ul2 111 ingress)
+ local t0_222=$(tc_rule_stats_get $ul2 222 ingress)
+
+ # Generate 16384 echo requests, each with a random flow label.
+ for ((i=0; i < 16384; ++i)); do
+ ip vrf exec v$h1 $PING6 2001:db8:2::2 -F 0 -c 1 -q &> /dev/null
+ done
+
+ local t1_111=$(tc_rule_stats_get $ul2 111 ingress)
+ local t1_222=$(tc_rule_stats_get $ul2 222 ingress)
+
+ local d111=$((t1_111 - t0_111))
+ local d222=$((t1_222 - t0_222))
+ multipath_eval "$what" $weight1 $weight2 $d111 $d222
+
+ ip nexthop replace id 103 group 101/102
+ sysctl_restore net.ipv6.fib_multipath_hash_policy
+}
+
+multipath6_l4_test()
+{
+ local what=$1; shift
+ local weight1=$1; shift
+ local weight2=$1; shift
+
+ sysctl_set net.ipv6.fib_multipath_hash_policy 1
+ ip nexthop replace id 103 group 101,$weight1/102,$weight2
+
+ local t0_111=$(tc_rule_stats_get $ul2 111 ingress)
+ local t0_222=$(tc_rule_stats_get $ul2 222 ingress)
+
+ ip vrf exec v$h1 \
+ $MZ $h1 -6 -q -p 64 -A 2001:db8:1::1 -B 2001:db8:2::2 \
+ -d 1msec -t udp "sp=1024,dp=0-32768"
+
+ local t1_111=$(tc_rule_stats_get $ul2 111 ingress)
+ local t1_222=$(tc_rule_stats_get $ul2 222 ingress)
+
+ local d111=$((t1_111 - t0_111))
+ local d222=$((t1_222 - t0_222))
+ multipath_eval "$what" $weight1 $weight2 $d111 $d222
+
+ ip nexthop replace id 103 group 101/102
+ sysctl_restore net.ipv6.fib_multipath_hash_policy
+}
+
+ping_ipv4()
+{
+ ping_test $h1 192.0.2.18
+}
+
+ping_ipv6()
+{
+ ping6_test $h1 2001:db8:2::2
+}
+
+multipath_ipv4()
+{
+ log_info "Running IPv4 multipath tests"
+ multipath4_test "ECMP" 1 1
+ multipath4_test "Weighted MP 2:1" 2 1
+ multipath4_test "Weighted MP 11:45" 11 45
+}
+
+multipath_ipv6()
+{
+ log_info "Running IPv6 multipath tests"
+ multipath6_test "ECMP" 1 1
+ multipath6_test "Weighted MP 2:1" 2 1
+ multipath6_test "Weighted MP 11:45" 11 45
+}
+
+multipath_ipv6_l4()
+{
+ log_info "Running IPv6 L4 hash multipath tests"
+ multipath6_l4_test "ECMP" 1 1
+ multipath6_l4_test "Weighted MP 2:1" 2 1
+ multipath6_l4_test "Weighted MP 11:45" 11 45
+}
+
+trap cleanup EXIT
+
+setup_prepare
+setup_wait
+tests_run
+
+exit $EXIT_STATUS
#!/bin/bash
# SPDX-License-Identifier: GPL-2.0
-ALL_TESTS="ping_ipv4 ping_ipv6 multipath_test"
+ALL_TESTS="
+ ping_ipv4
+ ping_ipv6
+ multipath_test
+ ping_ipv4_blackhole
+ ping_ipv6_blackhole
+"
NUM_NETIFS=8
source lib.sh
multipath4_test "Weighted MP 2:1" 2 1
multipath4_test "Weighted MP 11:45" 11 45
+ log_info "Running IPv4 multipath tests with IPv6 link-local nexthops"
+ ip nexthop replace id 101 via fe80:2::22 dev $rp12
+ ip nexthop replace id 102 via fe80:3::23 dev $rp13
+
+ multipath4_test "ECMP" 1 1
+ multipath4_test "Weighted MP 2:1" 2 1
+ multipath4_test "Weighted MP 11:45" 11 45
+
+ ip nexthop replace id 102 via 169.254.3.23 dev $rp13
+ ip nexthop replace id 101 via 169.254.2.22 dev $rp12
+
log_info "Running IPv6 multipath tests"
multipath6_test "ECMP" 1 1
multipath6_test "Weighted MP 2:1" 2 1
multipath6_l4_test "Weighted MP 11:45" 11 45
}
+ping_ipv4_blackhole()
+{
+ RET=0
+
+ ip nexthop add id 1001 blackhole
+ ip nexthop add id 1002 group 1001
+
+ ip route replace 198.51.100.0/24 vrf vrf-r1 nhid 1001
+ ping_do $h1 198.51.100.2
+ check_fail $? "ping did not fail when using a blackhole nexthop"
+
+ ip route replace 198.51.100.0/24 vrf vrf-r1 nhid 1002
+ ping_do $h1 198.51.100.2
+ check_fail $? "ping did not fail when using a blackhole nexthop group"
+
+ ip route replace 198.51.100.0/24 vrf vrf-r1 nhid 103
+ ping_do $h1 198.51.100.2
+ check_err $? "ping failed with a valid nexthop"
+
+ log_test "IPv4 blackhole ping"
+
+ ip nexthop del id 1002
+ ip nexthop del id 1001
+}
+
+ping_ipv6_blackhole()
+{
+ RET=0
+
+ ip -6 nexthop add id 1001 blackhole
+ ip nexthop add id 1002 group 1001
+
+ ip route replace 2001:db8:2::/64 vrf vrf-r1 nhid 1001
+ ping6_do $h1 2001:db8:2::2
+ check_fail $? "ping did not fail when using a blackhole nexthop"
+
+ ip route replace 2001:db8:2::/64 vrf vrf-r1 nhid 1002
+ ping6_do $h1 2001:db8:2::2
+ check_fail $? "ping did not fail when using a blackhole nexthop group"
+
+ ip route replace 2001:db8:2::/64 vrf vrf-r1 nhid 106
+ ping6_do $h1 2001:db8:2::2
+ check_err $? "ping failed with a valid nexthop"
+
+ log_test "IPv6 blackhole ping"
+
+ ip nexthop del id 1002
+ ip -6 nexthop del id 1001
+}
+
setup_prepare()
{
h1=${NETIFS[p1]}
router1_create
router2_create
- routing_nh_obj
forwarding_enable
}
--- /dev/null
+#!/bin/bash
+# SPDX-License-Identifier: GPL-2.0
+
+ALL_TESTS="
+ ping_ipv4
+ ping_ipv6
+"
+
+NUM_NETIFS=4
+source lib.sh
+source tc_common.sh
+
+h1_create()
+{
+ vrf_create "vrf-h1"
+ ip link set dev $h1 master vrf-h1
+
+ ip link set dev vrf-h1 up
+ ip link set dev $h1 up
+
+ ip address add 192.0.2.2/24 dev $h1
+ ip address add 2001:db8:1::2/64 dev $h1
+
+ ip route add 198.51.100.0/24 vrf vrf-h1 nexthop via 192.0.2.1
+ ip route add 2001:db8:2::/64 vrf vrf-h1 nexthop via 2001:db8:1::1
+}
+
+h1_destroy()
+{
+ ip route del 2001:db8:2::/64 vrf vrf-h1
+ ip route del 198.51.100.0/24 vrf vrf-h1
+
+ ip address del 2001:db8:1::2/64 dev $h1
+ ip address del 192.0.2.2/24 dev $h1
+
+ ip link set dev $h1 down
+ vrf_destroy "vrf-h1"
+}
+
+h2_create()
+{
+ vrf_create "vrf-h2"
+ ip link set dev $h2 master vrf-h2
+
+ ip link set dev vrf-h2 up
+ ip link set dev $h2 up
+
+ ip address add 198.51.100.2/24 dev $h2
+ ip address add 2001:db8:2::2/64 dev $h2
+
+ ip route add 192.0.2.0/24 vrf vrf-h2 nexthop via 198.51.100.1
+ ip route add 2001:db8:1::/64 vrf vrf-h2 nexthop via 2001:db8:2::1
+}
+
+h2_destroy()
+{
+ ip route del 2001:db8:1::/64 vrf vrf-h2
+ ip route del 192.0.2.0/24 vrf vrf-h2
+
+ ip address del 2001:db8:2::2/64 dev $h2
+ ip address del 198.51.100.2/24 dev $h2
+
+ ip link set dev $h2 down
+ vrf_destroy "vrf-h2"
+}
+
+router_create()
+{
+ ip link set dev $rp1 up
+ ip link set dev $rp2 up
+
+ tc qdisc add dev $rp2 clsact
+
+ ip address add 192.0.2.1/24 dev $rp1
+ ip address add 2001:db8:1::1/64 dev $rp1
+
+ ip address add 198.51.100.1/24 dev $rp2
+ ip address add 2001:db8:2::1/64 dev $rp2
+}
+
+router_destroy()
+{
+ ip address del 2001:db8:2::1/64 dev $rp2
+ ip address del 198.51.100.1/24 dev $rp2
+
+ ip address del 2001:db8:1::1/64 dev $rp1
+ ip address del 192.0.2.1/24 dev $rp1
+
+ tc qdisc del dev $rp2 clsact
+
+ ip link set dev $rp2 down
+ ip link set dev $rp1 down
+}
+
+routing_nh_obj()
+{
+ # Create the nexthops as AF_INET6, so that IPv4 and IPv6 routes could
+ # use them.
+ ip -6 nexthop add id 101 dev $rp1
+ ip -6 nexthop add id 102 dev $rp2
+
+ ip route replace 192.0.2.0/24 nhid 101
+ ip route replace 2001:db8:1::/64 nhid 101
+ ip route replace 198.51.100.0/24 nhid 102
+ ip route replace 2001:db8:2::/64 nhid 102
+}
+
+setup_prepare()
+{
+ h1=${NETIFS[p1]}
+ rp1=${NETIFS[p2]}
+
+ rp2=${NETIFS[p3]}
+ h2=${NETIFS[p4]}
+
+ rp1mac=$(mac_get $rp1)
+
+ vrf_prepare
+
+ h1_create
+ h2_create
+
+ router_create
+
+ forwarding_enable
+}
+
+cleanup()
+{
+ pre_cleanup
+
+ forwarding_restore
+
+ router_destroy
+
+ h2_destroy
+ h1_destroy
+
+ vrf_cleanup
+}
+
+ping_ipv4()
+{
+ ping_test $h1 198.51.100.2
+}
+
+ping_ipv6()
+{
+ ping6_test $h1 2001:db8:2::2
+}
+
+trap cleanup EXIT
+
+setup_prepare
+setup_wait
+routing_nh_obj
+
+tests_run
+
+exit $EXIT_STATUS
sin=""
sout=""
cin=""
+cinsent=""
cout=""
ksft_skip=4
timeout=30
cleanup()
{
rm -f "$cin" "$cout"
- rm -f "$sin" "$sout"
+ rm -f "$sin" "$sout" "$cinsent"
cleanup_partial
}
exit $ksft_skip
fi
+print_file_err()
+{
+ ls -l "$1" 1>&2
+ echo "Trailing bytes are: "
+ tail -c 27 "$1"
+}
+
check_transfer()
{
in=$1
echo "[ FAIL ] $what does not match (in, out):"
print_file_err "$in"
print_file_err "$out"
+ ret=1
return 1
fi
fi
}
+link_failure()
+{
+ ns="$1"
+
+ l=$((RANDOM%4))
+ l=$((l+1))
+
+ veth="ns1eth$l"
+ ip -net "$ns" link set "$veth" down
+}
+
+# $1: IP address
+is_v6()
+{
+ [ -z "${1##*:*}" ]
+}
+
do_transfer()
{
listener_ns="$1"
cl_proto="$3"
srv_proto="$4"
connect_addr="$5"
- rm_nr_ns1="$6"
- rm_nr_ns2="$7"
- speed="$8"
+ test_link_fail="$6"
+ rm_nr_ns1="$7"
+ rm_nr_ns2="$8"
+ speed="$9"
port=$((10000+$TEST_COUNT))
TEST_COUNT=$((TEST_COUNT+1))
mptcp_connect="./mptcp_connect -r"
fi
- ip netns exec ${listener_ns} $mptcp_connect -t $timeout -l -p $port -s ${srv_proto} 0.0.0.0 < "$sin" > "$sout" &
+ local local_addr
+ if is_v6 "${connect_addr}"; then
+ local_addr="::"
+ else
+ local_addr="0.0.0.0"
+ fi
+
+ ip netns exec ${listener_ns} $mptcp_connect -t $timeout -l -p $port \
+ -s ${srv_proto} ${local_addr} < "$sin" > "$sout" &
spid=$!
sleep 1
- ip netns exec ${connector_ns} $mptcp_connect -t $timeout -p $port -s ${cl_proto} $connect_addr < "$cin" > "$cout" &
+ if [ "$test_link_fail" -eq 0 ];then
+ ip netns exec ${connector_ns} $mptcp_connect -t $timeout -p $port -s ${cl_proto} $connect_addr < "$cin" > "$cout" &
+ else
+ ( cat "$cin" ; sleep 2; link_failure $listener_ns ; cat "$cin" ) | tee "$cinsent" | \
+ ip netns exec ${connector_ns} $mptcp_connect -t $timeout -p $port -s ${cl_proto} $connect_addr > "$cout" &
+ fi
cpid=$!
if [ $rm_nr_ns1 -gt 0 ]; then
ip netns exec ${connector_ns} ss -nita 1>&2 -o "dport = :$port"
cat "$capout"
+ ret=1
return 1
fi
check_transfer $sin $cout "file received by client"
retc=$?
- check_transfer $cin $sout "file received by server"
+ if [ "$test_link_fail" -eq 0 ];then
+ check_transfer $cin $sout "file received by server"
+ else
+ check_transfer $cinsent $sout "file received by server"
+ fi
rets=$?
if [ $retc -eq 0 ] && [ $rets -eq 0 ];then
{
name=$1
who=$2
+ size=$3
- SIZE=1
-
- dd if=/dev/urandom of="$name" bs=1024 count=$SIZE 2> /dev/null
+ dd if=/dev/urandom of="$name" bs=1024 count=$size 2> /dev/null
echo -e "\nMPTCP_TEST_FILE_END_MARKER" >> "$name"
- echo "Created $name (size $SIZE KB) containing data sent by $who"
+ echo "Created $name (size $size KB) containing data sent by $who"
}
run_tests()
listener_ns="$1"
connector_ns="$2"
connect_addr="$3"
- rm_nr_ns1="${4:-0}"
- rm_nr_ns2="${5:-0}"
- speed="${6:-fast}"
+ test_linkfail="${4:-0}"
+ rm_nr_ns1="${5:-0}"
+ rm_nr_ns2="${6:-0}"
+ speed="${7:-fast}"
lret=0
+ oldin=""
+
+ if [ "$test_linkfail" -eq 1 ];then
+ size=$((RANDOM%1024))
+ size=$((size+1))
+ size=$((size*128))
+
+ oldin=$(mktemp)
+ cp "$cin" "$oldin"
+ make_file "$cin" "client" $size
+ fi
do_transfer ${listener_ns} ${connector_ns} MPTCP MPTCP ${connect_addr} \
- ${rm_nr_ns1} ${rm_nr_ns2} ${speed}
+ ${test_linkfail} ${rm_nr_ns1} ${rm_nr_ns2} ${speed}
lret=$?
+
+ if [ "$test_linkfail" -eq 1 ];then
+ cp "$oldin" "$cin"
+ rm -f "$oldin"
+ fi
+
if [ $lret -ne 0 ]; then
ret=$lret
return
sin=$(mktemp)
sout=$(mktemp)
cin=$(mktemp)
+cinsent=$(mktemp)
cout=$(mktemp)
init
-make_file "$cin" "client"
-make_file "$sin" "server"
+make_file "$cin" "client" 1
+make_file "$sin" "server" 1
trap cleanup EXIT
run_tests $ns1 $ns2 10.0.1.1
chk_join_nr "multiple subflows and signal" 3 3 3
chk_add_nr 1 1
+# accept and use add_addr with additional subflows and link loss
+reset
+ip netns exec $ns1 ./pm_nl_ctl limits 0 3
+ip netns exec $ns1 ./pm_nl_ctl add 10.0.2.1 flags signal
+ip netns exec $ns2 ./pm_nl_ctl limits 1 3
+ip netns exec $ns2 ./pm_nl_ctl add 10.0.3.2 flags subflow
+ip netns exec $ns2 ./pm_nl_ctl add 10.0.4.2 flags subflow
+run_tests $ns1 $ns2 10.0.1.1 1
+chk_join_nr "multiple flows, signal, link failure" 3 3 3
+chk_add_nr 1 1
+
# add_addr timeout
reset_with_add_addr_timeout
ip netns exec $ns1 ./pm_nl_ctl limits 0 1
ip netns exec $ns2 ./pm_nl_ctl limits 1 1
ip netns exec $ns1 ./pm_nl_ctl add 10.0.2.1 flags signal
-run_tests $ns1 $ns2 10.0.1.1 0 0 slow
+run_tests $ns1 $ns2 10.0.1.1 0 0 0 slow
chk_join_nr "signal address, ADD_ADDR timeout" 1 1 1
chk_add_nr 4 0
ip netns exec $ns1 ./pm_nl_ctl limits 0 1
ip netns exec $ns2 ./pm_nl_ctl limits 0 1
ip netns exec $ns2 ./pm_nl_ctl add 10.0.3.2 flags subflow
-run_tests $ns1 $ns2 10.0.1.1 0 1 slow
+run_tests $ns1 $ns2 10.0.1.1 0 0 1 slow
chk_join_nr "remove single subflow" 1 1 1
chk_rm_nr 1 1
ip netns exec $ns2 ./pm_nl_ctl limits 0 2
ip netns exec $ns2 ./pm_nl_ctl add 10.0.2.2 flags subflow
ip netns exec $ns2 ./pm_nl_ctl add 10.0.3.2 flags subflow
-run_tests $ns1 $ns2 10.0.1.1 0 2 slow
+run_tests $ns1 $ns2 10.0.1.1 0 0 2 slow
chk_join_nr "remove multiple subflows" 2 2 2
chk_rm_nr 2 2
ip netns exec $ns1 ./pm_nl_ctl limits 0 1
ip netns exec $ns1 ./pm_nl_ctl add 10.0.2.1 flags signal
ip netns exec $ns2 ./pm_nl_ctl limits 1 1
-run_tests $ns1 $ns2 10.0.1.1 1 0 slow
+run_tests $ns1 $ns2 10.0.1.1 0 1 0 slow
chk_join_nr "remove single address" 1 1 1
chk_add_nr 1 1
chk_rm_nr 0 0
ip netns exec $ns1 ./pm_nl_ctl add 10.0.2.1 flags signal
ip netns exec $ns2 ./pm_nl_ctl limits 1 2
ip netns exec $ns2 ./pm_nl_ctl add 10.0.3.2 flags subflow
-run_tests $ns1 $ns2 10.0.1.1 1 1 slow
+run_tests $ns1 $ns2 10.0.1.1 0 1 1 slow
chk_join_nr "remove subflow and signal" 2 2 2
chk_add_nr 1 1
chk_rm_nr 1 1
ip netns exec $ns2 ./pm_nl_ctl limits 1 3
ip netns exec $ns2 ./pm_nl_ctl add 10.0.3.2 flags subflow
ip netns exec $ns2 ./pm_nl_ctl add 10.0.4.2 flags subflow
-run_tests $ns1 $ns2 10.0.1.1 1 2 slow
+run_tests $ns1 $ns2 10.0.1.1 0 1 2 slow
chk_join_nr "remove subflows and signal" 3 3 3
chk_add_nr 1 1
chk_rm_nr 2 2
+# subflow IPv6
+reset
+ip netns exec $ns1 ./pm_nl_ctl limits 0 1
+ip netns exec $ns2 ./pm_nl_ctl limits 0 1
+ip netns exec $ns2 ./pm_nl_ctl add dead:beef:3::2 flags subflow
+run_tests $ns1 $ns2 dead:beef:1::1 0 0 0 slow
+chk_join_nr "single subflow IPv6" 1 1 1
+
+# add_address, unused IPv6
+reset
+ip netns exec $ns1 ./pm_nl_ctl add dead:beef:2::1 flags signal
+run_tests $ns1 $ns2 dead:beef:1::1 0 0 0 slow
+chk_join_nr "unused signal address IPv6" 0 0 0
+chk_add_nr 1 1
+
+# signal address IPv6
+reset
+ip netns exec $ns1 ./pm_nl_ctl limits 0 1
+ip netns exec $ns1 ./pm_nl_ctl add dead:beef:2::1 flags signal
+ip netns exec $ns2 ./pm_nl_ctl limits 1 1
+run_tests $ns1 $ns2 dead:beef:1::1 0 0 0 slow
+chk_join_nr "single address IPv6" 1 1 1
+chk_add_nr 1 1
+
+# add_addr timeout IPv6
+reset_with_add_addr_timeout 6
+ip netns exec $ns1 ./pm_nl_ctl limits 0 1
+ip netns exec $ns2 ./pm_nl_ctl limits 1 1
+ip netns exec $ns1 ./pm_nl_ctl add dead:beef:2::1 flags signal
+run_tests $ns1 $ns2 dead:beef:1::1 0 0 0 slow
+chk_join_nr "signal address, ADD_ADDR6 timeout" 1 1 1
+chk_add_nr 4 0
+
+# single address IPv6, remove
+reset
+ip netns exec $ns1 ./pm_nl_ctl limits 0 1
+ip netns exec $ns1 ./pm_nl_ctl add dead:beef:2::1 flags signal
+ip netns exec $ns2 ./pm_nl_ctl limits 1 1
+run_tests $ns1 $ns2 dead:beef:1::1 0 1 0 slow
+chk_join_nr "remove single address IPv6" 1 1 1
+chk_add_nr 1 1
+chk_rm_nr 0 0
+
+# subflow and signal IPv6, remove
+reset
+ip netns exec $ns1 ./pm_nl_ctl limits 0 2
+ip netns exec $ns1 ./pm_nl_ctl add dead:beef:2::1 flags signal
+ip netns exec $ns2 ./pm_nl_ctl limits 1 2
+ip netns exec $ns2 ./pm_nl_ctl add dead:beef:3::2 flags subflow
+run_tests $ns1 $ns2 dead:beef:1::1 0 1 1 slow
+chk_join_nr "remove subflow and signal IPv6" 2 2 2
+chk_add_nr 1 1
+chk_rm_nr 1 1
+
# single subflow, syncookies
reset_with_cookies
ip netns exec $ns1 ./pm_nl_ctl limits 0 1
FIXTURE_VARIANT(tls)
{
- unsigned int tls_version;
+ u16 tls_version;
+ u16 cipher_type;
};
-FIXTURE_VARIANT_ADD(tls, 12)
+FIXTURE_VARIANT_ADD(tls, 12_gcm)
{
.tls_version = TLS_1_2_VERSION,
+ .cipher_type = TLS_CIPHER_AES_GCM_128,
};
-FIXTURE_VARIANT_ADD(tls, 13)
+FIXTURE_VARIANT_ADD(tls, 13_gcm)
{
.tls_version = TLS_1_3_VERSION,
+ .cipher_type = TLS_CIPHER_AES_GCM_128,
+};
+
+FIXTURE_VARIANT_ADD(tls, 12_chacha)
+{
+ .tls_version = TLS_1_2_VERSION,
+ .cipher_type = TLS_CIPHER_CHACHA20_POLY1305,
+};
+
+FIXTURE_VARIANT_ADD(tls, 13_chacha)
+{
+ .tls_version = TLS_1_3_VERSION,
+ .cipher_type = TLS_CIPHER_CHACHA20_POLY1305,
};
FIXTURE_SETUP(tls)
{
- struct tls12_crypto_info_aes_gcm_128 tls12;
+ union tls_crypto_context tls12;
struct sockaddr_in addr;
socklen_t len;
int sfd, ret;
+ size_t tls12_sz;
self->notls = false;
len = sizeof(addr);
memset(&tls12, 0, sizeof(tls12));
tls12.info.version = variant->tls_version;
- tls12.info.cipher_type = TLS_CIPHER_AES_GCM_128;
+ tls12.info.cipher_type = variant->cipher_type;
+ switch (variant->cipher_type) {
+ case TLS_CIPHER_CHACHA20_POLY1305:
+ tls12_sz = sizeof(tls12_crypto_info_chacha20_poly1305);
+ break;
+ case TLS_CIPHER_AES_GCM_128:
+ tls12_sz = sizeof(tls12_crypto_info_aes_gcm_128);
+ break;
+ default:
+ tls12_sz = 0;
+ }
addr.sin_family = AF_INET;
addr.sin_addr.s_addr = htonl(INADDR_ANY);
if (!self->notls) {
ret = setsockopt(self->fd, SOL_TLS, TLS_TX, &tls12,
- sizeof(tls12));
+ tls12_sz);
ASSERT_EQ(ret, 0);
}
ASSERT_EQ(ret, 0);
ret = setsockopt(self->cfd, SOL_TLS, TLS_RX, &tls12,
- sizeof(tls12));
+ tls12_sz);
ASSERT_EQ(ret, 0);
}
int perf_event_disable(int fd);
int perf_event_reset(int fd);
+struct perf_event_read {
+ __u64 nr;
+ __u64 l1d_misses;
+};
+
#if !defined(__GLIBC_PREREQ) || !__GLIBC_PREREQ(2, 30)
#include <unistd.h>
#include <sys/syscall.h>
# SPDX-License-Identifier: GPL-2.0-only
rfi_flush
+entry_flush
# SPDX-License-Identifier: GPL-2.0+
-TEST_GEN_PROGS := rfi_flush spectre_v2
+TEST_GEN_PROGS := rfi_flush entry_flush spectre_v2
top_srcdir = ../../../../..
CFLAGS += -I../../../../../usr/include
$(OUTPUT)/spectre_v2: CFLAGS += -m64
$(OUTPUT)/spectre_v2: ../pmu/event.c branch_loops.S
+$(OUTPUT)/rfi_flush: flush_utils.c
+$(OUTPUT)/entry_flush: flush_utils.c
--- /dev/null
+// SPDX-License-Identifier: GPL-2.0+
+
+/*
+ * Copyright 2018 IBM Corporation.
+ */
+
+#define __SANE_USERSPACE_TYPES__
+
+#include <sys/types.h>
+#include <stdint.h>
+#include <malloc.h>
+#include <unistd.h>
+#include <signal.h>
+#include <stdlib.h>
+#include <string.h>
+#include <stdio.h>
+#include "utils.h"
+#include "flush_utils.h"
+
+int entry_flush_test(void)
+{
+ char *p;
+ int repetitions = 10;
+ int fd, passes = 0, iter, rc = 0;
+ struct perf_event_read v;
+ __u64 l1d_misses_total = 0;
+ unsigned long iterations = 100000, zero_size = 24 * 1024;
+ unsigned long l1d_misses_expected;
+ int rfi_flush_orig;
+ int entry_flush, entry_flush_orig;
+
+ SKIP_IF(geteuid() != 0);
+
+ // The PMU event we use only works on Power7 or later
+ SKIP_IF(!have_hwcap(PPC_FEATURE_ARCH_2_06));
+
+ if (read_debugfs_file("powerpc/rfi_flush", &rfi_flush_orig) < 0) {
+ perror("Unable to read powerpc/rfi_flush debugfs file");
+ SKIP_IF(1);
+ }
+
+ if (read_debugfs_file("powerpc/entry_flush", &entry_flush_orig) < 0) {
+ perror("Unable to read powerpc/entry_flush debugfs file");
+ SKIP_IF(1);
+ }
+
+ if (rfi_flush_orig != 0) {
+ if (write_debugfs_file("powerpc/rfi_flush", 0) < 0) {
+ perror("error writing to powerpc/rfi_flush debugfs file");
+ FAIL_IF(1);
+ }
+ }
+
+ entry_flush = entry_flush_orig;
+
+ fd = perf_event_open_counter(PERF_TYPE_RAW, /* L1d miss */ 0x400f0, -1);
+ FAIL_IF(fd < 0);
+
+ p = (char *)memalign(zero_size, CACHELINE_SIZE);
+
+ FAIL_IF(perf_event_enable(fd));
+
+ // disable L1 prefetching
+ set_dscr(1);
+
+ iter = repetitions;
+
+ /*
+ * We expect to see l1d miss for each cacheline access when entry_flush
+ * is set. Allow a small variation on this.
+ */
+ l1d_misses_expected = iterations * (zero_size / CACHELINE_SIZE - 2);
+
+again:
+ FAIL_IF(perf_event_reset(fd));
+
+ syscall_loop(p, iterations, zero_size);
+
+ FAIL_IF(read(fd, &v, sizeof(v)) != sizeof(v));
+
+ if (entry_flush && v.l1d_misses >= l1d_misses_expected)
+ passes++;
+ else if (!entry_flush && v.l1d_misses < (l1d_misses_expected / 2))
+ passes++;
+
+ l1d_misses_total += v.l1d_misses;
+
+ while (--iter)
+ goto again;
+
+ if (passes < repetitions) {
+ printf("FAIL (L1D misses with entry_flush=%d: %llu %c %lu) [%d/%d failures]\n",
+ entry_flush, l1d_misses_total, entry_flush ? '<' : '>',
+ entry_flush ? repetitions * l1d_misses_expected :
+ repetitions * l1d_misses_expected / 2,
+ repetitions - passes, repetitions);
+ rc = 1;
+ } else {
+ printf("PASS (L1D misses with entry_flush=%d: %llu %c %lu) [%d/%d pass]\n",
+ entry_flush, l1d_misses_total, entry_flush ? '>' : '<',
+ entry_flush ? repetitions * l1d_misses_expected :
+ repetitions * l1d_misses_expected / 2,
+ passes, repetitions);
+ }
+
+ if (entry_flush == entry_flush_orig) {
+ entry_flush = !entry_flush_orig;
+ if (write_debugfs_file("powerpc/entry_flush", entry_flush) < 0) {
+ perror("error writing to powerpc/entry_flush debugfs file");
+ return 1;
+ }
+ iter = repetitions;
+ l1d_misses_total = 0;
+ passes = 0;
+ goto again;
+ }
+
+ perf_event_disable(fd);
+ close(fd);
+
+ set_dscr(0);
+
+ if (write_debugfs_file("powerpc/rfi_flush", rfi_flush_orig) < 0) {
+ perror("unable to restore original value of powerpc/rfi_flush debugfs file");
+ return 1;
+ }
+
+ if (write_debugfs_file("powerpc/entry_flush", entry_flush_orig) < 0) {
+ perror("unable to restore original value of powerpc/entry_flush debugfs file");
+ return 1;
+ }
+
+ return rc;
+}
+
+int main(int argc, char *argv[])
+{
+ return test_harness(entry_flush_test, "entry_flush_test");
+}
--- /dev/null
+// SPDX-License-Identifier: GPL-2.0+
+
+/*
+ * Copyright 2018 IBM Corporation.
+ */
+
+#define __SANE_USERSPACE_TYPES__
+
+#include <sys/types.h>
+#include <stdint.h>
+#include <unistd.h>
+#include <signal.h>
+#include <stdlib.h>
+#include <string.h>
+#include <stdio.h>
+#include "utils.h"
+#include "flush_utils.h"
+
+static inline __u64 load(void *addr)
+{
+ __u64 tmp;
+
+ asm volatile("ld %0,0(%1)" : "=r"(tmp) : "b"(addr));
+
+ return tmp;
+}
+
+void syscall_loop(char *p, unsigned long iterations,
+ unsigned long zero_size)
+{
+ for (unsigned long i = 0; i < iterations; i++) {
+ for (unsigned long j = 0; j < zero_size; j += CACHELINE_SIZE)
+ load(p + j);
+ getppid();
+ }
+}
+
+static void sigill_handler(int signr, siginfo_t *info, void *unused)
+{
+ static int warned;
+ ucontext_t *ctx = (ucontext_t *)unused;
+ unsigned long *pc = &UCONTEXT_NIA(ctx);
+
+ /* mtspr 3,RS to check for move to DSCR below */
+ if ((*((unsigned int *)*pc) & 0xfc1fffff) == 0x7c0303a6) {
+ if (!warned++)
+ printf("WARNING: Skipping over dscr setup. Consider running 'ppc64_cpu --dscr=1' manually.\n");
+ *pc += 4;
+ } else {
+ printf("SIGILL at %p\n", pc);
+ abort();
+ }
+}
+
+void set_dscr(unsigned long val)
+{
+ static int init;
+ struct sigaction sa;
+
+ if (!init) {
+ memset(&sa, 0, sizeof(sa));
+ sa.sa_sigaction = sigill_handler;
+ sa.sa_flags = SA_SIGINFO;
+ if (sigaction(SIGILL, &sa, NULL))
+ perror("sigill_handler");
+ init = 1;
+ }
+
+ asm volatile("mtspr %1,%0" : : "r" (val), "i" (SPRN_DSCR));
+}
--- /dev/null
+/* SPDX-License-Identifier: GPL-2.0+ */
+
+/*
+ * Copyright 2018 IBM Corporation.
+ */
+
+#ifndef _SELFTESTS_POWERPC_SECURITY_FLUSH_UTILS_H
+#define _SELFTESTS_POWERPC_SECURITY_FLUSH_UTILS_H
+
+#define CACHELINE_SIZE 128
+
+void syscall_loop(char *p, unsigned long iterations,
+ unsigned long zero_size);
+
+void set_dscr(unsigned long val);
+
+#endif /* _SELFTESTS_POWERPC_SECURITY_FLUSH_UTILS_H */
#include <stdint.h>
#include <malloc.h>
#include <unistd.h>
-#include <signal.h>
#include <stdlib.h>
#include <string.h>
#include <stdio.h>
#include "utils.h"
+#include "flush_utils.h"
-#define CACHELINE_SIZE 128
-
-struct perf_event_read {
- __u64 nr;
- __u64 l1d_misses;
-};
-
-static inline __u64 load(void *addr)
-{
- __u64 tmp;
-
- asm volatile("ld %0,0(%1)" : "=r"(tmp) : "b"(addr));
-
- return tmp;
-}
-
-static void syscall_loop(char *p, unsigned long iterations,
- unsigned long zero_size)
-{
- for (unsigned long i = 0; i < iterations; i++) {
- for (unsigned long j = 0; j < zero_size; j += CACHELINE_SIZE)
- load(p + j);
- getppid();
- }
-}
-
-static void sigill_handler(int signr, siginfo_t *info, void *unused)
-{
- static int warned = 0;
- ucontext_t *ctx = (ucontext_t *)unused;
- unsigned long *pc = &UCONTEXT_NIA(ctx);
-
- /* mtspr 3,RS to check for move to DSCR below */
- if ((*((unsigned int *)*pc) & 0xfc1fffff) == 0x7c0303a6) {
- if (!warned++)
- printf("WARNING: Skipping over dscr setup. Consider running 'ppc64_cpu --dscr=1' manually.\n");
- *pc += 4;
- } else {
- printf("SIGILL at %p\n", pc);
- abort();
- }
-}
-
-static void set_dscr(unsigned long val)
-{
- static int init = 0;
- struct sigaction sa;
-
- if (!init) {
- memset(&sa, 0, sizeof(sa));
- sa.sa_sigaction = sigill_handler;
- sa.sa_flags = SA_SIGINFO;
- if (sigaction(SIGILL, &sa, NULL))
- perror("sigill_handler");
- init = 1;
- }
-
- asm volatile("mtspr %1,%0" : : "r" (val), "i" (SPRN_DSCR));
-}
int rfi_flush_test(void)
{
__u64 l1d_misses_total = 0;
unsigned long iterations = 100000, zero_size = 24 * 1024;
unsigned long l1d_misses_expected;
- int rfi_flush_org, rfi_flush;
+ int rfi_flush_orig, rfi_flush;
+ int have_entry_flush, entry_flush_orig;
SKIP_IF(geteuid() != 0);
// The PMU event we use only works on Power7 or later
SKIP_IF(!have_hwcap(PPC_FEATURE_ARCH_2_06));
- if (read_debugfs_file("powerpc/rfi_flush", &rfi_flush_org)) {
+ if (read_debugfs_file("powerpc/rfi_flush", &rfi_flush_orig) < 0) {
perror("Unable to read powerpc/rfi_flush debugfs file");
SKIP_IF(1);
}
- rfi_flush = rfi_flush_org;
+ if (read_debugfs_file("powerpc/entry_flush", &entry_flush_orig) < 0) {
+ have_entry_flush = 0;
+ } else {
+ have_entry_flush = 1;
+
+ if (entry_flush_orig != 0) {
+ if (write_debugfs_file("powerpc/entry_flush", 0) < 0) {
+ perror("error writing to powerpc/entry_flush debugfs file");
+ return 1;
+ }
+ }
+ }
+
+ rfi_flush = rfi_flush_orig;
fd = perf_event_open_counter(PERF_TYPE_RAW, /* L1d miss */ 0x400f0, -1);
FAIL_IF(fd < 0);
FAIL_IF(perf_event_enable(fd));
+ // disable L1 prefetching
set_dscr(1);
iter = repetitions;
repetitions * l1d_misses_expected / 2,
passes, repetitions);
- if (rfi_flush == rfi_flush_org) {
- rfi_flush = !rfi_flush_org;
+ if (rfi_flush == rfi_flush_orig) {
+ rfi_flush = !rfi_flush_orig;
if (write_debugfs_file("powerpc/rfi_flush", rfi_flush) < 0) {
perror("error writing to powerpc/rfi_flush debugfs file");
return 1;
set_dscr(0);
- if (write_debugfs_file("powerpc/rfi_flush", rfi_flush_org) < 0) {
+ if (write_debugfs_file("powerpc/rfi_flush", rfi_flush_orig) < 0) {
perror("unable to restore original value of powerpc/rfi_flush debugfs file");
return 1;
}
+ if (have_entry_flush) {
+ if (write_debugfs_file("powerpc/entry_flush", entry_flush_orig) < 0) {
+ perror("unable to restore original value of powerpc/entry_flush "
+ "debugfs file");
+ return 1;
+ }
+ }
+
return rc;
}
* and the code is stored as a positive value. \
*/ \
if (_result < 0) { \
- SYSCALL_RET(_regs) = -result; \
+ SYSCALL_RET(_regs) = -_result; \
(_regs).ccr |= 0x10000000; \
} else { \
- SYSCALL_RET(_regs) = result; \
+ SYSCALL_RET(_regs) = _result; \
(_regs).ccr &= ~0x10000000; \
} \
} while (0)
#define SYSCALL_RET(_regs) (_regs).a[(_regs).windowbase * 4 + 2]
#elif defined(__sh__)
# define ARCH_REGS struct pt_regs
-# define SYSCALL_NUM(_regs) (_regs).gpr[3]
-# define SYSCALL_RET(_regs) (_regs).gpr[0]
+# define SYSCALL_NUM(_regs) (_regs).regs[3]
+# define SYSCALL_RET(_regs) (_regs).regs[0]
#else
# error "Do not know how to find your architecture's registers and syscalls"
#endif
CONFIG_NET_IFE_SKBTCINDEX=m
CONFIG_NET_SCH_FIFO=y
CONFIG_NET_SCH_ETS=m
+CONFIG_NET_SCH_RED=m
#
## Network testing
projects / linux-2.6-microblaze.git / commitdiff