Some Zhaoxin xHCI controllers follow usb3.1 spec,
but only support gen1 speed 5G. While in Linux kernel,
if xHCI suspport usb3.1,root hub speed will show on 10G.
To fix this issue, read usb speed ID supported by xHCI
to determine root hub speed.
The patch is scheduled to be submitted to the kernel mainline in 2021.
v1->v2:
- Use quirks instead of vendor id.
Signed-off-by: LeoLiu-oc <LeoLiu-oc(a)zhaoxin.com>
---
drivers/usb/host/xhci.c | 17 +++++++++++++++++
1 file changed, 17 insertions(+)
diff --git a/drivers/usb/host/xhci.c b/drivers/usb/host/xhci.c
index 326dd640a5ff..4cfad3b8b574 100644
--- a/drivers/usb/host/xhci.c
+++ b/drivers/usb/host/xhci.c
@@ -5065,6 +5065,7 @@ int xhci_gen_setup(struct usb_hcd *hcd,
xhci_get_quirks_t get_quirks)
*/
struct device *dev = hcd->self.sysdev;
unsigned int minor_rev;
+ u8 i;
int retval;
/* Accept arbitrarily long scatter-gather lists */
@@ -5119,6 +5120,22 @@ int xhci_gen_setup(struct usb_hcd *hcd,
xhci_get_quirks_t get_quirks)
hcd->self.root_hub->speed = USB_SPEED_SUPER_PLUS;
break;
}
+
+ /* usb3.1 has gen1 and gen2, Some zx's xHCI controller that follow
usb3.1 spec
+ * but only support gen1
+ */
+ if (xhci->quirks == XHCI_ZHAOXIN_HOST) {
+ minor_rev = 0;
+ for (i = 0; i < xhci->usb3_rhub.psi_count; i++) {
+ if (XHCI_EXT_PORT_PSIV(xhci->usb3_rhub.psi[i]) >= 5)
+ minor_rev = 1;
+ }
+ if (minor_rev != 1) {
+ hcd->speed = HCD_USB3;
+ hcd->self.root_hub->speed = USB_SPEED_SUPER;
+ }
+ }
+
xhci_info(xhci, "Host supports USB 3.%x %sSuperSpeed\n",
minor_rev,
minor_rev ? "Enhanced " : "");
--
2.20.1
The driver crc32c-intel match CPUs supporting X86_FEATURE_XMM4_2.
On platforms with Zhaoxin CPUs supporting this X86 feature, when
crc32c-intel and crc32c-generic are both registered, system will use
crc32c-intel because its .cra_priority is greater than crc32c-generic.
When doing lmbench3 Create and Delete file test on partitions with
ext4 enabling metadata checksum, found using crc32c-generic driver could
get about 20% performance gain than using the driver crc32c-intel on
some Zhaoxin CPUs. Lower-level testing result is that with the same
input value the generic C implementation takes fewer time than the
crc32c instruction implementation on these CPUs. This case expect to use
crc32c-generic driver for these CPUs to get performance gain.
The presence of crc32c is enumerated by CPUID.01:ECX[SSE4.2] = 1, and
these CPUs other SSE4.2 instructions is ok.
Add a synthetic flag to indicates low performance CRC32C instruction
implementation, set this flag in Zhaoxin CPUs specific init phase, and
exclude CPUs which setting this flag from the driver crc32c-intel.
LeoLiu-oc (3):
ck: x86/cpufeatures: Add low performance CRC32C instruction CPU
feature
ck: x86/cpu: Set low performance CRC32C flag on some Zhaoxin CPUs
ck: crypto: x86/crc32c-intel Exclude low performance CRC32C
instruction CPUs
arch/x86/crypto/crc32c-intel_glue.c | 5 +++++
arch/x86/include/asm/cpufeatures.h | 1 +
arch/x86/kernel/cpu/centaur.c | 7 +++++++
arch/x86/kernel/cpu/cpuid-deps.c | 1 +
arch/x86/kernel/cpu/zhaoxin.c | 7 +++++++
5 files changed, 21 insertions(+)
--
2.20.1
This bug is found in Zhaoxin platform, but it's a commom code bug.
Fail sequence:
step1: Unbind UHCI controller from native driver;
step2: Bind UHCI controller to vfio-pci, which will put UHCI controller
in one vfio
group's device list and set UHCI's dev->driver_data to struct
vfio-pci(for UHCI)
step3: Unbind EHCI controller from native driver, will try to tell UHCI
native driver
that "I'm removed by set companion_hcd->self.hs_companion to
NULL. However,
companion_hcd get from UHCI's dev->driver_data that has modified
by vfio-pci
already.So, the vfio-pci structure will be damaged!
step4: Bind EHCI controller to vfio-pci driver, which will put EHCI
controller in the
same vfio group as UHCI controller;
... ...
step5: Unbind UHCI controller from vfio-pci, which will delete UHCI from
vfio group'
device list that has been damaged in step 3. So,delete operation
can random
result into a NULL pointer dereference with the below stack dump.
step6: Bind UHCI controller to native driver;
step7: Unbind EHCI controller from vfio-pci, which will try to remove
EHCI controller
from the vfio group;
step8: Bind EHCI controller to native driver;
[ 929.114641] uhci_hcd 0000:00:10.0: remove, state 1 [ 929.114652] usb
usb1: USB disconnect, device number 1 [ 929.114655] usb 1-1: USB
disconnect, device number 2 [ 929.270313] usb 1-2: USB disconnect,
device number 3 [ 929.318404] uhci_hcd 0000:00:10.0: USB bus 1
deregistered [ 929.343029] uhci_hcd 0000:00:10.1: remove, state 4 [
929.343045] usb usb3: USB disconnect, device number 1 [ 929.343685]
uhci_hcd 0000:00:10.1: USB bus 3 deregistered [ 929.369087] ehci-pci
0000:00:10.7: remove, state 4 [ 929.369102] usb usb4: USB disconnect,
device number 1 [ 929.370325] ehci-pci 0000:00:10.7: USB bus 4
deregistered [ 932.398494] BUG: unable to handle kernel NULL pointer
dereference at 0000000000000000 [ 932.398496] PGD 42a67d067 P4D
42a67d067 PUD 42a65f067 PMD 0 [ 932.398502] Oops: 0002 [#2] SMP NOPTI
[ 932.398505] CPU: 2 PID: 7824 Comm: vfio_unbind.sh Tainted: P D
4.19.65-2020051917-rainos #1
[ 932.398506] Hardware name: Shanghai Zhaoxin Semiconductor Co., Ltd.
HX002EH/HX002EH,
BIOS HX002EH0_01_R480_R_200408 04/08/2020 [ 932.398513] RIP:
0010:vfio_device_put+0x31/0xa0 [vfio] [ 932.398515] Code: 89 e5 41 54
53 4c 8b 67 18 48 89 fb 49 8d 74 24 30 e8 e3 0e f3 de
84 c0 74 67 48 8b 53 20 48 8b 43 28 48 8b 7b 18 48 89 42 08 <48> 89 10
48 b8 00 01 00 00 00 00 ad de 48 89 43 20 48 b8 00 02 00 [ 932.398516]
RSP: 0018:ffffbbfd04cffc18 EFLAGS: 00010202 [ 932.398518] RAX:
0000000000000000 RBX: ffff92c7ea717880 RCX: 0000000000000000 [
932.398519] RDX: ffff92c7ea713620 RSI: ffff92c7ea713630 RDI:
ffff92c7ea713600 [ 932.398521] RBP: ffffbbfd04cffc28 R08:
ffff92c7f02a8080 R09: ffff92c7efc03980 [ 932.398522] R10:
ffffbbfd04cff9a8 R11: 0000000000000000 R12: ffff92c7ea713600 [
932.398523] R13: ffff92c7ed8bb0a8 R14: ffff92c7ea717880 R15:
0000000000000000 [ 932.398525] FS: 00007f3031500740(0000)
GS:ffff92c7f0280000(0000) knlGS:0000000000000000 [ 932.398526] CS: 0010
DS: 0000 ES: 0000 CR0: 0000000080050033 [ 932.398527] CR2:
0000000000000000 CR3: 0000000428626004 CR4: 0000000000160ee0 [
932.398528] Call Trace:
[ 932.398534] vfio_del_group_dev+0xe8/0x2a0 [vfio] [ 932.398539] ?
__blocking_notifier_call_chain+0x52/0x60
[ 932.398542] ? do_wait_intr_irq+0x90/0x90 [ 932.398546] ?
iommu_bus_notifier+0x75/0x100 [ 932.398551] vfio_pci_remove+0x20/0xa0
[vfio_pci] [ 932.398554] pci_device_remove+0x3e/0xc0 [ 932.398557]
device_release_driver_internal+0x17a/0x240
[ 932.398560] device_release_driver+0x12/0x20 [ 932.398561]
unbind_store+0xee/0x180 [ 932.398564] drv_attr_store+0x27/0x40 [
932.398567] sysfs_kf_write+0x3c/0x50 [ 932.398568]
kernfs_fop_write+0x125/0x1a0 [ 932.398572] __vfs_write+0x3a/0x190 [
932.398575] ? apparmor_file_permission+0x1a/0x20
[ 932.398577] ? security_file_permission+0x3b/0xc0
[ 932.398581] ? _cond_resched+0x1a/0x50 [ 932.398582]
vfs_write+0xb8/0x1b0 [ 932.398584] ksys_write+0x5c/0xe0 [ 932.398586]
__x64_sys_write+0x1a/0x20 [ 932.398589] do_syscall_64+0x5a/0x110 [
932.398592] entry_SYSCALL_64_after_hwframe+0x44/0xa9
Using virt-manager/qemu to boot guest os, we can see the same fail sequence!
Fix this by check for UHCI driver loaded or not before modifiy UHCI's
dev->driver_data, which will happen in ehci native driver probe/remove.
This patch was submitted to mainline kernel but not accepted by upstream
maintainer whose reason is "Given that it's currently needed in only one
place, it seems reasonable to leave this as a "gentlemen's agreement" in
userspace for the time being instead of adding it to the kernel."
We think the kernel driver should fix this bug regardless of userspaces
behavior.
https://lkml.org/lkml/2020/7/22/493
Signed-off-by: LeoLiu-oc <LeoLiu-oc(a)zhaoxin.com>
---
drivers/usb/core/hcd-pci.c | 7 +++++++
1 file changed, 7 insertions(+)
diff --git a/drivers/usb/core/hcd-pci.c b/drivers/usb/core/hcd-pci.c
index 7537681355f6..57ac942acf12 100644
--- a/drivers/usb/core/hcd-pci.c
+++ b/drivers/usb/core/hcd-pci.c
@@ -34,6 +34,8 @@ static DECLARE_RWSEM(companions_rwsem);
#define CL_OHCI PCI_CLASS_SERIAL_USB_OHCI
#define CL_EHCI PCI_CLASS_SERIAL_USB_EHCI
+#define PCI_DEV_DRV_FLAG 2
+
static inline int is_ohci_or_uhci(struct pci_dev *pdev)
{
return pdev->class == CL_OHCI || pdev->class == CL_UHCI;
@@ -69,6 +71,9 @@ static void for_each_companion(struct pci_dev *pdev,
struct usb_hcd *hcd,
companion->class != CL_EHCI)
continue;
+ if (!(companion->priv_flags & PCI_DEV_DRV_FLAG))
+ continue;
+
companion_hcd = pci_get_drvdata(companion);
if (!companion_hcd || !companion_hcd->self.root_hub)
continue;
@@ -253,6 +258,7 @@ int usb_hcd_pci_probe(struct pci_dev *dev, const
struct pci_device_id *id)
}
pci_set_master(dev);
+ dev->priv_flags |= PCI_DEV_DRV_FLAG;
/* Note: dev_set_drvdata must be called while holding the rwsem */
if (dev->class == CL_EHCI) {
@@ -325,6 +331,7 @@ void usb_hcd_pci_remove(struct pci_dev *dev)
local_irq_disable();
usb_hcd_irq(0, hcd);
local_irq_enable();
+ dev->priv_flags &= ~PCI_DEV_DRV_FLAG;
/* Note: dev_set_drvdata must be called while holding the rwsem */
if (dev->class == CL_EHCI) {
--
2.20.1
On 25/03/2021 17:56, Artie Ding wrote:
>
> 在 2021/3/25 下午5:27, Tony W Wang-oc 写道:
>> On 25/03/2021 17:15, 丁宁(富邻) wrote:
>>> I have a question here. If crc32c instruction on some Zhaoxin CPUs
>>> slower than generic C
>>> implementation, then why not clear the X86_FEATURE_XMM4_2 flag?
>>>
>> The presence of crc32c and some other instructions supports are
>> enumerated by CPUID.01:ECX[SSE4.2] = 1 that called X86_FEATURE_XMM4_2 in
>> kernel, other instructions are ok except the crc32c instruction on these
>> Zhaoxin CPUs.
>>
>> Sincerely
>> TonyWWangoc
>
> Got it. It seems there is an issue for crc32c instruction. It's not nice
> to change intel driver for Zhaoxin CPUs issue.
>
> It's better to make it clear with a individual flag for crc32c instruction.
>
As crc32c-intel is the common driver that match X86_FEATURE_XMM4_2, add
specific code for other CPUs should be reasonable.
Sincerely
TonyWWangoc
> BR
>
> Artie
>
>>
>>> Best Regards
>>> Artie Ding(丁宁)
>>> ------------------原始邮件 ------------------
>>> 发件人:Tony W Wang-oc <TonyWWang-oc(a)zhaoxin.com>
>>> 发送时间:Thu Mar 25 16:37:49 2021
>>> 收件人:丁宁(富邻) <fulin.dn(a)alibaba-inc.com>, LeoLiu-oc
>>> <LeoLiu-oc(a)zhaoxin.com>, cloud-kernel
>>> <cloud-kernel(a)lists.openanolis.org>
>>> 抄送:TimGuo <TimGuo(a)zhaoxin.com>, CobeChen <CobeChen(a)zhaoxin.com>
>>> 主题:Re: 回复:[ck] [patch v1] ck: crypto: x86/crc32c-intel - Don't
>>> match some Zhaoxin CPUs
>>> On 25/03/2021 15:53, 丁宁(富邻) wrote:
>>>> If X86_FEATURE_XMM4_2 meens different hw implement to Zhaoxin CPUs,
>>>> a new flag is needed to distinguish
>>>> them.
>>>>
>>> The X86_FEATURE_XMM4_2 on Zhaoxin CPUs have the same meaning with Intel.
>>> crc32c instruction on some Zhaoxin CPUs slower than generic C
>>> implementation, should avoid these Zhaoxin CPUs using the driver
>>> crc32c-intel which will call crc32c instruction.
>>>
>>> Sincerely
>>> TonyWWangoc
>>>
>>>> Artie
>>>>
>>>>
>>>> ------------------原始邮件 ------------------
>>>> 发件人:LeoLiu-oc <LeoLiu-oc(a)zhaoxin.com>
>>>> 发送时间:Mon Mar 22 18:44:54 2021
>>>> 收件人: <cloud-kernel(a)lists.openanolis.org>
>>>> 抄送: <TonyWWang-oc(a)zhaoxin.com>, <TimGuo(a)zhaoxin.com>,
>>>> <CobeChen(a)zhaoxin.com>
>>>> 主题:[ck] [patch v1] ck: crypto: x86/crc32c-intel - Don't match some
>>>> Zhaoxin CPUs
>>>>
>>>> The driver crc32c-intel match CPUs supporting X86_FEATURE_XMM4_2.
>>>> On platforms with Zhaoxin CPUs supporting this X86 feature,
>>>> when crc32c-intel and crc32c-generic are both registered,
>>>> system will use crc32c-intel because its .cra_priority is greater
>>>> than crc32c-generic.
>>>>
>>>> When doing lmbench3 Create and Delete file test on partitions with
>>>> ext4 enabling metadata checksum, found using crc32c-generic driver
>>>> could get about 20% performance gain than using the driver
>>>> crc32c-intel on some Zhaoxin CPUs.
>>>>
>>>> This case expect to use crc32c-generic driver for these Zhaoxin CPUs
>>>> to get performance gain, so remove these Zhaoxin CPUs support from
>>>> crc32c-intel.
>>>>
>>>> This patch was submitted to mainline kernel but not accepted by
>>>> upstream
>>>> maintainer whose reason is "Then create a BUG flag for it,".
>>>>
>>>> We think this is not a CPU bug for Zhaoxin CPUs. So should patch the
>>>> crc32c driver for Zhaoxin CPUs but not report a BUG.
>>>>
>>>> https://lkml.org/lkml/2020/12/11/308
>>>>
>>>> Signed-off-by: LeoLiu-oc <LeoLiu-oc(a)zhaoxin.com>
>>>> ---
>>>> arch/x86/crypto/crc32c-intel_glue.c | 5 +++++
>>>> 1 file changed, 5 insertions(+)
>>>>
>>>> diff --git a/arch/x86/crypto/crc32c-intel_glue.c
>>>> b/arch/x86/crypto/crc32c-intel_glue.c
>>>> index 5773e1161072..d994bd49761a 100644
>>>> --- a/arch/x86/crypto/crc32c-intel_glue.c
>>>> +++ b/arch/x86/crypto/crc32c-intel_glue.c
>>>> @@ -242,8 +242,13 @@ MODULE_DEVICE_TABLE(x86cpu, crc32c_cpu_id);
>>>>
>>>> static int __init crc32c_intel_mod_init(void)
>>>> {
>>>> +struct cpuinfo_x86 *c = &boot_cpu_data;
>>>> if (!x86_match_cpu(crc32c_cpu_id))
>>>> return -ENODEV;
>>>> +if ((c->x86_vendor == X86_VENDOR_ZHAOXIN || c->x86_vendor ==
>>>> X86_VENDOR_CENTAUR) &&
>>>> +(c->x86 <= 7 && c->x86_model <= 59)) {
>>>> +return -ENODEV;
>>>> +}
>>>> #ifdef CONFIG_X86_64
>>>> if (boot_cpu_has(X86_FEATURE_PCLMULQDQ)) {
>>>> alg.update = crc32c_pcl_intel_update;
> .