(The USB forwarding directive is just for forwarding a MIDI keyboard, I can try without it if needed (Edit: same thing without it))
The QEMU window spawns, and I can the select the macOS disk as a boot device, but after 1-2 minutes I'm getting a kernel panic about something happening in the com.apple.kec.corecrypto(11.1) module of macOS:
And before that I have this screen (outlining an "Err(0xE) <- EB.WL.PWLFNV - RT.GV failure error)?
I'm using a standard Lenovo Thinkpad T460s (2016) on Ubuntu 20.04 LTS with extended RAM and SSD, and with the original Intel CPU and iGPU.
Would you have any idea about the possible cause of the issue?
Regards,
Debug logs
$ uname -a \
> ; echo "${DISPLAY}" \
> ; echo 1 | sudo tee /sys/module/kvm/parameters/ignore_msrs \
> ; grep NAME /etc/os-release \
> ; df -h . \
> ; qemu-system-x86_64 --version \
> ; libvirtd --version \
> ; free -mh \
> ; nproc \
> ; egrep -c '(svm|vmx)' /proc/cpuinfo \
> ; ls -lha /dev/kvm \
> ; ls -lha /tmp/.X11-unix/ \
> ; ps aux | grep dockerd \
> ; docker ps | grep osx \
> ; grep "docker\|kvm\|virt" /etc/group
Linux myusername-ThinkPad-T460s 5.8.0-59-generic #66~20.04.1-Ubuntu SMP Thu Jun 17 11:14:10 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux
:0
[sudo] Mot de passe de myusername :
1
NAME="Ubuntu"
PRETTY_NAME="Ubuntu 20.04.2 LTS"
VERSION_CODENAME=focal
UBUNTU_CODENAME=focal
Sys. de fichiers Taille Utilisé Dispo Uti% Monté sur
/dev/mapper/vgubuntu-root 914G 352G 516G 41% /
QEMU emulator version 4.2.1 (Debian 1:4.2-3ubuntu6.16)
Copyright (c) 2003-2019 Fabrice Bellard and the QEMU Project developers
libvirtd (libvirt) 6.0.0
total utilisé libre partagé tamp/cache disponible
Mem: 19Gi 11Gi 294Mi 2,3Gi 7,8Gi 5,5Gi
Partition d'échange: 975Mi 956Mi 19Mi
4
8
crw-rw----+ 1 root kvm 10, 232 juin 29 23:59 /dev/kvm
total 52K
drwxrwxrwt 2 root root 4,0K juin 29 23:59 .
drwxrwxrwt 34 root root 44K juil. 3 10:09 ..
srwxrwxrwx 1 myusername myusername 0 juin 29 23:59 X0
root 115319 0.3 0.3 1452676 61836 ? Ssl juil.02 6:19 /usr/bin/dockerd -H fd:// --containerd=/run/containerd/containerd.sock
myusername 409993 0.0 0.0 11608 2644 pts/65 S+ 10:19 0:00 grep --color=auto dockerd
560c1f0800fd sickcodes/docker-osx:big-sur "/bin/bash -c 'sudo …" 18 minutes ago Up 18 minutes 0.0.0.0:50922->10022/tcp nostalgic_hypatia
kvm:x:108:myusername
docker:x:142:myusername
libvirt:x:146:myusername
libvirt-qemu:x:64055:libvirt-qemu
libvirt-dnsmasq:x:147:
Originally created by @3xr on GitHub (Jul 3, 2021).
Hello,
When trying to run the Big Sur image using the following command, after having installed at the requirements and dependencies:
```bash
docker run -it \
--device /dev/kvm \
-p 50922:10022 \
-e DEVICE_MODEL="iMacPro1,1" \
-e WIDTH=1300 \
-e HEIGHT=700 \
-v /tmp/.X11-unix:/tmp/.X11-unix \
-e "DISPLAY=${DISPLAY:-:0.0}" \
-e GENERATE_UNIQUE=true \
-e EXTRA="-device virtio-serial-pci -device usb-host,hostbus=1,hostport=4" \
-e MASTER_PLIST_URL=https://raw.githubusercontent.com/sickcodes/osx-serial-generator/master/config-custom.plist \
sickcodes/docker-osx:big-sur
```
(The USB forwarding directive is just for forwarding a MIDI keyboard, I can try without it if needed (Edit: same thing without it))
The QEMU window spawns, and I can the select the macOS disk as a boot device, but after 1-2 minutes I'm getting a kernel panic about something happening in the `com.apple.kec.corecrypto(11.1)` module of macOS:

And before that I have this screen (outlining an "`Err(0xE) <- EB.WL.PWLFNV - RT.GV failure` error)?

I'm using a standard Lenovo Thinkpad T460s (2016) on Ubuntu 20.04 LTS with extended RAM and SSD, and with the original Intel CPU and iGPU.
Would you have any idea about the possible cause of the issue?
Regards,
## Debug logs
```
$ uname -a \
> ; echo "${DISPLAY}" \
> ; echo 1 | sudo tee /sys/module/kvm/parameters/ignore_msrs \
> ; grep NAME /etc/os-release \
> ; df -h . \
> ; qemu-system-x86_64 --version \
> ; libvirtd --version \
> ; free -mh \
> ; nproc \
> ; egrep -c '(svm|vmx)' /proc/cpuinfo \
> ; ls -lha /dev/kvm \
> ; ls -lha /tmp/.X11-unix/ \
> ; ps aux | grep dockerd \
> ; docker ps | grep osx \
> ; grep "docker\|kvm\|virt" /etc/group
Linux myusername-ThinkPad-T460s 5.8.0-59-generic #66~20.04.1-Ubuntu SMP Thu Jun 17 11:14:10 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux
:0
[sudo] Mot de passe de myusername :
1
NAME="Ubuntu"
PRETTY_NAME="Ubuntu 20.04.2 LTS"
VERSION_CODENAME=focal
UBUNTU_CODENAME=focal
Sys. de fichiers Taille Utilisé Dispo Uti% Monté sur
/dev/mapper/vgubuntu-root 914G 352G 516G 41% /
QEMU emulator version 4.2.1 (Debian 1:4.2-3ubuntu6.16)
Copyright (c) 2003-2019 Fabrice Bellard and the QEMU Project developers
libvirtd (libvirt) 6.0.0
total utilisé libre partagé tamp/cache disponible
Mem: 19Gi 11Gi 294Mi 2,3Gi 7,8Gi 5,5Gi
Partition d'échange: 975Mi 956Mi 19Mi
4
8
crw-rw----+ 1 root kvm 10, 232 juin 29 23:59 /dev/kvm
total 52K
drwxrwxrwt 2 root root 4,0K juin 29 23:59 .
drwxrwxrwt 34 root root 44K juil. 3 10:09 ..
srwxrwxrwx 1 myusername myusername 0 juin 29 23:59 X0
root 115319 0.3 0.3 1452676 61836 ? Ssl juil.02 6:19 /usr/bin/dockerd -H fd:// --containerd=/run/containerd/containerd.sock
myusername 409993 0.0 0.0 11608 2644 pts/65 S+ 10:19 0:00 grep --color=auto dockerd
560c1f0800fd sickcodes/docker-osx:big-sur "/bin/bash -c 'sudo …" 18 minutes ago Up 18 minutes 0.0.0.0:50922->10022/tcp nostalgic_hypatia
kvm:x:108:myusername
docker:x:142:myusername
libvirt:x:146:myusername
libvirt-qemu:x:64055:libvirt-qemu
libvirt-dnsmasq:x:147:
```
adam
added the bug label 2025-12-29 05:24:47 +01:00
@3xr commented on GitHub (Jul 3, 2021):
I encounter the same issue with the Catalina image, while we can see a larger backtrace thanks to the higher screen resolution:

I deleted the former Docker images and ran the following command this time:
```bash
date ; time docker run -i \
--device /dev/kvm \
-p 50922:10022 \
-e DEVICE_MODEL="iMacPro1,1" \
-e WIDTH=1300 \
-e HEIGHT=700 \
-v /tmp/.X11-unix:/tmp/.X11-unix \
-e "DISPLAY=${DISPLAY:-:0.0}" \
-e EXTRA="-device virtio-serial-pci -device usb-host,hostbus=1,hostport=4" \
sickcodes/docker-osx:latest
```
As I precised above, removing the USB forwarding line changes nothing at all.
As you can see in the backtrace, this "Unknown Opcode" CPU fault seems to lay in the _gcm_init routine of Apple's CoreCrypto library, which seems to depend both on the SSSE3 and AES-NI instruction extension sets.
My host CPU (Intel Core i7-6600U) correctly supports both (it doesn't support AVX-512 which doesn't seem to be known to be involved here).
I think that it would be good to disassemble the Catalina kernel at the address provided by the backtrace in order to perform helpful investigation, but I don't know how to extract it. Maybe that the CR registers give a hint with this kind of exception?
An important thing to note is that while both the Big Sur and Catalina images provided by Docker-OSX fail with this error at boot, sosumi which also relies on QEMU (but with a different stack behind) boots Catalina well on my machine.
Your CPU configuration lines doesn't seem to contain something extravagant to me, although I'm not a QEMU expert. Maybe that these should be compared with more attention or detail with other macOS virtualization scripts.
Regards,
@3xr commented on GitHub (Jul 4, 2021):
Hello,
As I precised above, removing the USB forwarding line changes nothing at all.
As you can see in the backtrace, this "Unknown Opcode" CPU fault seems to lay in the [`_gcm_init`](https://github.com/vtky/corecrypto/blob/53a391ef0c6951c3b6cbe8283da5eff5b5e9970e/ccaes/src/intel/ghash-x86_64.s#L116) routine of Apple's CoreCrypto library, which seems to depend both on the SSSE3 and AES-NI instruction extension sets.
My host CPU (Intel Core i7-6600U) correctly supports both (it doesn't support AVX-512 which doesn't seem to be known to be involved here).
`$ grep ssse3 /proc/cpuinfo | head -1`
`flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc art arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc cpuid aperfmperf pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 sdbg fma cx16 xtpr pdcm pcid sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm abm 3dnowprefetch cpuid_fault epb invpcid_single pti ssbd ibrs ibpb stibp tpr_shadow vnmi flexpriority ept vpid ept_ad fsgsbase tsc_adjust bmi1 hle avx2 smep bmi2 erms invpcid rtm mpx rdseed adx smap clflushopt intel_pt xsaveopt xsavec xgetbv1 xsaves dtherm ida arat pln pts hwp hwp_notify hwp_act_window hwp_epp md_clear flush_l1d`
I think that it would be good to disassemble the Catalina kernel at the address provided by the backtrace in order to perform helpful investigation, but I don't know how to extract it. Maybe that the CR registers give a hint with this kind of exception?
An important thing to note is that while both the Big Sur and Catalina images provided by Docker-OSX fail with this error at boot, `sosumi` which also relies on QEMU (but with a different stack behind) boots Catalina well on my machine.
Your [CPU configuration lines](https://github.com/sickcodes/Docker-OSX/search?q=ssse3) doesn't seem to contain something extravagant to me, although I'm not a QEMU expert. Maybe that these should be compared with more attention or detail with other macOS virtualization scripts.
Regards,
Actually the code simply lays in its own kernel extension file (the macOS Base System/System/Library/Extensions/corecrypto.kext/Contents/MacOS/corecrypto file from the diff/home/arch/OSX-KVM/BaseSystem.img image from the container).
Actually the code at _gcm_init + 8e is not SSSE3 or AES-NI (SSSE3 instructions like pshufb or SSE2 instructions like movdqa or pslldq perform well according to the position in the control flow), it's a pclmulqdq from the CLMUL (2010) instruction set extension.
Brace yourself for IDA screenshot
An instruction set extension that my processor indicates to support, obviously
@3xr commented on GitHub (Jul 4, 2021):
Actually the code simply lays in its own kernel extension file (the `macOS Base System/System/Library/Extensions/corecrypto.kext/Contents/MacOS/corecrypto` file from the `diff/home/arch/OSX-KVM/BaseSystem.img` image from the container).
Actually the code at `_gcm_init + 8e` is not SSSE3 or AES-NI (SSSE3 instructions like `pshufb` or SSE2 instructions like `movdqa` or `pslldq` perform well according to the position in the control flow), it's a `pclmulqdq` from the [CLMUL (2010)](https://en.wikipedia.org/wiki/CLMUL_instruction_set) instruction set extension.
Brace yourself for IDA screenshot

An instruction set extension that my processor indicates to support, obviously
`$ grep pclmulqdq /proc/cpuinfo | head -1`
`flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc art arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc cpuid aperfmperf pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 sdbg fma cx16 xtpr pdcm pcid sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm abm 3dnowprefetch cpuid_fault epb invpcid_single pti ssbd ibrs ibpb stibp tpr_shadow vnmi flexpriority ept vpid ept_ad fsgsbase tsc_adjust bmi1 hle avx2 smep bmi2 erms invpcid rtm mpx rdseed adx smap clflushopt intel_pt xsaveopt xsavec xgetbv1 xsaves dtherm ida arat pln pts hwp hwp_notify hwp_act_window hwp_epp md_clear flush_l1d`
I wonder why the crash, then.
It looks like it may be linked to the +ssse3 flag in the Docker-OSX configuration, which is not present in Sosumi`'s. The CoreCrypto module does a CPUID flag check on this feature just before calling. ^^
I'd suggest to align on the -cpu flags of Sosumi and remove the +ssse3 (see the command above). (Or maybe that adding the +pclmulqdq flag as specified in qemu-system-x86_64 -cpu help would change something, if it does not only affect the returned cpuid.)
@3xr commented on GitHub (Jul 4, 2021):
It looks like it may be linked to the `+ssse3` flag in the Docker-OSX configuration, which is not present in Sosumi`'s. The CoreCrypto module does a CPUID flag check on this feature just before calling. ^^
See: https://opensource.apple.com/source/xnu/xnu-1504.9.17/osfmk/i386/cpuid.h.auto.html
```c
#define CPUID_FEATURE_SSSE3 _HBit(9) /* Supplemental SSE3 instructions */
```

I'd suggest to align on the `-cpu` flags of Sosumi and remove the `+ssse3` (see the command above). (Or maybe that adding the `+pclmulqdq` flag as specified in `qemu-system-x86_64 -cpu help` would change something, if it does not only affect the returned `cpuid`.)
Also as we speak, I'm working on https://github.com/sickcodes/dock-droid and lots of random qemu args that we currently have can be removed, namely due to QEMU 6, but also older stuff.
@sickcodes commented on GitHub (Jul 4, 2021):
RE: Sosumi boot args:

Also there are some changes made recently upstream in KVM-Opencore to prepare for Monterrey that I haven't merged yet as I haven't had time to test:
`https://raw.githubusercontent.com/sickcodes/osx-serial-generator/v13/config-custom.plist`
As seen in:
https://github.com/thenickdude/KVM-Opencore/releases/tag/v13
Changes ready to merge:
https://github.com/sickcodes/osx-serial-generator/pull/13
Also as we speak, I'm working on https://github.com/sickcodes/dock-droid and lots of random qemu args that we currently have can be removed, namely due to QEMU 6, but also older stuff.
For example, simply using:
```
-cpu host \
-smp 8 \
```
And also adding:
```
-display sdl,gl=on \
```
Testing right now
https://github.com/sickcodes/osx-serial-generator/pull/13
Regarding your last diff, is this a change that you done or a change that you are projecting to do? It seems that the mentions of CPU configuration in your repository didn't change yet.
Regards
@3xr commented on GitHub (Jul 8, 2021):
Hello,
Regarding your last diff, is this a change that you done or a change that you are projecting to do? It seems that the mentions of [CPU configuration](https://github.com/sickcodes/Docker-OSX/search?q=ssse3) in your repository didn't change yet.
Regards
@sickcodes commented on GitHub (Jul 9, 2021):
Added (just now) in https://github.com/sickcodes/Docker-OSX/commit/866a429848e8bca422a10712dae0711cbbdad99b
You can now use:
`-e CPUID_FLAGS='vendor=GenuineIntel,+invtsc,vmware-cpuid-freq=on,+pcid,+ssse3,+sse4.2,+popcnt,+avx,+aes,+xsave,+xsaveopt,check,' \`
`Available CPUs:
x86 486 (alias configured by machine type)
x86 486-v1
x86 Broadwell (alias configured by machine type)
x86 Broadwell-IBRS (alias of Broadwell-v3)
x86 Broadwell-noTSX (alias of Broadwell-v2)
x86 Broadwell-noTSX-IBRS (alias of Broadwell-v4)
x86 Broadwell-v1 Intel Core Processor (Broadwell)
x86 Broadwell-v2 Intel Core Processor (Broadwell, no TSX)
x86 Broadwell-v3 Intel Core Processor (Broadwell, IBRS)
x86 Broadwell-v4 Intel Core Processor (Broadwell, no TSX, IBRS)
x86 Cascadelake-Server (alias configured by machine type)
x86 Cascadelake-Server-noTSX (alias of Cascadelake-Server-v3)
x86 Cascadelake-Server-v1 Intel Xeon Processor (Cascadelake)
x86 Cascadelake-Server-v2 Intel Xeon Processor (Cascadelake)
x86 Cascadelake-Server-v3 Intel Xeon Processor (Cascadelake)
x86 Conroe (alias configured by machine type)
x86 Conroe-v1 Intel Celeron_4x0 (Conroe/Merom Class Core 2)
x86 Cooperlake (alias configured by machine type)
x86 Cooperlake-v1 Intel Xeon Processor (Cooperlake)
x86 Denverton (alias configured by machine type)
x86 Denverton-v1 Intel Atom Processor (Denverton)
x86 Dhyana (alias configured by machine type)
x86 Dhyana-v1 Hygon Dhyana Processor
x86 EPYC (alias configured by machine type)
x86 EPYC-IBPB (alias of EPYC-v2)
x86 EPYC-Milan (alias configured by machine type)
x86 EPYC-Milan-v1 AMD EPYC-Milan Processor
x86 EPYC-Rome (alias configured by machine type)
x86 EPYC-Rome-v1 AMD EPYC-Rome Processor
x86 EPYC-Rome-v2 AMD EPYC-Rome Processor
x86 EPYC-v1 AMD EPYC Processor
x86 EPYC-v2 AMD EPYC Processor (with IBPB)
x86 EPYC-v3 AMD EPYC Processor
x86 Haswell (alias configured by machine type)
x86 Haswell-IBRS (alias of Haswell-v3)
x86 Haswell-noTSX (alias of Haswell-v2)
x86 Haswell-noTSX-IBRS (alias of Haswell-v4)
x86 Haswell-v1 Intel Core Processor (Haswell)
x86 Haswell-v2 Intel Core Processor (Haswell, no TSX)
x86 Haswell-v3 Intel Core Processor (Haswell, IBRS)
x86 Haswell-v4 Intel Core Processor (Haswell, no TSX, IBRS)
x86 Icelake-Client (alias configured by machine type)
x86 Icelake-Client-noTSX (alias of Icelake-Client-v2)
x86 Icelake-Client-v1 Intel Core Processor (Icelake)
x86 Icelake-Client-v2 Intel Core Processor (Icelake)
x86 Icelake-Server (alias configured by machine type)
x86 Icelake-Server-noTSX (alias of Icelake-Server-v2)
x86 Icelake-Server-v1 Intel Xeon Processor (Icelake)
x86 Icelake-Server-v2 Intel Xeon Processor (Icelake)
x86 IvyBridge (alias configured by machine type)
x86 IvyBridge-IBRS (alias of IvyBridge-v2)
x86 IvyBridge-v1 Intel Xeon E3-12xx v2 (Ivy Bridge)
x86 IvyBridge-v2 Intel Xeon E3-12xx v2 (Ivy Bridge, IBRS)
x86 KnightsMill (alias configured by machine type)
x86 KnightsMill-v1 Intel Xeon Phi Processor (Knights Mill)
x86 Nehalem (alias configured by machine type)
x86 Nehalem-IBRS (alias of Nehalem-v2)
x86 Nehalem-v1 Intel Core i7 9xx (Nehalem Class Core i7)
x86 Nehalem-v2 Intel Core i7 9xx (Nehalem Core i7, IBRS update)
x86 Opteron_G1 (alias configured by machine type)
x86 Opteron_G1-v1 AMD Opteron 240 (Gen 1 Class Opteron)
x86 Opteron_G2 (alias configured by machine type)
x86 Opteron_G2-v1 AMD Opteron 22xx (Gen 2 Class Opteron)
x86 Opteron_G3 (alias configured by machine type)
x86 Opteron_G3-v1 AMD Opteron 23xx (Gen 3 Class Opteron)
x86 Opteron_G4 (alias configured by machine type)
x86 Opteron_G4-v1 AMD Opteron 62xx class CPU
x86 Opteron_G5 (alias configured by machine type)
x86 Opteron_G5-v1 AMD Opteron 63xx class CPU
x86 Penryn (alias configured by machine type)
x86 Penryn-v1 Intel Core 2 Duo P9xxx (Penryn Class Core 2)
x86 SandyBridge (alias configured by machine type)
x86 SandyBridge-IBRS (alias of SandyBridge-v2)
x86 SandyBridge-v1 Intel Xeon E312xx (Sandy Bridge)
x86 SandyBridge-v2 Intel Xeon E312xx (Sandy Bridge, IBRS update)
x86 Skylake-Client (alias configured by machine type)
x86 Skylake-Client-IBRS (alias of Skylake-Client-v2)
x86 Skylake-Client-noTSX-IBRS (alias of Skylake-Client-v3)
x86 Skylake-Client-v1 Intel Core Processor (Skylake)
x86 Skylake-Client-v2 Intel Core Processor (Skylake, IBRS)
x86 Skylake-Client-v3 Intel Core Processor (Skylake, IBRS)
x86 Skylake-Server (alias configured by machine type)
x86 Skylake-Server-IBRS (alias of Skylake-Server-v2)
x86 Skylake-Server-noTSX-IBRS (alias of Skylake-Server-v3)
x86 Skylake-Server-v1 Intel Xeon Processor (Skylake)
x86 Skylake-Server-v2 Intel Xeon Processor (Skylake, IBRS)
x86 Skylake-Server-v3 Intel Xeon Processor (Skylake, IBRS)
x86 Snowridge (alias configured by machine type)
x86 Snowridge-v1 Intel Atom Processor (SnowRidge)
x86 Snowridge-v2 Intel Atom Processor (Snowridge, no MPX)
x86 Westmere (alias configured by machine type)
x86 Westmere-IBRS (alias of Westmere-v2)
x86 Westmere-v1 Westmere E56xx/L56xx/X56xx (Nehalem-C)
x86 Westmere-v2 Westmere E56xx/L56xx/X56xx (IBRS update)
x86 athlon (alias configured by machine type)
x86 athlon-v1 QEMU Virtual CPU version 2.5+
x86 core2duo (alias configured by machine type)
x86 core2duo-v1 Intel(R) Core(TM)2 Duo CPU T7700 @ 2.40GHz
x86 coreduo (alias configured by machine type)
x86 coreduo-v1 Genuine Intel(R) CPU T2600 @ 2.16GHz
x86 kvm32 (alias configured by machine type)
x86 kvm32-v1 Common 32-bit KVM processor
x86 kvm64 (alias configured by machine type)
x86 kvm64-v1 Common KVM processor
x86 n270 (alias configured by machine type)
x86 n270-v1 Intel(R) Atom(TM) CPU N270 @ 1.60GHz
x86 pentium (alias configured by machine type)
x86 pentium-v1
x86 pentium2 (alias configured by machine type)
x86 pentium2-v1
x86 pentium3 (alias configured by machine type)
x86 pentium3-v1
x86 phenom (alias configured by machine type)
x86 phenom-v1 AMD Phenom(tm) 9550 Quad-Core Processor
x86 qemu32 (alias configured by machine type)
x86 qemu32-v1 QEMU Virtual CPU version 2.5+
x86 qemu64 (alias configured by machine type)
x86 qemu64-v1 QEMU Virtual CPU version 2.5+
x86 base base CPU model type with no features enabled
x86 host KVM processor with all supported host features
x86 max Enables all features supported by the accelerator in the current host
@idzikovski commented on GitHub (Jul 24, 2021):
`Available CPUs:
x86 486 (alias configured by machine type)
x86 486-v1
x86 Broadwell (alias configured by machine type)
x86 Broadwell-IBRS (alias of Broadwell-v3)
x86 Broadwell-noTSX (alias of Broadwell-v2)
x86 Broadwell-noTSX-IBRS (alias of Broadwell-v4)
x86 Broadwell-v1 Intel Core Processor (Broadwell)
x86 Broadwell-v2 Intel Core Processor (Broadwell, no TSX)
x86 Broadwell-v3 Intel Core Processor (Broadwell, IBRS)
x86 Broadwell-v4 Intel Core Processor (Broadwell, no TSX, IBRS)
x86 Cascadelake-Server (alias configured by machine type)
x86 Cascadelake-Server-noTSX (alias of Cascadelake-Server-v3)
x86 Cascadelake-Server-v1 Intel Xeon Processor (Cascadelake)
x86 Cascadelake-Server-v2 Intel Xeon Processor (Cascadelake)
x86 Cascadelake-Server-v3 Intel Xeon Processor (Cascadelake)
x86 Conroe (alias configured by machine type)
x86 Conroe-v1 Intel Celeron_4x0 (Conroe/Merom Class Core 2)
x86 Cooperlake (alias configured by machine type)
x86 Cooperlake-v1 Intel Xeon Processor (Cooperlake)
x86 Denverton (alias configured by machine type)
x86 Denverton-v1 Intel Atom Processor (Denverton)
x86 Dhyana (alias configured by machine type)
x86 Dhyana-v1 Hygon Dhyana Processor
x86 EPYC (alias configured by machine type)
x86 EPYC-IBPB (alias of EPYC-v2)
x86 EPYC-Milan (alias configured by machine type)
x86 EPYC-Milan-v1 AMD EPYC-Milan Processor
x86 EPYC-Rome (alias configured by machine type)
x86 EPYC-Rome-v1 AMD EPYC-Rome Processor
x86 EPYC-Rome-v2 AMD EPYC-Rome Processor
x86 EPYC-v1 AMD EPYC Processor
x86 EPYC-v2 AMD EPYC Processor (with IBPB)
x86 EPYC-v3 AMD EPYC Processor
x86 Haswell (alias configured by machine type)
x86 Haswell-IBRS (alias of Haswell-v3)
x86 Haswell-noTSX (alias of Haswell-v2)
x86 Haswell-noTSX-IBRS (alias of Haswell-v4)
x86 Haswell-v1 Intel Core Processor (Haswell)
x86 Haswell-v2 Intel Core Processor (Haswell, no TSX)
x86 Haswell-v3 Intel Core Processor (Haswell, IBRS)
x86 Haswell-v4 Intel Core Processor (Haswell, no TSX, IBRS)
x86 Icelake-Client (alias configured by machine type)
x86 Icelake-Client-noTSX (alias of Icelake-Client-v2)
x86 Icelake-Client-v1 Intel Core Processor (Icelake)
x86 Icelake-Client-v2 Intel Core Processor (Icelake)
x86 Icelake-Server (alias configured by machine type)
x86 Icelake-Server-noTSX (alias of Icelake-Server-v2)
x86 Icelake-Server-v1 Intel Xeon Processor (Icelake)
x86 Icelake-Server-v2 Intel Xeon Processor (Icelake)
x86 IvyBridge (alias configured by machine type)
x86 IvyBridge-IBRS (alias of IvyBridge-v2)
x86 IvyBridge-v1 Intel Xeon E3-12xx v2 (Ivy Bridge)
x86 IvyBridge-v2 Intel Xeon E3-12xx v2 (Ivy Bridge, IBRS)
x86 KnightsMill (alias configured by machine type)
x86 KnightsMill-v1 Intel Xeon Phi Processor (Knights Mill)
x86 Nehalem (alias configured by machine type)
x86 Nehalem-IBRS (alias of Nehalem-v2)
x86 Nehalem-v1 Intel Core i7 9xx (Nehalem Class Core i7)
x86 Nehalem-v2 Intel Core i7 9xx (Nehalem Core i7, IBRS update)
x86 Opteron_G1 (alias configured by machine type)
x86 Opteron_G1-v1 AMD Opteron 240 (Gen 1 Class Opteron)
x86 Opteron_G2 (alias configured by machine type)
x86 Opteron_G2-v1 AMD Opteron 22xx (Gen 2 Class Opteron)
x86 Opteron_G3 (alias configured by machine type)
x86 Opteron_G3-v1 AMD Opteron 23xx (Gen 3 Class Opteron)
x86 Opteron_G4 (alias configured by machine type)
x86 Opteron_G4-v1 AMD Opteron 62xx class CPU
x86 Opteron_G5 (alias configured by machine type)
x86 Opteron_G5-v1 AMD Opteron 63xx class CPU
x86 Penryn (alias configured by machine type)
x86 Penryn-v1 Intel Core 2 Duo P9xxx (Penryn Class Core 2)
x86 SandyBridge (alias configured by machine type)
x86 SandyBridge-IBRS (alias of SandyBridge-v2)
x86 SandyBridge-v1 Intel Xeon E312xx (Sandy Bridge)
x86 SandyBridge-v2 Intel Xeon E312xx (Sandy Bridge, IBRS update)
x86 Skylake-Client (alias configured by machine type)
x86 Skylake-Client-IBRS (alias of Skylake-Client-v2)
x86 Skylake-Client-noTSX-IBRS (alias of Skylake-Client-v3)
x86 Skylake-Client-v1 Intel Core Processor (Skylake)
x86 Skylake-Client-v2 Intel Core Processor (Skylake, IBRS)
x86 Skylake-Client-v3 Intel Core Processor (Skylake, IBRS)
x86 Skylake-Server (alias configured by machine type)
x86 Skylake-Server-IBRS (alias of Skylake-Server-v2)
x86 Skylake-Server-noTSX-IBRS (alias of Skylake-Server-v3)
x86 Skylake-Server-v1 Intel Xeon Processor (Skylake)
x86 Skylake-Server-v2 Intel Xeon Processor (Skylake, IBRS)
x86 Skylake-Server-v3 Intel Xeon Processor (Skylake, IBRS)
x86 Snowridge (alias configured by machine type)
x86 Snowridge-v1 Intel Atom Processor (SnowRidge)
x86 Snowridge-v2 Intel Atom Processor (Snowridge, no MPX)
x86 Westmere (alias configured by machine type)
x86 Westmere-IBRS (alias of Westmere-v2)
x86 Westmere-v1 Westmere E56xx/L56xx/X56xx (Nehalem-C)
x86 Westmere-v2 Westmere E56xx/L56xx/X56xx (IBRS update)
x86 athlon (alias configured by machine type)
x86 athlon-v1 QEMU Virtual CPU version 2.5+
x86 core2duo (alias configured by machine type)
x86 core2duo-v1 Intel(R) Core(TM)2 Duo CPU T7700 @ 2.40GHz
x86 coreduo (alias configured by machine type)
x86 coreduo-v1 Genuine Intel(R) CPU T2600 @ 2.16GHz
x86 kvm32 (alias configured by machine type)
x86 kvm32-v1 Common 32-bit KVM processor
x86 kvm64 (alias configured by machine type)
x86 kvm64-v1 Common KVM processor
x86 n270 (alias configured by machine type)
x86 n270-v1 Intel(R) Atom(TM) CPU N270 @ 1.60GHz
x86 pentium (alias configured by machine type)
x86 pentium-v1
x86 pentium2 (alias configured by machine type)
x86 pentium2-v1
x86 pentium3 (alias configured by machine type)
x86 pentium3-v1
x86 phenom (alias configured by machine type)
x86 phenom-v1 AMD Phenom(tm) 9550 Quad-Core Processor
x86 qemu32 (alias configured by machine type)
x86 qemu32-v1 QEMU Virtual CPU version 2.5+
x86 qemu64 (alias configured by machine type)
x86 qemu64-v1 QEMU Virtual CPU version 2.5+
x86 base base CPU model type with no features enabled
x86 host KVM processor with all supported host features
x86 max Enables all features supported by the accelerator in the current host
Recognized CPUID flags:
3dnow 3dnowext 3dnowprefetch abm ace2 ace2-en acpi adx aes amd-no-ssb
amd-ssbd amd-stibp apic arat arch-capabilities avic avx avx2
avx512-4fmaps avx512-4vnniw avx512-bf16 avx512-vpopcntdq avx512bitalg
avx512bw avx512cd avx512dq avx512er avx512f avx512ifma avx512pf
avx512vbmi avx512vbmi2 avx512vl avx512vnni bmi1 bmi2 cid cldemote clflush
clflushopt clwb clzero cmov cmp-legacy core-capability cr8legacy cx16 cx8
dca de decodeassists ds ds-cpl dtes64 erms est extapic f16c flushbyasid
fma fma4 fpu fsgsbase fsrm fxsr fxsr-opt gfni hle ht hypervisor ia64 ibpb
ibrs ibrs-all ibs intel-pt invpcid invtsc kvm-asyncpf kvm-hint-dedicated
kvm-mmu kvm-nopiodelay kvm-poll-control kvm-pv-eoi kvm-pv-ipi
kvm-pv-sched-yield kvm-pv-tlb-flush kvm-pv-unhalt kvm-steal-time kvmclock
kvmclock kvmclock-stable-bit la57 lahf-lm lbrv lm lwp mca mce md-clear
mds-no misalignsse mmx mmxext monitor movbe movdir64b movdiri mpx msr
mtrr nodeid-msr npt nrip-save nx osvw pae pat pause-filter pbe pcid
pclmulqdq pcommit pdcm pdpe1gb perfctr-core perfctr-nb pfthreshold pge
phe phe-en pku pmm pmm-en pn pni popcnt pschange-mc-no pse pse36 rdctl-no
rdpid rdrand rdseed rdtscp rsba rtm sep sha-ni skinit skip-l1dfl-vmentry
smap smep smx spec-ctrl split-lock-detect ss ssb-no ssbd sse sse2 sse4.1
sse4.2 sse4a ssse3 stibp svm svm-lock svme-addr-chk syscall taa-no tbm
tce tm tm2 topoext tsc tsc-adjust tsc-deadline tsc-scale tsx-ctrl umip
v-vmsave-vmload vaes vgif virt-ssbd vmcb-clean vme vmx vmx-activity-hlt
vmx-activity-shutdown vmx-activity-wait-sipi vmx-apicv-register
vmx-apicv-vid vmx-apicv-x2apic vmx-apicv-xapic vmx-cr3-load-noexit
vmx-cr3-store-noexit vmx-cr8-load-exit vmx-cr8-store-exit vmx-desc-exit
vmx-encls-exit vmx-entry-ia32e-mode vmx-entry-load-bndcfgs
vmx-entry-load-efer vmx-entry-load-pat vmx-entry-load-perf-global-ctrl
vmx-entry-load-rtit-ctl vmx-entry-noload-debugctl vmx-ept vmx-ept-1gb
vmx-ept-2mb vmx-ept-advanced-exitinfo vmx-ept-execonly vmx-eptad
vmx-eptp-switching vmx-exit-ack-intr vmx-exit-clear-bndcfgs
vmx-exit-clear-rtit-ctl vmx-exit-load-efer vmx-exit-load-pat
vmx-exit-load-perf-global-ctrl vmx-exit-nosave-debugctl
vmx-exit-save-efer vmx-exit-save-pat vmx-exit-save-preemption-timer
vmx-flexpriority vmx-hlt-exit vmx-ins-outs vmx-intr-exit vmx-invept
vmx-invept-all-context vmx-invept-single-context
vmx-invept-single-context vmx-invept-single-context-noglobals
vmx-invlpg-exit vmx-invpcid-exit vmx-invvpid vmx-invvpid-all-context
vmx-invvpid-single-addr vmx-io-bitmap vmx-io-exit vmx-monitor-exit
vmx-movdr-exit vmx-msr-bitmap vmx-mtf vmx-mwait-exit vmx-nmi-exit
vmx-page-walk-4 vmx-page-walk-5 vmx-pause-exit vmx-ple vmx-pml
vmx-posted-intr vmx-preemption-timer vmx-rdpmc-exit vmx-rdrand-exit
vmx-rdseed-exit vmx-rdtsc-exit vmx-rdtscp-exit vmx-secondary-ctls
vmx-shadow-vmcs vmx-store-lma vmx-true-ctls vmx-tsc-offset
vmx-unrestricted-guest vmx-vintr-pending vmx-vmfunc
vmx-vmwrite-vmexit-fields vmx-vnmi vmx-vnmi-pending vmx-vpid
vmx-wbinvd-exit vmx-xsaves vmx-zero-len-inject vpclmulqdq waitpkg
wbnoinvd wdt x2apic xcrypt xcrypt-en xgetbv1 xop xsave xsavec xsaveerptr
xsaveopt xsaves xstore xstore-en xtpr
`
This produces the halt reboot error, so yes this bug is related to CPUID_FLAGS, any ideas which one is the culprit and if it's required for all systems? I was going to do trial and error till I found it using
-e CPUID_FLAGS= \
@sickcodes commented on GitHub (Jul 24, 2021):
I was able to reproduce this issue when the host does not have KVM.
```
docker run -it \
-e CPU=Penryn \
-e CPUID_FLAGS= \
-e KVM= \
-p 50922:10022 \
-v /tmp/.X11-unix:/tmp/.X11-unix \
-e "DISPLAY=${DISPLAY:-:0.0}" \
sickcodes/docker-osx:latest
```
This produces the halt reboot error, so yes this bug is related to CPUID_FLAGS, any ideas which one is the culprit and if it's required for all systems? I was going to do trial and error till I found it using
` -e CPUID_FLAGS= \`
@sickcodes commented on GitHub (Jul 24, 2021):
For contrast and to emphasize the CPUID flag importance, this passes the start but seg faults right after.
```bash
docker run -it \
-e CPU=Penryn \
-e CPUID_FLAGS="vendor=GenuineIntel," \
-e KVM= \
-p 50922:10022 \
-v /tmp/.X11-unix:/tmp/.X11-unix \
-e "DISPLAY=${DISPLAY:-:0.0}" \
sickcodes/docker-osx:latest
```
```
any ideas which one is the culprit and if it's required for all systems?
Nope, I am not that knowledgable. I am simply trying to run this image using the instructions in the readme. My goal is to do a proof of concept for a mac build agent for CI/CD running in docker
@idzikovski commented on GitHub (Jul 24, 2021):
>any ideas which one is the culprit and if it's required for all systems?
Nope, I am not that knowledgable. I am simply trying to run this image using the instructions in the readme. My goal is to do a proof of concept for a mac build agent for CI/CD running in docker
I am able to replicate the exact error that @3xr displayed above and it's only occurring when qemu is unallowed to use /dev/kvm or when it's not passed in on my host or when I remove all the CPUID flags by setting the variable to nothing.
Can you confirm it's doing it with the pure vanilla run command?
@sickcodes commented on GitHub (Jul 24, 2021):
Using `-e CPU=host \` instead of Penryn may work.
I am able to replicate the exact error that @3xr displayed above and it's only occurring when qemu is unallowed to use /dev/kvm or when it's not passed in on my host or when I remove all the CPUID flags by setting the variable to nothing.
Can you confirm it's doing it with the pure vanilla run command?
```bash
docker run -it \
--device /dev/kvm \
-p 50922:10022 \
-v /tmp/.X11-unix:/tmp/.X11-unix \
-e "DISPLAY=${DISPLAY:-:0.0}" \
sickcodes/docker-osx:latest
```
With this command I didn't get the panic but I encountered the issue where the mouse gets stuck in the screen where you can choose between disk utility and reinstall
@idzikovski commented on GitHub (Jul 26, 2021):
With this command I didn't get the panic but I encountered the issue where the mouse gets stuck in the screen where you can choose between disk utility and reinstall
it's only occurring when qemu is unallowed to use /dev/kvm or when it's not passed in on my host
Thanks for the remarks!
Actually it seems that it may be linked that the --userns-remap=default option is enabled on my Docker setup, and that the dockremap user which is created automatically by Docker in such conditions was not added to the kvm or libvirt groups.
I have tried to add the dockremap user to the kvm group, but I can't download a truckload of container images to re-test it right now as I'm on a countryside connection.
@3xr commented on GitHub (Aug 18, 2021):
> it's only occurring when qemu is unallowed to use /dev/kvm or when it's not passed in on my host
Thanks for the remarks!
Actually it seems that it may be linked that the `--userns-remap=default` option is enabled on my Docker setup, and that the `dockremap` user which is created automatically by Docker in such conditions was not added to the `kvm` or `libvirt` groups.
```json
$ cat /etc/docker/daemon.json
{
"userns-remap": "default"
}
$ grep dockremap /etc/passwd
dockremap:x:131:143::/home/dockremap:/bin/false
```
I have tried to add the `dockremap` user to the `kvm` group, but I can't download a truckload of container images to re-test it right now as I'm on a countryside connection.
ForceOcWriteFlash
Type: plist boolean
Failsafe: false
Description: Enables writing to flash memory for all OpenCore system variables.
Note: This value should be disabled on most types of firmware but is left configurable to account for firmware that may have issues with volatile variable storage overflows or similar. Boot issues across multiple OSes can be observed on e.g. Lenovo Thinkpad T430 and T530 without this quirk. Apple variables related to Secure Boot and hibernation are exempt from this for security reasons. Furthermore, some OpenCore variables are exempt for different reasons, such as the boot log due to an available user option, and the TSC frequency due to timing issues. When toggling this option, a NVRAM reset may be required to ensure full functionality.
@sickcodes commented on GitHub (Oct 31, 2021):
Note with Thinkpads the following from https://dortania.github.io/docs/latest/Configuration.html
>ForceOcWriteFlash
>Type: plist boolean
>Failsafe: false
>Description: Enables writing to flash memory for all OpenCore system variables.
>Note: This value should be disabled on most types of firmware but is left configurable to account for firmware that may have issues with volatile variable storage overflows or similar. Boot issues across multiple OSes can be observed on e.g. Lenovo Thinkpad T430 and T530 without this quirk. Apple variables related to Secure Boot and hibernation are exempt from this for security reasons. Furthermore, some OpenCore variables are exempt for different reasons, such as the boot log due to an available user option, and the TSC frequency due to timing issues. When toggling this option, a NVRAM reset may be required to ensure full functionality.
```
@murilopereirame commented on GitHub (Oct 30, 2023):
Using -e CPU=host \ instead of Penryn may work.
I am able to replicate the exact error that @3xr displayed above and it's only occurring when qemu is unallowed to use /dev/kvm or when it's not passed in on my host or when I remove all the CPUID flags by setting the variable to nothing.
Can you confirm it's doing it with the pure vanilla run command?
Using the flag -e CPU=host solves the problem of corecrypto but maybe throw a permission error related to KVM, not big deal, just adjust the permissions of /dev/kvm and re-run the command
@murilopereirame commented on GitHub (Oct 30, 2023):
> Using `-e CPU=host \` instead of Penryn may work.
>
> I am able to replicate the exact error that @3xr displayed above and it's only occurring when qemu is unallowed to use /dev/kvm or when it's not passed in on my host or when I remove all the CPUID flags by setting the variable to nothing.
>
> Can you confirm it's doing it with the pure vanilla run command?
>
> ```shell
> docker run -it \
> --device /dev/kvm \
> -p 50922:10022 \
> -v /tmp/.X11-unix:/tmp/.X11-unix \
> -e "DISPLAY=${DISPLAY:-:0.0}" \
> sickcodes/docker-osx:latest
> ```
Using the flag `-e CPU=host` solves the problem of corecrypto but maybe throw a permission error related to KVM, not big deal, just adjust the permissions of /dev/kvm and re-run the command
@JAicewizard commented on GitHub (Jun 1, 2024):
See https://github.com/sickcodes/Docker-OSX/issues/380#issuecomment-2143425540, maybe also resolves this issue. Also running a thinkpad bug a t14 amd gen 1, for me a bios update did it.
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
Originally created by @3xr on GitHub (Jul 3, 2021).
Hello,
When trying to run the Big Sur image using the following command, after having installed at the requirements and dependencies:
(The USB forwarding directive is just for forwarding a MIDI keyboard, I can try without it if needed (Edit: same thing without it))
The QEMU window spawns, and I can the select the macOS disk as a boot device, but after 1-2 minutes I'm getting a kernel panic about something happening in the
com.apple.kec.corecrypto(11.1)module of macOS:And before that I have this screen (outlining an "
Err(0xE) <- EB.WL.PWLFNV - RT.GV failureerror)?I'm using a standard Lenovo Thinkpad T460s (2016) on Ubuntu 20.04 LTS with extended RAM and SSD, and with the original Intel CPU and iGPU.
Would you have any idea about the possible cause of the issue?
Regards,
Debug logs
@3xr commented on GitHub (Jul 3, 2021):
I encounter the same issue with the Catalina image, while we can see a larger backtrace thanks to the higher screen resolution:
I deleted the former Docker images and ran the following command this time:
@sickcodes commented on GitHub (Jul 4, 2021):
Try this style of USB passthrough:
-e EXTRA='-device usb-host,hostbus=1,hostaddr=4' \@3xr commented on GitHub (Jul 4, 2021):
Hello,
As I precised above, removing the USB forwarding line changes nothing at all.
As you can see in the backtrace, this "Unknown Opcode" CPU fault seems to lay in the
_gcm_initroutine of Apple's CoreCrypto library, which seems to depend both on the SSSE3 and AES-NI instruction extension sets.My host CPU (Intel Core i7-6600U) correctly supports both (it doesn't support AVX-512 which doesn't seem to be known to be involved here).
$ grep ssse3 /proc/cpuinfo | head -1flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc art arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc cpuid aperfmperf pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 sdbg fma cx16 xtpr pdcm pcid sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm abm 3dnowprefetch cpuid_fault epb invpcid_single pti ssbd ibrs ibpb stibp tpr_shadow vnmi flexpriority ept vpid ept_ad fsgsbase tsc_adjust bmi1 hle avx2 smep bmi2 erms invpcid rtm mpx rdseed adx smap clflushopt intel_pt xsaveopt xsavec xgetbv1 xsaves dtherm ida arat pln pts hwp hwp_notify hwp_act_window hwp_epp md_clear flush_l1dI think that it would be good to disassemble the Catalina kernel at the address provided by the backtrace in order to perform helpful investigation, but I don't know how to extract it. Maybe that the CR registers give a hint with this kind of exception?
An important thing to note is that while both the Big Sur and Catalina images provided by Docker-OSX fail with this error at boot,
sosumiwhich also relies on QEMU (but with a different stack behind) boots Catalina well on my machine.Your CPU configuration lines doesn't seem to contain something extravagant to me, although I'm not a QEMU expert. Maybe that these should be compared with more attention or detail with other macOS virtualization scripts.
Regards,
@3xr commented on GitHub (Jul 4, 2021):
Actually the code simply lays in its own kernel extension file (the
macOS Base System/System/Library/Extensions/corecrypto.kext/Contents/MacOS/corecryptofile from thediff/home/arch/OSX-KVM/BaseSystem.imgimage from the container).Actually the code at
_gcm_init + 8eis not SSSE3 or AES-NI (SSSE3 instructions likepshufbor SSE2 instructions likemovdqaorpslldqperform well according to the position in the control flow), it's apclmulqdqfrom the CLMUL (2010) instruction set extension.Brace yourself for IDA screenshot
An instruction set extension that my processor indicates to support, obviously
$ grep pclmulqdq /proc/cpuinfo | head -1flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc art arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc cpuid aperfmperf pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 sdbg fma cx16 xtpr pdcm pcid sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm abm 3dnowprefetch cpuid_fault epb invpcid_single pti ssbd ibrs ibpb stibp tpr_shadow vnmi flexpriority ept vpid ept_ad fsgsbase tsc_adjust bmi1 hle avx2 smep bmi2 erms invpcid rtm mpx rdseed adx smap clflushopt intel_pt xsaveopt xsavec xgetbv1 xsaves dtherm ida arat pln pts hwp hwp_notify hwp_act_window hwp_epp md_clear flush_l1dI wonder why the crash, then.
@3xr commented on GitHub (Jul 4, 2021):
Additional insight:
sosumi(non-panicking)/snap/sosumi/18/usr/bin/qemu-system-x86_64 -enable-kvm -m 2G -machine q35,accel=kvm -smp 4,cores=2 -cpu Penryn,vendor=GenuineIntel,kvm=on,+sse3,+sse4.2,+aes,+xsave,+avx,+xsaveopt,+xsavec,+xgetbv1,+avx2,+bmi2,+smep,+bmi1,+fma,+movbe,+invtsc -device isa-applesmc,osk=ourhardworkbythesewordsguardedpleasedontsteal(c)AppleComputerInc -smbios type=2 -object rng-random,id=rng0,filename=/dev/urandom -device virtio-rng-pci,rng=rng0 -serial mon:stdio -drive if=pflash,format=raw,readonly,file=/snap/sosumi/18/firmware/OVMF_CODE.fd -drive if=pflash,format=raw,file=/home/myusername/snap/sosumi/common/firmware/OVMF_VARS-1024x768.fd -device virtio-vga,virgl=on -display sdl,gl=on -L /snap/sosumi/18/usr/share/seabios/ -L /snap/sosumi/18/usr/lib/ipxe/qemu/ -audiodev pa,id=pa,server=/run/user/1000/pulse/native -device ich9-intel-hda -device hda-output,audiodev=pa -usb -device usb-kbd -device usb-mouse -netdev user,id=net0 -device vmxnet3,netdev=net0,id=net0 -drive id=ESP,if=virtio,format=qcow2,file=/home/myusername/snap/sosumi/common/ESP.qcow2 -drive id=SystemDisk,if=virtio,file=/home/myusername/snap/sosumi/common/macos.qcow2 -device usb-host,vendorid=0x0763,productid=0x3111,guest-reset=falseexec qemu-system-x86_64 -m 3000 -cpu Penryn,vendor=GenuineIntel,+invtsc,vmware-cpuid-freq=on,+pcid,+ssse3,+sse4.2,+popcnt,+avx,+aes,+xsave,+xsaveopt,check, -machine q35,accel=kvm:tcg -smp 4,cores=4 -usb -device usb-kbd -device usb-tablet -device 'isa-applesmc,osk=ourhardworkbythesewordsguardedpleasedontsteal(c)AppleComputerInc' -drive if=pflash,format=raw,readonly,file=/home/arch/OSX-KVM/OVMF_CODE.fd -drive if=pflash,format=raw,file=/home/arch/OSX-KVM/OVMF_VARS-1024x768.fd -smbios type=2 -audiodev alsa,id=hda -device ich9-intel-hda -device hda-duplex,audiodev=hda -device ich9-ahci,id=sata -drive id=OpenCoreBoot,if=none,snapshot=on,format=qcow2,file=/home/arch/OSX-KVM/OpenCore-Catalina/OpenCore.qcow2 -device ide-hd,bus=sata.2,drive=OpenCoreBoot -device ide-hd,bus=sata.3,drive=InstallMedia -drive id=InstallMedia,if=none,file=/home/arch/OSX-KVM/BaseSystem.img,format=qcow2 -drive id=MacHDD,if=none,file=/home/arch/OSX-KVM/mac_hdd_ng.img,format=qcow2 -device ide-hd,bus=sata.4,drive=MacHDD -netdev user,id=net0,hostfwd=tcp::10022-:22,hostfwd=tcp::5900-:5900, -device vmxnet3,netdev=net0,id=net0,mac=52:54:00:09:49:17 -monitor stdio -vga vmwareThe
corecryptokernel extension binary fromsosumi's image also has apclmulqdqinstruction at_gcm_init + 8e.@3xr commented on GitHub (Jul 4, 2021):
It looks like it may be linked to the
+ssse3flag in the Docker-OSX configuration, which is not present in Sosumi`'s. The CoreCrypto module does a CPUID flag check on this feature just before calling. ^^See: https://opensource.apple.com/source/xnu/xnu-1504.9.17/osfmk/i386/cpuid.h.auto.html
I'd suggest to align on the
-cpuflags of Sosumi and remove the+ssse3(see the command above). (Or maybe that adding the+pclmulqdqflag as specified inqemu-system-x86_64 -cpu helpwould change something, if it does not only affect the returnedcpuid.)@sickcodes commented on GitHub (Jul 4, 2021):
This is excellent, thank you for posting all these, I'll remove the offending flag in about 20 minutes 😁
@sickcodes commented on GitHub (Jul 4, 2021):
RE: Sosumi boot args:
Also there are some changes made recently upstream in KVM-Opencore to prepare for Monterrey that I haven't merged yet as I haven't had time to test:
https://raw.githubusercontent.com/sickcodes/osx-serial-generator/v13/config-custom.plistAs seen in:
https://github.com/thenickdude/KVM-Opencore/releases/tag/v13
Changes ready to merge:
https://github.com/sickcodes/osx-serial-generator/pull/13
Also as we speak, I'm working on https://github.com/sickcodes/dock-droid and lots of random qemu args that we currently have can be removed, namely due to QEMU 6, but also older stuff.
For example, simply using:
And also adding:
Testing right now
https://github.com/sickcodes/osx-serial-generator/pull/13
@3xr commented on GitHub (Jul 8, 2021):
Hello,
Regarding your last diff, is this a change that you done or a change that you are projecting to do? It seems that the mentions of CPU configuration in your repository didn't change yet.
Regards
@sickcodes commented on GitHub (Jul 9, 2021):
Added (just now) in https://github.com/sickcodes/Docker-OSX/commit/866a429848e8bca422a10712dae0711cbbdad99b
You can now use:
-e CPUID_FLAGS='vendor=GenuineIntel,+invtsc,vmware-cpuid-freq=on,+pcid,+ssse3,+sse4.2,+popcnt,+avx,+aes,+xsave,+xsaveopt,check,' \@idzikovski commented on GitHub (Jul 23, 2021):
I am having this issue as well. Is there any way to overcome this?
@sickcodes commented on GitHub (Jul 23, 2021):
Can you let me know the name of the CPU you are using?
Please post the output of:
qemu-system-x86_64 -cpu helpFor example, here are mine:
@idzikovski commented on GitHub (Jul 24, 2021):
`Available CPUs:
x86 486 (alias configured by machine type)
x86 486-v1
x86 Broadwell (alias configured by machine type)
x86 Broadwell-IBRS (alias of Broadwell-v3)
x86 Broadwell-noTSX (alias of Broadwell-v2)
x86 Broadwell-noTSX-IBRS (alias of Broadwell-v4)
x86 Broadwell-v1 Intel Core Processor (Broadwell)
x86 Broadwell-v2 Intel Core Processor (Broadwell, no TSX)
x86 Broadwell-v3 Intel Core Processor (Broadwell, IBRS)
x86 Broadwell-v4 Intel Core Processor (Broadwell, no TSX, IBRS)
x86 Cascadelake-Server (alias configured by machine type)
x86 Cascadelake-Server-noTSX (alias of Cascadelake-Server-v3)
x86 Cascadelake-Server-v1 Intel Xeon Processor (Cascadelake)
x86 Cascadelake-Server-v2 Intel Xeon Processor (Cascadelake)
x86 Cascadelake-Server-v3 Intel Xeon Processor (Cascadelake)
x86 Conroe (alias configured by machine type)
x86 Conroe-v1 Intel Celeron_4x0 (Conroe/Merom Class Core 2)
x86 Cooperlake (alias configured by machine type)
x86 Cooperlake-v1 Intel Xeon Processor (Cooperlake)
x86 Denverton (alias configured by machine type)
x86 Denverton-v1 Intel Atom Processor (Denverton)
x86 Dhyana (alias configured by machine type)
x86 Dhyana-v1 Hygon Dhyana Processor
x86 EPYC (alias configured by machine type)
x86 EPYC-IBPB (alias of EPYC-v2)
x86 EPYC-Milan (alias configured by machine type)
x86 EPYC-Milan-v1 AMD EPYC-Milan Processor
x86 EPYC-Rome (alias configured by machine type)
x86 EPYC-Rome-v1 AMD EPYC-Rome Processor
x86 EPYC-Rome-v2 AMD EPYC-Rome Processor
x86 EPYC-v1 AMD EPYC Processor
x86 EPYC-v2 AMD EPYC Processor (with IBPB)
x86 EPYC-v3 AMD EPYC Processor
x86 Haswell (alias configured by machine type)
x86 Haswell-IBRS (alias of Haswell-v3)
x86 Haswell-noTSX (alias of Haswell-v2)
x86 Haswell-noTSX-IBRS (alias of Haswell-v4)
x86 Haswell-v1 Intel Core Processor (Haswell)
x86 Haswell-v2 Intel Core Processor (Haswell, no TSX)
x86 Haswell-v3 Intel Core Processor (Haswell, IBRS)
x86 Haswell-v4 Intel Core Processor (Haswell, no TSX, IBRS)
x86 Icelake-Client (alias configured by machine type)
x86 Icelake-Client-noTSX (alias of Icelake-Client-v2)
x86 Icelake-Client-v1 Intel Core Processor (Icelake)
x86 Icelake-Client-v2 Intel Core Processor (Icelake)
x86 Icelake-Server (alias configured by machine type)
x86 Icelake-Server-noTSX (alias of Icelake-Server-v2)
x86 Icelake-Server-v1 Intel Xeon Processor (Icelake)
x86 Icelake-Server-v2 Intel Xeon Processor (Icelake)
x86 IvyBridge (alias configured by machine type)
x86 IvyBridge-IBRS (alias of IvyBridge-v2)
x86 IvyBridge-v1 Intel Xeon E3-12xx v2 (Ivy Bridge)
x86 IvyBridge-v2 Intel Xeon E3-12xx v2 (Ivy Bridge, IBRS)
x86 KnightsMill (alias configured by machine type)
x86 KnightsMill-v1 Intel Xeon Phi Processor (Knights Mill)
x86 Nehalem (alias configured by machine type)
x86 Nehalem-IBRS (alias of Nehalem-v2)
x86 Nehalem-v1 Intel Core i7 9xx (Nehalem Class Core i7)
x86 Nehalem-v2 Intel Core i7 9xx (Nehalem Core i7, IBRS update)
x86 Opteron_G1 (alias configured by machine type)
x86 Opteron_G1-v1 AMD Opteron 240 (Gen 1 Class Opteron)
x86 Opteron_G2 (alias configured by machine type)
x86 Opteron_G2-v1 AMD Opteron 22xx (Gen 2 Class Opteron)
x86 Opteron_G3 (alias configured by machine type)
x86 Opteron_G3-v1 AMD Opteron 23xx (Gen 3 Class Opteron)
x86 Opteron_G4 (alias configured by machine type)
x86 Opteron_G4-v1 AMD Opteron 62xx class CPU
x86 Opteron_G5 (alias configured by machine type)
x86 Opteron_G5-v1 AMD Opteron 63xx class CPU
x86 Penryn (alias configured by machine type)
x86 Penryn-v1 Intel Core 2 Duo P9xxx (Penryn Class Core 2)
x86 SandyBridge (alias configured by machine type)
x86 SandyBridge-IBRS (alias of SandyBridge-v2)
x86 SandyBridge-v1 Intel Xeon E312xx (Sandy Bridge)
x86 SandyBridge-v2 Intel Xeon E312xx (Sandy Bridge, IBRS update)
x86 Skylake-Client (alias configured by machine type)
x86 Skylake-Client-IBRS (alias of Skylake-Client-v2)
x86 Skylake-Client-noTSX-IBRS (alias of Skylake-Client-v3)
x86 Skylake-Client-v1 Intel Core Processor (Skylake)
x86 Skylake-Client-v2 Intel Core Processor (Skylake, IBRS)
x86 Skylake-Client-v3 Intel Core Processor (Skylake, IBRS)
x86 Skylake-Server (alias configured by machine type)
x86 Skylake-Server-IBRS (alias of Skylake-Server-v2)
x86 Skylake-Server-noTSX-IBRS (alias of Skylake-Server-v3)
x86 Skylake-Server-v1 Intel Xeon Processor (Skylake)
x86 Skylake-Server-v2 Intel Xeon Processor (Skylake, IBRS)
x86 Skylake-Server-v3 Intel Xeon Processor (Skylake, IBRS)
x86 Snowridge (alias configured by machine type)
x86 Snowridge-v1 Intel Atom Processor (SnowRidge)
x86 Snowridge-v2 Intel Atom Processor (Snowridge, no MPX)
x86 Westmere (alias configured by machine type)
x86 Westmere-IBRS (alias of Westmere-v2)
x86 Westmere-v1 Westmere E56xx/L56xx/X56xx (Nehalem-C)
x86 Westmere-v2 Westmere E56xx/L56xx/X56xx (IBRS update)
x86 athlon (alias configured by machine type)
x86 athlon-v1 QEMU Virtual CPU version 2.5+
x86 core2duo (alias configured by machine type)
x86 core2duo-v1 Intel(R) Core(TM)2 Duo CPU T7700 @ 2.40GHz
x86 coreduo (alias configured by machine type)
x86 coreduo-v1 Genuine Intel(R) CPU T2600 @ 2.16GHz
x86 kvm32 (alias configured by machine type)
x86 kvm32-v1 Common 32-bit KVM processor
x86 kvm64 (alias configured by machine type)
x86 kvm64-v1 Common KVM processor
x86 n270 (alias configured by machine type)
x86 n270-v1 Intel(R) Atom(TM) CPU N270 @ 1.60GHz
x86 pentium (alias configured by machine type)
x86 pentium-v1
x86 pentium2 (alias configured by machine type)
x86 pentium2-v1
x86 pentium3 (alias configured by machine type)
x86 pentium3-v1
x86 phenom (alias configured by machine type)
x86 phenom-v1 AMD Phenom(tm) 9550 Quad-Core Processor
x86 qemu32 (alias configured by machine type)
x86 qemu32-v1 QEMU Virtual CPU version 2.5+
x86 qemu64 (alias configured by machine type)
x86 qemu64-v1 QEMU Virtual CPU version 2.5+
x86 base base CPU model type with no features enabled
x86 host KVM processor with all supported host features
x86 max Enables all features supported by the accelerator in the current host
Recognized CPUID flags:
3dnow 3dnowext 3dnowprefetch abm ace2 ace2-en acpi adx aes amd-no-ssb
amd-ssbd amd-stibp apic arat arch-capabilities avic avx avx2
avx512-4fmaps avx512-4vnniw avx512-bf16 avx512-vpopcntdq avx512bitalg
avx512bw avx512cd avx512dq avx512er avx512f avx512ifma avx512pf
avx512vbmi avx512vbmi2 avx512vl avx512vnni bmi1 bmi2 cid cldemote clflush
clflushopt clwb clzero cmov cmp-legacy core-capability cr8legacy cx16 cx8
dca de decodeassists ds ds-cpl dtes64 erms est extapic f16c flushbyasid
fma fma4 fpu fsgsbase fsrm fxsr fxsr-opt gfni hle ht hypervisor ia64 ibpb
ibrs ibrs-all ibs intel-pt invpcid invtsc kvm-asyncpf kvm-hint-dedicated
kvm-mmu kvm-nopiodelay kvm-poll-control kvm-pv-eoi kvm-pv-ipi
kvm-pv-sched-yield kvm-pv-tlb-flush kvm-pv-unhalt kvm-steal-time kvmclock
kvmclock kvmclock-stable-bit la57 lahf-lm lbrv lm lwp mca mce md-clear
mds-no misalignsse mmx mmxext monitor movbe movdir64b movdiri mpx msr
mtrr nodeid-msr npt nrip-save nx osvw pae pat pause-filter pbe pcid
pclmulqdq pcommit pdcm pdpe1gb perfctr-core perfctr-nb pfthreshold pge
phe phe-en pku pmm pmm-en pn pni popcnt pschange-mc-no pse pse36 rdctl-no
rdpid rdrand rdseed rdtscp rsba rtm sep sha-ni skinit skip-l1dfl-vmentry
smap smep smx spec-ctrl split-lock-detect ss ssb-no ssbd sse sse2 sse4.1
sse4.2 sse4a ssse3 stibp svm svm-lock svme-addr-chk syscall taa-no tbm
tce tm tm2 topoext tsc tsc-adjust tsc-deadline tsc-scale tsx-ctrl umip
v-vmsave-vmload vaes vgif virt-ssbd vmcb-clean vme vmx vmx-activity-hlt
vmx-activity-shutdown vmx-activity-wait-sipi vmx-apicv-register
vmx-apicv-vid vmx-apicv-x2apic vmx-apicv-xapic vmx-cr3-load-noexit
vmx-cr3-store-noexit vmx-cr8-load-exit vmx-cr8-store-exit vmx-desc-exit
vmx-encls-exit vmx-entry-ia32e-mode vmx-entry-load-bndcfgs
vmx-entry-load-efer vmx-entry-load-pat vmx-entry-load-perf-global-ctrl
vmx-entry-load-rtit-ctl vmx-entry-noload-debugctl vmx-ept vmx-ept-1gb
vmx-ept-2mb vmx-ept-advanced-exitinfo vmx-ept-execonly vmx-eptad
vmx-eptp-switching vmx-exit-ack-intr vmx-exit-clear-bndcfgs
vmx-exit-clear-rtit-ctl vmx-exit-load-efer vmx-exit-load-pat
vmx-exit-load-perf-global-ctrl vmx-exit-nosave-debugctl
vmx-exit-save-efer vmx-exit-save-pat vmx-exit-save-preemption-timer
vmx-flexpriority vmx-hlt-exit vmx-ins-outs vmx-intr-exit vmx-invept
vmx-invept-all-context vmx-invept-single-context
vmx-invept-single-context vmx-invept-single-context-noglobals
vmx-invlpg-exit vmx-invpcid-exit vmx-invvpid vmx-invvpid-all-context
vmx-invvpid-single-addr vmx-io-bitmap vmx-io-exit vmx-monitor-exit
vmx-movdr-exit vmx-msr-bitmap vmx-mtf vmx-mwait-exit vmx-nmi-exit
vmx-page-walk-4 vmx-page-walk-5 vmx-pause-exit vmx-ple vmx-pml
vmx-posted-intr vmx-preemption-timer vmx-rdpmc-exit vmx-rdrand-exit
vmx-rdseed-exit vmx-rdtsc-exit vmx-rdtscp-exit vmx-secondary-ctls
vmx-shadow-vmcs vmx-store-lma vmx-true-ctls vmx-tsc-offset
vmx-unrestricted-guest vmx-vintr-pending vmx-vmfunc
vmx-vmwrite-vmexit-fields vmx-vnmi vmx-vnmi-pending vmx-vpid
vmx-wbinvd-exit vmx-xsaves vmx-zero-len-inject vpclmulqdq waitpkg
wbnoinvd wdt x2apic xcrypt xcrypt-en xgetbv1 xop xsave xsavec xsaveerptr
xsaveopt xsaves xstore xstore-en xtpr
`
@sickcodes commented on GitHub (Jul 24, 2021):
I was able to reproduce this issue when the host does not have KVM.
This produces the halt reboot error, so yes this bug is related to CPUID_FLAGS, any ideas which one is the culprit and if it's required for all systems? I was going to do trial and error till I found it using
-e CPUID_FLAGS= \@sickcodes commented on GitHub (Jul 24, 2021):
For contrast and to emphasize the CPUID flag importance, this passes the start but seg faults right after.
@idzikovski commented on GitHub (Jul 24, 2021):
Nope, I am not that knowledgable. I am simply trying to run this image using the instructions in the readme. My goal is to do a proof of concept for a mac build agent for CI/CD running in docker
@sickcodes commented on GitHub (Jul 24, 2021):
Using
-e CPU=host \instead of Penryn may work.I am able to replicate the exact error that @3xr displayed above and it's only occurring when qemu is unallowed to use /dev/kvm or when it's not passed in on my host or when I remove all the CPUID flags by setting the variable to nothing.
Can you confirm it's doing it with the pure vanilla run command?
@idzikovski commented on GitHub (Jul 26, 2021):
With this command I didn't get the panic but I encountered the issue where the mouse gets stuck in the screen where you can choose between disk utility and reinstall
@sickcodes commented on GitHub (Jul 28, 2021):
Mouse is unusable during the first screen which is normal, buy you can use the arrow keys
@3xr commented on GitHub (Aug 18, 2021):
Thanks for the remarks!
Actually it seems that it may be linked that the
--userns-remap=defaultoption is enabled on my Docker setup, and that thedockremapuser which is created automatically by Docker in such conditions was not added to thekvmorlibvirtgroups.I have tried to add the
dockremapuser to thekvmgroup, but I can't download a truckload of container images to re-test it right now as I'm on a countryside connection.@sickcodes commented on GitHub (Oct 31, 2021):
Note with Thinkpads the following from https://dortania.github.io/docs/latest/Configuration.html
@sickcodes commented on GitHub (Nov 30, 2021):
Possibly related https://github.com/reenigneorcim/SurPlus
@murilopereirame commented on GitHub (Oct 30, 2023):
Using the flag
-e CPU=hostsolves the problem of corecrypto but maybe throw a permission error related to KVM, not big deal, just adjust the permissions of /dev/kvm and re-run the command@JAicewizard commented on GitHub (Jun 1, 2024):
See https://github.com/sickcodes/Docker-OSX/issues/380#issuecomment-2143425540, maybe also resolves this issue. Also running a thinkpad bug a t14 amd gen 1, for me a bios update did it.