nasm output format should default to elfx32 for the x32 ABI on x86_64
The NASM assembler needs to output to a special ELF format (elfx32) when compiling for the x32 ABI of x86_64.
The original bugreport is at https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1013630:
Currently svt-av1 FTBFS on x32 with the following error:
[ 11%] Building ASM_NASM object Source/Lib/Common/ASM_SSE2/CMakeFiles/COMMON_ASM_SSE2.dir/intrapred_sse2.asm.o
cd /<<PKGBUILDDIR>>/obj-x86_64-linux-gnux32/Source/Lib/Common/ASM_SSE2 && /usr/bin/yasm -DARCH_X86_64=1 -DEN_AVX512_SUPPORT=0 -DEXCLUDE_HASH=0 -DREPRODUCIBLE_BUILDS=1 -DSAFECLIB_STR_NULL_SLACK=1 -D_FORTIFY_SOURCE=2 -I/<<PKGBUILDDIR>>/. -I/<<PKGBUILDDIR>>/Source/API -I/<<PKGBUILDDIR>>/Source/Lib/Common/Codec -I/<<PKGBUILDDIR>>/Source/Lib/Common/C_DEFAULT -I/<<PKGBUILDDIR>>/Source/Lib/Common/ASM_SSE2 -DUNIX64 -f elf -o CMakeFiles/COMMON_ASM_SSE2.dir/intrapred_sse2.asm.o /<<PKGBUILDDIR>>/Source/Lib/Common/ASM_SSE2/intrapred_sse2.asm
/<<PKGBUILDDIR>>/Source/Lib/Common/ASM_SSE2/intrapred_sse2.asm:57: warning: `rcx' is a register in 64-bit mode
/<<PKGBUILDDIR>>/Source/Lib/Common/ASM_SSE2/intrapred_sse2.asm:57: error: undefined symbol `rcx' (first use)
/<<PKGBUILDDIR>>/Source/Lib/Common/ASM_SSE2/intrapred_sse2.asm:57: error: (Each undefined symbol is reported only once.)
/<<PKGBUILDDIR>>/Source/Lib/Common/ASM_SSE2/intrapred_sse2.asm:58: warning: `rdx' is a register in 64-bit mode
/<<PKGBUILDDIR>>/Source/Lib/Common/ASM_SSE2/intrapred_sse2.asm:58: error: undefined symbol `rdx' (first use)
/<<PKGBUILDDIR>>/Source/Lib/Common/ASM_SSE2/intrapred_sse2.asm:66: warning: `rdi' is a register in 64-bit mode
/<<PKGBUILDDIR>>/Source/Lib/Common/ASM_SSE2/intrapred_sse2.asm:66: error: undefined symbol `rdi' (first use)
/<<PKGBUILDDIR>>/Source/Lib/Common/ASM_SSE2/intrapred_sse2.asm:67: warning: `rdi' is a register in 64-bit mode
/<<PKGBUILDDIR>>/Source/Lib/Common/ASM_SSE2/intrapred_sse2.asm:67: warning: `rsi' is a register in 64-bit mode
/<<PKGBUILDDIR>>/Source/Lib/Common/ASM_SSE2/intrapred_sse2.asm:67: error: undefined symbol `rsi' (first use)
/<<PKGBUILDDIR>>/Source/Lib/Common/ASM_SSE2/intrapred_sse2.asm:68: warning: `rdi' is a register in 64-bit mode
/<<PKGBUILDDIR>>/Source/Lib/Common/ASM_SSE2/intrapred_sse2.asm:68: warning: `rdi' is a register in 64-bit mode
/<<PKGBUILDDIR>>/Source/Lib/Common/ASM_SSE2/intrapred_sse2.asm:68: warning: `rsi' is a register in 64-bit mode
/<<PKGBUILDDIR>>/Source/Lib/Common/ASM_SSE2/intrapred_sse2.asm:69: warning: `rdi' is a register in 64-bit mode
/<<PKGBUILDDIR>>/Source/Lib/Common/ASM_SSE2/intrapred_sse2.asm:70: warning: `rdi' is a register in 64-bit mode
/<<PKGBUILDDIR>>/Source/Lib/Common/ASM_SSE2/intrapred_sse2.asm:70: warning: `rsi' is a register in 64-bit mode
/<<PKGBUILDDIR>>/Source/Lib/Common/ASM_SSE2/intrapred_sse2.asm:81: warning: `rcx' is a register in 64-bit mode
/<<PKGBUILDDIR>>/Source/Lib/Common/ASM_SSE2/intrapred_sse2.asm:87: warning: `rdi' is a register in 64-bit mode
/<<PKGBUILDDIR>>/Source/Lib/Common/ASM_SSE2/intrapred_sse2.asm:88: warning: `rdi' is a register in 64-bit mode
/<<PKGBUILDDIR>>/Source/Lib/Common/ASM_SSE2/intrapred_sse2.asm:88: warning: `rsi' is a register in 64-bit mode
[...]
As you can see, yasm is called with "-f elf" and not "-f elfx32"