Swift/Ninja: compile_commands.json mixes path-separators on Windows

cc @etcwilde @compnerd

I've labeled this a Swift issue, but the escaping logic looks similar to the general cmNinjaTargetGenerator::ExportObjectCompileCommand . I haven't verified if the same behavior is exhibited here.

P.S. thanks for getting compile_commands.json generating Evan! After manually patching up these path separators, I'm getting functional LSP for Swift on Windows!

yeah... I was not sure how many levels of escaping were needed. Thanks for reporting this. Great to hear that it's close to working though.

Actually, @z2oh, since you have a windows box with CMake, can you generate me a compile_commands.json file from a project with a C file and a Swift file so that I have something to compare the commands with? Thanks.

[
{
  "directory": "C:/workspace/example/swift-cmake-examples/3_bidirectional_cxx_interop/build",
  "command": "C:\\Users\\JeremyDay\\AppData\\Local\\Programs\\Swift\\Toolchains\\0.0.0+Asserts\\usr\\bin\\swiftc.exe -j 20 -num-threads 20 -c -Onone -g -libc MDd -incremental -output-file-map src\\CMakeFiles\\a_swift.dir\\Debug\\output-file-map.json C:\\workspace\\example\\swift-cmake-examples\\3_bidirectional_cxx_interop\\src\\a.swift",
  "file": "C:\\workspace\\example\\swift-cmake-examples\\3_bidirectional_cxx_interop\\src\\a.swift",
  "output": "src/CMakeFiles/a_swift.dir/a.swift.obj"
},
{
  "directory": "C:/workspace/example/swift-cmake-examples/3_bidirectional_cxx_interop/build",
  "command": "C:\\Users\\JeremyDay\\AppData\\Local\\Programs\\Swift\\Toolchains\\0.0.0+Asserts\\usr\\bin\\clang.exe -O0 -D_DEBUG -D_DLL -D_MT -Xclang --dependent-lib=msvcrtd -g -Xclang -gcodeview -o src\\CMakeFiles\\a_c.dir\\a.c.obj -c C:\\workspace\\example\\swift-cmake-examples\\3_bidirectional_cxx_interop\\src\\a.c",
  "file": "C:\\workspace\\example\\swift-cmake-examples\\3_bidirectional_cxx_interop\\src\\a.c",
  "output": "src\\CMakeFiles\\a_c.dir\\a.c.obj"
}
]

Here you go! Looks like the output value is escaped differently between C/Swift, but both mix the path separators.

Thanks for this. From the looks of it, it's just output that has the wrong file path separator? Since SourceKit-LSP also supports C/C++, are you able see if sourcekit-lsp fails to handle the C file escaping on Windows? There may just be bugs to fix on all fronts.

mentioned in merge request !9150 (merged)

@z2oh, if you can try out !9150 (merged), I think this should fix at least the CMake portion of the bug, fixing the output field filepath separators. I don't have a Windows box readily available so if able, if you could checkout that MR and verify that the swift output matches the path style for the C output, that would be great. Thanks.

With !9150 (merged) I get:

[
{
  "directory": "C:/workspace/example/swift-cmake-examples/3_bidirectional_cxx_interop/build",
  "command": "C:\\Users\\JeremyDay\\AppData\\Local\\Programs\\Swift\\Toolchains\\0.0.0+Asserts\\usr\\bin\\swiftc.exe -j 20 -num-threads 20 -c -Onone -g -libc MDd -incremental -output-file-map src\\CMakeFiles\\a_swift.dir\\Debug\\output-file-map.json C:\\workspace\\example\\swift-cmake-examples\\3_bidirectional_cxx_interop\\src\\a.swift",
  "file": "C:\\workspace\\example\\swift-cmake-examples\\3_bidirectional_cxx_interop\\src\\a.swift",
  "output": "src\\CMakeFiles\\a_swift.dir\\a.swift.obj"
},
{
  "directory": "C:/workspace/example/swift-cmake-examples/3_bidirectional_cxx_interop/build",
  "command": "C:\\Users\\JeremyDay\\AppData\\Local\\Programs\\Swift\\Toolchains\\0.0.0+Asserts\\usr\\bin\\clang.exe -O0 -D_DEBUG -D_DLL -D_MT -Xclang --dependent-lib=msvcrtd -g -Xclang -gcodeview -o src\\CMakeFiles\\a_c.dir\\a.c.obj -c C:\\workspace\\example\\swift-cmake-examples\\3_bidirectional_cxx_interop\\src\\a.c",
  "file": "C:\\workspace\\example\\swift-cmake-examples\\3_bidirectional_cxx_interop\\src\\a.c",
  "output": "src\\CMakeFiles\\a_c.dir\\a.c.obj"
}
]

So parity between Swift and C! I'm still not sure about directory , I would think that should be escaped too (will do some more investigation), but after !9150 (merged) the issue is not isolated to Swift.

I've gone ahead and asked on the LLVM discord what the filepaths should look like in the directory field. ninja -t compdb emits the expected Windows-native JSON-escaped path so I suspect CMake might be wrong here, though AFAIK, folks haven't complained about it not working. @compnerd, have you had issues with clangd not consuming the cmake-generated compiled_commands.json for C/C++ on Windows?

Other than that discrepancy, I think the number of escapes in the escape sequence sounds like a bug in sourcekit-lsp.

Other than that discrepancy, I think the number of escapes in the escape sequence sounds like a bug in sourcekit-lsp.

Agreed! I've filed this upstream: https://github.com/apple/sourcekit-lsp/issues/1020

I'm not sure what Windows tooling expects here; I was unable to find a solid answer when I was searching for it. I found a Youtube video demo of a Visual Studio extension which shows forward path separators being generated on Windows, but I don't think that's a reliable source.

Yep, I saw the radar float past.

As far as I am aware, they're a clang/LLVM thing for LSP servers, though there are/were other formats for the days when projects only had a single language (e.g. compile_flags.txt). The closest thing to a spec that I'm aware of for the JSON form is https://clang.llvm.org/docs/JSONCompilationDatabase.html. The examples for which are all unix/linux-style paths. The only thing that actually seems to call out that paths must be platform escaped is the command field.

command: The compile command as a single shell-escaped string. Arguments may be shell quoted and escaped following platform conventions, with ‘"’ and ‘\’ being the only special characters. Shell expansion is not supported.

There is more info in the clangd docs; https://clangd.llvm.org/design/compile-commands.

closed with merge request !9150 (merged)

Swift/Ninja: compile_commands.json mixes path-separators on Windows

Child items ...

Activity