Comment by amno

> Is this at the codegen layer?

On implementation level, it is a codegen layer. It uses a system of macros to generate instructions from a database of instructions. The database is specified manually:

https://github.com/sbcl/sbcl/tree/master/contrib/sb-simd/cod...

At compile-time, they are converted into "VOPs", i.e. intrinsic functions, which are used by the compiler to emit the actual machine instructions.

> can it auto-vectorize or anything like that?

Unfortunately, it can't.

> are these intrinsics you have to explicitly ask for?

Yes, more like higher-level intrinsics. You get quite some automation, but you are requesting manually what you need. More like a DSL, than pure intrinsics. This is how you can use it (as an example):

(defun count-lines-and-words-ascii (sap size ws-init-state) (declare (type fixnum size) (type (unsigned-byte 8) ws-init-state) (type sb-sys:system-area-pointer sap) (optimize (speed 3) (safety 0))) (loop with loop-end of-type fixnum = (logandc2 size 127) for i of-type fixnum from 0 below loop-end by 128

    with 0x0  of-type u8.32 = (u8.32 #x00)
    with 0x20 of-type u8.32 = (u8.32 #x20)
    with 0x0A of-type u8.32 = (u8.32 #x0A)

    with wa of-type u64.4 = (u64.4 0)
    with la of-type u64.4 = (u64.4 0)

    with ws-prev of-type u8.32 = (u8.32 ws-init-state)

    for c1 = (u8.32-sap-ref sap (+ i 0))
    for c2 = (u8.32-sap-ref sap (+ i 32))
    for c3 = (u8.32-sap-ref sap (+ i 64))
    for c4 = (u8.32-sap-ref sap (+ i 96))

    do
       (flet ((process-chunk (curr prev)
                (let* ((ctrl  (u8.32-sat- (u8.32- curr 9) 4))
                       (ws (u8.32-or (u8.32= ctrl 0x0) (u8.32= curr 0x20)))

                       (ws-shift (u8.32-alignr ws (u8.32-permute128 prev ws #x21) 15))
                       (wmask    (u8.32-andc1 ws ws-shift))

                       (lmask (u8.32= curr 0x0A)))
                  (values wmask lmask ws))))

         (multiple-value-bind (wm lm prev) (process-chunk c1 ws-prev)
           (psetf wa (u64.4+ wa (u8.32-sad wm 0x0))
                  la (u64.4+ la (u8.32-sad lm 0x0))
                  ws-prev prev))
         (multiple-value-bind (wm lm prev) (process-chunk c2 ws-prev)
           (psetf wa (u64.4+ wa (u8.32-sad wm 0x0))
                  la (u64.4+ la (u8.32-sad lm 0x0))
                  ws-prev prev))
         (multiple-value-bind (wm lm prev) (process-chunk c3 ws-prev)
           (psetf wa (u64.4+ wa (u8.32-sad wm 0x0))
                  la (u64.4+ la (u8.32-sad lm 0x0))
                  ws-prev prev))
         (multiple-value-bind (wm lm prev) (process-chunk c4 ws-prev)
           (psetf wa (u64.4+ wa (u8.32-sad wm 0x0))
                  la (u64.4+ la (u8.32-sad lm 0x0))
                  ws-prev prev)))
    finally
       (return
         (loop for j from loop-end below size

               with words of-type fixnum    = (sum-lanes wa)
               with lines of-type fixnum    = (sum-lanes la)
               with prev-ws of-type boolean = (logbitp 31 (u8.32-movemask ws-prev))
               with tlines of-type fixnum   = 0
               with twords of-type fixnum   = 0
               for byte of-type fixnum      = (sb-sys:sap-ref-8 sap j)
               for curr-ws of-type boolean  = (or (= byte 32) (<= 9 byte 13))

               do
                  (when (= byte 10) (incf tlines))
                  (when (and (not curr-ws) prev-ws) (incf twords))
                  (setf prev-ws curr-ws)
               finally
                  (return (values (the fixnum (+ lines tlines))
                                  (the fixnum (+ words twords))
                                  nil))))))