Silk: Open-source cooperative fiber scheduler

(github.com)

77 points | by animetyan 3 days ago

12 comments

  • audidude 0 minutes ago
    I've been doing this with something very similar in GNOME for years now. It also does io_uring, profiler integration, and makes fibers implemented as futures themselves.

    https://github.com/GNOME/libdex/

  • nasretdinov 6 hours ago
    Looks really interesting. I assume this suggests that ClickHouse is going to gradually switch to using this library for network and I/O, thus addressing the main weakness (in my mind) of C++ thread-per-connection servers, which is, they (surprisingly!) create too many threads and can't really handle more than, say, a thousand active connections at the same time. It mostly matters for async INSERTs in this case of course, not for SELECTs, although generally it applies to both.
  • enduku 4 hours ago
    The `Cilk` angle is interesting. There’s still room for small runtimes focused just on fork/join recursion.

    I’ve been working on one for C: https://github.com/xtellect/cactus

    It’s narrower than Silk/SeaStar: continuation stealing for CPU-bound recursive code, not a general async I/O fiber runtime.

  • jeffreygoesto 5 hours ago
    Is this comparable to Sea star [0]?

    [0] https://github.com/scylladb/seastar

    • nepthar 1 hour ago
      I just took a look a both (thanks for the pointer to seastar! cool project). I'd say "sort of but not exactly". It seems like two approaches to solving the same problem.

      Silk seems like mostly a scheduler that uses C++'s native coroutines. Seastar is a future-based framework. Both contain a scheduler and enable async coding patterns.

  • bbkane 2 hours ago
    Looks like its using MinIO for the S3 benchmark, which is now archived. I wonder what they'll switch to (if anything)
  • feverzsj 5 hours ago
    Seems not exception safe when task switching during unwind.
  • znnajdla 1 hour ago
    those who don't learn Erlang/Elixir are bound to reinvent the Erlang VM and OTP model.
    • Imustaskforhelp 1 hour ago
      Honestly, erlang was designed for telephone use cases and for high concurrency, erlang as an idea is fundamentally really awesome.

      And elixir on top of erlang really makes the syntax easier to use. Gleam is interesting too!

      Depending upon the amount of concurrency needed, erlang can be absolutely good for the situation.

      The only gripe that I might have seen online with Elixir might be that it is functional language (and I think has its own way of doing things) and a lot of the times people start with other languages which can scale to only a certain point.

      Either way, Erlang VM/OTP model does feel really cool to me. Fun fact: but Erlang was a virtual machine before it was cool (Java)

      I definitely wish to learn elixir someday (don't quote me on that though :-D), just because of how unique Erlang/Elixir are compared to many languages in such sense of scalability.

      Although not a 1:1 Apples but I also wish to know more when golang is more fit as compared to elixir as currently, I usually prefer golang as a swiss knife with its decent stdlib,cross-compilation,portability features.

  • yvdriess 5 hours ago
    Play on Cilk?
  • idlepig 5 hours ago
    [flagged]
  • embirdating 5 hours ago
    [dead]
  • fleahunter 2 hours ago
    [flagged]
  • SeattleAntifa 2 hours ago
    [dead]