I’m losing my mind over emacs as a python IDE. It seems that no matter which LSP concept I attempt, the experience is suboptimal. At best it is riddled with odd issues, and at worst almost bordering unusable.

I work as a python developer, and I work on quite large code bases. The LSP experience is essential for me given the complexity, but I’m at the point of giving up, even though I’ve been an emacs user for the better part of a decade.

I use doom emacs, by the way, but I’m quite happy to try anything to solve this. I have considered to build an emacs config from scratch again to see if I can narrow down the experience, but right now I have not invested the time. But again, considering how frustrated and desparate I am, everything is on the table.

Let me describe what I need and what I’ve tried.

My requirements

I need my lsp to understand my environment in order to load in a custom PYTHONPATH variable. This I handle with a .envrc file and the emacs module envrc. This has worked without any problems so far.

For the actual LSP, I use python-lsp-server but I am not at all religious about it.

I run mypy on my codebase, and I’d like to use ruff for checking/formating, but I’ve used the other tools (pycodestyle/black) for now.

lsp-mode

The lsp-mode is the one that is closest to being good, had it not been for the abysmal behavior over time. It seems that some memory leaking happens, and over time it becomes sluggish, sometimes blocking emacs for multiple seconds.

eglot

Eglot seems a lot faster, but has functional problems. I’ve often had to restart eglot when opening new python modules. I’ve tried to use eglot for months now, accepting the quirks to gain the speed, but the restart behavior is annoying, and I often get weird errors such as error in process filter: Wrong type argument: plistp, [] repeatedly until I restart eglot again. No luck debugging that error so far either.

I’m also not super excited about the .dir-locals.el approach to customization, but should the other issues resolve, I’m willing to figure that one out.

lsp-bridge

I was quite excited to hear about a third option that was super fast. However, I’ve not manage to get it to work with the envrc package, so I’ve parked that one.

So, I turn to you, kind strangers on the internet. Do you have a well-functioning python LSP setup in emacs? Have you experienced some of the issues that I have described and did you manage to solve it?

Do you perhaps have a described approach to building a great python setup with emacs? Please help me!

  • yyoncho@alien.topBEnglish
    1·
    1 year ago

    lsp-mode The lsp-mode is the one that is closest to being good, had it not been for the abysmal behavior over time. It seems that some memory leaking happens, and over time it becomes sluggish, sometimes blocking emacs for multiple seconds.

    We cannot fix issues like that if we don’t know about them. As a side note, using Emacs LSP fork(https://github.com/emacs-lsp/emacs) most likely will make the blocking go away.

    • JDRiverRun@alien.topBEnglish
      1·
      1 year ago

      Can you comment on what the future for this JSON-handling-in-a-separate-thread emacs fork looks like, currently? Any hope of upstreaming some perhaps more general version of limited multi-threaded support of this kind? Other “downstream” forks (like emacs-mac) miss the benefits.

      • CJ6_@alien.topBEnglish
        1·
        1 year ago

        It’s unclear to me whether it should be expected to work on macOS anyway. The README does say “Unix”, but there’s an open issue about crashing on macOS.

        Maybe someone here has a personal experience they can share.

  • MoerliYT@alien.topBEnglish
    1·
    1 year ago

    I’ve had my fair issues with large code bases too. I used doom emacs as well but had some time a couple of month so I configured emacs from scratch. Together with that I switched to lsp-bridge. I had some issues myself. Some of them I could fix myself and push upstream others I had to fix alongside the developers. Although most of the devs working on the lsp-bridge repo I have interacted with aren’t native English speakers or aren’t perfectly fluent in it, helped me with all the problems I had until now. I surges you give lsp-bridge another try and open issues for the problems you have.

  • tikhonjelvis@alien.topBEnglish
    1·
    1 year ago

    Have you tried to see if using direnv.el instead of the envrc package fixes the lsp-bridge issues? If it does, it’s probably an issue related to how envrc interacts with temp buffers. From the envrc-mode README:

    …it’s possible you’ve found code that runs a process in a temp buffer and neglects to propagate your environment to that buffer before doing so.

    A couple of common Emacs commands that suffer from this defect are also patched directly via advice in envrc.el — shell-command-to-string is a prominent example!

    The inheritenv package was designed to handle this case in general.

    I haven’t tried any of this myself, so this is just speculation.

  • Nippurdelagash@alien.topBEnglish
    1·
    1 year ago

    I had a similar issue with lsp-mode where it started to lag everything in python mode. Turns out that if you have setup a proxy server with something like:

    (setq socks-noproxy '("127.0.0.1"))
(setq socks-server '("Default server" "127.0.0.1" 8010 5))

    lsp ends up routing the client-server calls through your proxy.

  • ClerkOfCopmanhurst@alien.topBEnglish
    1·
    1 year ago

    Even if things weren’t broken now, they will be in a couple months. If you take the fragility of emacs to its logical conclusion as I have, you end up writing emacs instead of writing software.

    Diddling python is but an intermediary phase to non-coding management, but diddle you must to prove you’ve been in the trenches. Alas many a doughboy fell under emacs’s hypnotic spell and never poked their heads out to see the war’s been long over.

  • terminal_prognosis@alien.topBEnglish
    1·
    1 year ago

    Since I work on large Python projects, use direnv to manage my environment, and lsp-mode, I thought I might be able to help, but reading your long message I can’t find any specific issue except it sounds like your LSP server is misbehaving.

    I do not get lagging or memory leaking in Emacs from using it. What process is getting large? Emacs or the LSP server? You don’t give us any clues.

    • dubcroster@alien.topOPBEnglish
      1·
      1 year ago

      Thanks for replying!

      Yeah, I was very loose on details here, and this is probably more because of my frustration than anything.

      I set up my private laptop with Eglot yesterday with a single small python project from scratch and had some of the same issues.

      So I don’t think my issue is so much related to my code base, but more how the default setup for both lsp-mode and eglot is either out of the box with emacs or out of the box with doom emacs.

      If you are indeed finding good success with just lsp-mode and work with large Python projects, that’s extremely valuable for me to hear. It means to me that this is fixable and I should just spend the time.

      • terminal_prognosis@alien.topBEnglish
        1·
        1 year ago

        Though I am therefore no use in solving your problems, if I’ve avoided them. I’m using pyright, and ruff. I am using the lsp-pyright package, and it looks like I configured ruff with:

          (lsp-register-client
   (make-lsp-client
    :new-connection (lsp-stdio-connection (lambda () (list "ruff-lsp")))
    :activation-fn (lsp-activate-on "python")
    :add-on? t
    :server-id 'ruff))
  • carnivorousdrew@alien.topBEnglish
    1·
    1 year ago

    Hi, I work with big codebases as well and yes, no lsp client or server works well for python when you are dealing with a big codebase with a bunch of dependencies. It will slow down and get into bottlenecks easily. lsp-bridge provides some advantages to that but I don’t see it as a real solution. If your complaints are mainly about speed, well then there is no fix to that unless you work with smaller projects.

    If your complaints are about setting up a project and configuring the virtual environment and lsp’s to pick it up etc… then I was working on a package called pyconf to automate much of that, but I haven’t picked it up in a while…

  • LongjumpingAd7260@alien.topBEnglish
    1·
    1 year ago

    vanilla emacs 29 (eglot built-in) + eglot + poetry + pyright.
    smooth, fast, stable.

    You should config eglot to run the lsp (pyright in my case) through “poetry run”. So poetry will handle the environment for you.
    I switched from doom emacs to vanilla emacs recently. No regrets.

  • noooit@alien.topBEnglish
    1·
    1 year ago

    jedi-language-server is the only one that works ok. The choice of the language server is really important, so be careful from next time.
    Eglot and etc is merely a json-rpc client.