Reped: A REPL-Centered Workflow for Data Science

Nowadays workflow for data science is editor-based. No matter vim, emacs, or IDE are all central parts when data scientists develop algorithms, or engineers develop new applications. People write scripts in the editors, copy them into the interactive shell (often called REPL, an acronym for Read, Evaluate, Print Loop), and execute the codes there. For better interactivity, people invent a bunch of tools, such as linter, auto-completion, and syntax highlighting, until LSP (language server protocol). However, all these tools are not free lunches. The editors are becoming more and more swollen and clumsy. We spend time and (sometimes) money to make ourselves skillful in VS Code or PyCharm. However, when we enjoy the fancy functions provided by these tools, we are bound with them, like an assembly line worker, rather than a free-minded, joyful coder. Some say, even though this is a fair argument in some way, most programmers nowadays do use IDEs. Yes, but with a historical reason. Today developing toolchains is dominated by software development, which is divided roughly into two camps: statically typed languages and dynamically typed languages. The former is much older than the latter (the LISP family has much fewer practitioners than the C family after all). So most dynamic programmers adopted the developing toolchains of static programming.

Last Friday morning on the way to the office, an idea hit me like an enlightenment: why not put REPL to the center of the stage, while keeping the editors playing their original role: save codes into files and disks? For the essential role of REPL in this settings, I call it reped (REPl + EDitor).

I'm really excited about this idea and spent two more days making it real. This toolchain is composed of 4 parts: an editor (of course) neovim. A REPL-editor communication tool "iron", written in Lua, initializes a REPL in neovim's built-in terminal, based on the language of the script edited, by far there are xonsh for Python, radian for R, and Julia with OhMyREPL package. The third part is a configuration system composed of Lua script, providing different functions and parameters for managed languages. The last part is a Python script to start the editor. It provides help messages, puts appropriate configuration files in the proper position, and set parameters that can't be determined beforehand.

After several days of usage, I must say its experiences outperformed any IDE I've used. First, it's blazing fast thanks to Lua's performance. And you know, timing really counts. Second, it's completely keyboard controlled. Your hands needn't leave your keyboard, turning coding into an exciting journey, like playing in the FC game console, back in childhood. Third, with the help of Lua and Python, you can control your system deep into every byte. It's your tool made by yourself.

Comments

Post a Comment

Popular posts from this blog

2023: On the Road

There're so many dramas happened in the past 3 years. People say there'll be more in the coming 2023. Basically, I agree with this. But no matter what happens in the physical world, we need to concentrate on something really important and interesting. In the past 2022, my most profound change was, that I started thinking seriously about the future living style. My family and friends mostly live like plants, fixed by our roots, and change ourselves to adapt to the environment. We are where we are, proud of its history and culture, and contained by it. It'll not be the case anymore. In 2022, I learned, read, listened, wrote, and spoke in second languages, especially English and Japanese. They bring me a new world. I also read several books on Chinese and middle east histories. The world before me is fusing as a whole sphere. China and Chinese people are a part, and the most familiar part for me, the starting point of my exploration. It gives me enough confidence as a foreig...
Read more

Yet another advice to kids

Sharon posted some advice to the youth (although she herself is young). She suggested being independent, with no fear of failure, which is excellent advice. I'd like to give some more "concrete" tips to kids or myself in childhood (if I have the chance to tell that me). First things first, the following suggestions can't make you more competitive than your peers or perform better in exams. or win the girl make you fascinated. They make your everyday life more exciting and fancy even if you're not rich or powerful. They're neither expensive (so you can achieve these goals even if not born into a rich family) nor painful. What you need is to know them, and keep doing them. OK, here we go. 1. Find a sport, enjoy it, and be good at it. Sport is a way to talk with your body, with the physical world, express yourself, and communicate with people. 2. Be honest, especially to yourself. Living honestly in Chinese society looks stupid and clumsy. Never mind. Being hones...
Read more

The Joy of Reading in Natural Light

Nowadays I read books in paperbacks much less than ebooks. Not because I don't like reading paper books, but after reading for so many years on screen, my eyes are very sensitive to the brightness of reading materials. It's very hard for me to read characters when the material is not bright enough. Unfortunately, the light in my living room is not bright enough for me. This morning it was cloudy, while in the afternoon it got clear, and the blue sky and white clouds appeared. So I moved the desk near the window. To my surprise, the characters on paper books are much more clear and more comfortable to read than in artificial light. Maybe such a simple fact is not worthy to mention. But to me, a programmer spent too much time with the keyboard and screen, this reminds me of the memories of childhood, reading a book, or writing homework in the yard. The characters blurred at sunset. That's when we were going to have super. The window of my living room is east-forwarding, so to...
Read more