Darkmatter in Cyberspace

In the Economist article The world’s waste problem is growing fast , Mr. Oliver did an excellent job as a journalist: digging and gathering truth, putting them on the table, so everyone have to face it, think about it and try to fix it in different ways. The article is an introduction to Mr. Oliver’s book, Wasteland. In the book he wrote down what he saw in some special places, where he called wasteland, to disclose some shady corners in global and local economic network, such as plastic pollution, toxic colonialism, planned obsolescence, etc. Plastic pollution is the nearest problem related with our daily life. In China years ago the plastic bags are free in super markets and grocery stores. Then the government (at least in Beijing) enacted a law about mandatory paid plastic bags. From then on there’s a charge for plastic bags in super markets, while free in most small grocery stores. People are more and more aware of the plastic pollutions. However, at least by my observation, the ...

A couple of days ago I read 2 articles about LLM (large language model) and ChatGPT. The first one is How ChatGPT and Other LLMs Work—and Where They Could Go Next on Wired. It mainly discussed the following topic: The main idea behind the LLM: with some user-provided words (usually a quesion), LLM choose the most suitable following without understanding the meaning of the question and the sentence it emits. Technical lineage: neural network to transformer, then to LLM. How human participation promotes the model quality. Here the article introduced the basic idea of RLHF (reinforcement learning on human feedback), the silver bullet of ChatGPT. The second article How ChatGPT actually works contains far more technical detals. It explains why ChatGPT-4 is superior than its precursors (better alignment), the detailed steps of RLHF, how to evaluate the performance of a LLM, and finally the shortcomings of the methodology used in current model training. After reading above articles, ...