【Solana Complete Review (Part 1)】History of Solana / Basic Overview and Technical Features such as PoH / Challenges / Overview and Challenges of the Expected Technology Firedancer / @solana
In-depth explanation of Solana as of the first half of 2024.
Good morning.
Mitsui from web3 researcher.
Today's article is contributed by CryptCrypt..Entitled "Solana Complete Review," the article provides a thorough explanation of Solana's history, technical features, and ecosystem.
Please read to the end!
1. History of Solana
2. Solana Basic Overview
4. Other Solana Innovations
5. Solana's challenges
6. Firedancer
7. Can Firedancer really solve Solana's problems?
What makes Solana unique?
While there are many blockchain projects out there with different approaches, Solana's focus on the fundamental elements of the blockchain is very unique in its approach.In particular, Solana's focus is on time.After all, Solana has proven that introducing a decentralized clock to the blockchain is more efficient than anyone could have imagined.
1. history of the birth of Solana
Anatoly Yakovenko is the founder of the Solana blockchain, and his background as a technologist played a major role in the launch of Solana.
A native of Ukraine, Yakovenko earned a degree in computer science from the University of Illinois at Urbana-Champaign, U.S.A., and worked for several technology companies, including Qualcomm, where he spent 12 years developing basic smartphone technology.He then spent 12 years developing the underlying technology for smartphones at Qualcomm, followed by a period of innovation at Mesosphere, a distributed systems company, and Dropbox, a cloud storage company.These experiences provided an important foundation for his later transition to blockchain technology.
Yakovenko's interest in blockchain was not so great at first, but as he gained experience mining Bitcoin in parallel with his research in deep learning techniques, he began to realize the potential of blockchain.In particular, in 2017, he came up with Solana's core concept of a decentralized clock, or "Proof of History," on the cryptocurrency blockchain using Bitcoin's hash function SHA256, which he believed could be achieved without sacrificing security or decentralizationI was convinced that this could be done without sacrificing security or decentralization.
In late 2017, Yakovenko officially launched the project with his then colleagues at Qualcomm, Greg Fitzgerald, Steven Akridge, and Raj Gokal.That blockchain white papernamed after Solana Beach, California, was quietly released in November 2017.
2. Solana Basic Overview
Solana is a high-performance blockchain designed for the development of decentralized applications that implement smart contracts, and is sometimes referred to as an "Ethereum killer" because it attempts to solve the scalability problem that is considered Ethereum's weakness.
Solana designed the blockchain with a long-term vision in mind from the beginning of its development.This was born out of Yakovenko's experience at Qualcomm, where he saw the capabilities of telecommunications technology grow nearly twice as fast each year.As a result, it is still known for its ultra-fast transaction processing capabilities and low transaction fees in the fast-evolving blockchain landscape.
The underlying technology is the aforementioned Proof of History (PoH) technology.As will be explained in more detail later, it time-stamps each transaction on the blockchain, greatly improving the synchronization efficiency of the entire network.This method has the advantage of being faster and more energy efficient than traditional blockchain.
It allows Solana to process up to 65,000 transactions per second (TPS).The speed will be further increased with the implementation of Firedancer, described below.Thus, Solana facilitates user-friendly application development by offering low fees and 400 ms block time with high processing performance.
Incidentally, although Proof of History is part of the consensus process, the core mechanism for determining the block generation right holders is based on Proof of Stake (PoS).
This is a point that is easily misunderstood because various sources state that "Proof of History is adopted for the consensus mechanism," but it would be more accurate to think of Proof of History as only a core function to streamline consensus.It means that while using the basic Proof of Stake framework, the combination with Proof of History provides enhanced processing performance.
As such, Solana's architecture is inspired by the database technology used by Google and Microsoft, and is also influenced by Filecoin, a decentralized data storage project.
These unique ideas allow Solana to achieve much faster transaction speeds compared to competing chains such as Ethereum and Cardano.
In particular, the combination of Proof of History and Proof of Stake allows validators on the blockchain to vote on the timestamps of different blocks in the chain, making the chain somewhat decentralized, faster, and more secure3. Proof of History
3. Proof of History
Solana's Proof of History (PoH) is a technology that solves the problem of time synchronization in decentralized systems.Typically, blockchain technology has difficulty synchronizing exact time between different nodes.
In the case of Bitcoin, for example, approval of new blocks requires time-consuming mining, which is an indirect way of agreeing on time across the network.Proof of History, on the other hand, provides a direct approach to this problem.It records the exact time at which a transaction occurred and uses that information to establish the order of transactions.
PoH essentially acts as a "time recorder."This process can be thought of as a "digital time stamp generator" that continuously generates evidence of time from one piece of data to the next.
Specifically, starting with the initial data (or state), new data (hash values) are generated one after another based on that data.Each of these hashes is a link in a chain, as the next hash value is created based on the previous hash value.
One of the characteristics of this "chained data generation process" is that parallel processing is impossible.The reason for this is that the generation of each hash is dependent on the previous hash.In order to generate a new hash, the previous hash is needed, and the process must be processed one by one in sequence.This makes the overall computation very fast, but does not allow for multiple calculations to be performed in parallel.
This chained hash generation allows Solana to record the exact time of a transaction and process it quickly and efficiently accordingly.
The system is difficult to tamper with, and once data is added to the blockchain, it is virtually impossible to alter past data without affecting subsequent data.
In addition, the timestamp generated by PoH provides proof of the time at which a transaction was reliably made, thus preserving data integrity and transparency across the decentralized network.In other words, PoH is a pre-consensus mechanism and serves to streamline the subsequent consensus process.
4. other Solana innovations
In Solana, there are eight core features, including PoH, which we have already discussed.We have already discussed the PoH, but there are three other technologies that are of particular interest.(To learn more about each feature, check out this site.)
They are Turbine, Gulf Stream, and Sealevel.They are used in conjunction with Proof of History (PoH) to significantly improve Solana's performance and scalability.
As a prerequisite, Solana has a "leader node".The leader is responsible for generating new blocks4 blocks(1.6 seconds), and its role rotates every 4 blocks(1.6 seconds).
While a node is a leader, it packs as many transactions as possible into the four blocks it is generating and presents these blocks containing transactions to a group of related nodes, called a Solana cluster.These nodes validate the transactions by referencing the timestamps attached by the PoH and quickly pass the data to other related nodes on the network.
Leader selection is pseudo-random, but the amount of SOLs staked affects the probability of becoming a leader.Nodes that cheat will have their stakes reduced, and the reduced funds will be added to the block generation reward.
Turbine
Turbine is Solana's block propagation protocol designed to efficiently distribute large amounts of data.In this protocol, the leader (the node that generates the block) splits the block into smaller packets and sends them to the first group of validators.These validators redistribute the packets they receive to their own "neighbor" groups.The network is organized in a hierarchical tree-like fashion, with data propagating quickly down the hierarchy.This allows for efficient distribution of data to all validators and significant bandwidth savings.
Gulf Stream
Gulf Stream is a technology that streamlines transaction preprocessing and transmission: in the Solana network, each validator knows the "scheduled leaders" who will generate the next block.Based on this information, clients and validators send transactions directly to these scheduled leaders in advance.This reduces transaction validation time and overall system latency.
Sealevel
Sealevel provides a runtime environment in which a large number of smart contracts can be executed in parallel.The runtime environment is the infrastructure in which the program (in this case, the smart contract) is executed; Sealevel allows transactions that do not have dependencies to be extracted and executed in parallel by predefining the state of the data to be read and written by the transaction.Sealevel allows transactions to be executed in parallel.This is accomplished by Proof of History providing a timestamp for each transaction; PoH is responsible for establishing the order of transactions, and Sealevel performs parallel processing of undependent transactions based on this established order.
These technologies work in conjunction with PoH to optimize Solana's performance: Turbine and Gulf Stream provide efficient data propagation and transaction transfer, and Sealevel processes multiple transactions simultaneously on top of this robust foundation.a blockchain that achieves high speed processing by thoroughly streamlining each and every process.
For those of you scratching your heads, I've created a simple analogy to help you understand how Solana works.
Imagine a small company with 180 employees.This company has many different departments (accounting, sales, marketing, customer service, etc.).Every employee knows how all the other departments work.Additionally, each employee must take on a random leadership role every 1.6 hours.During that time, the leader must sign documents sent from various departments.
While which employee is chosen as leader is essentially random, each employee's computer screen has a small window that displays the next 10 people who will assume the role of leader.This allows employees to give documents to their scheduled leaders in advance and get their work done faster (Gulf Stream).
Each time a leader signs a document, it is time-stamped (PoH) and efficiently sent back to the appropriate department (node cluster) for double-checking (Turbine).Once double-checking is complete, the documents are added to the company database.The work of each department is highly structured so that multiple documents can be processed at the same time while working together to avoid duplication (Sealevel).
This is the basic structure in place at Solana.The key point is to assign different tasks to different nodes on the network as needed and to time-stamp all transactions to ensure accuracy.Based on this, Solana has various efficiency mechanisms built into the consensus process.
5. challenges facing solana
Although Solana is known for its high throughput, it faces several significant challenges.
The first is repeated network outages.As many of you know, Solana has suffered several network outages since its mainnet launch in 2020.The main causes of these outages have been congestion and increased bot activity.Despite improvements to the network each time these failures occur, the problem has not been resolved as of 2024, and major concerns about network reliability continue.
Why do Solana's network failures occur so often?
This is because although Solana's scalability is not low, it has not kept pace with the speculative demand that is heating up in the bull market.And while various measures have been taken to scale throughput, the underlying architecture seems to be limited.
Solana has the ability to handle high transaction loads such as high-frequency trading, but its consensus process is very complex, and the entire network is prone to delays when transactions are concentrated.
These issues are shared by the Solana team, and in April 2024, Yakovenko commented to the community that "fixing bugs will serve as a temporary relief, but will be a difficult task with respect to more fundamental network updates."
And if Solana sees Ethereum as a competitor, the lack of client diversity is an important issue.Currently, the validator clients of the Solana network rely primarily on those developed by Solana Labs.The same can be pointed out for most blockchains, not just Solana.
In this situation, if a bug or vulnerability is discovered in a client, there is an increased risk of a major impact on the entire network.If multiple independent clients could exist, the network would be more resilient and more secure.However, this point will be improved over the next year.This will be discussed in more detail later in this report.
Finally, operational costs and hardware requirements are increasing, as Solana's validators require high-performance hardware, which tends to increase operational costs.In particular, as the network scales, the hardware performance required for the validator increases, and with it, the cost.
In general terms, this creates a barrier to entry for new participants and risks reducing network dispersion.(Currently, the number of validators is growing steadily, despite the high hardware requirements.)
6. firedancer
He explained that Solana is faced with the challenges of repeated network failures and underlying scalability limitations, lack of client diversity, and increasing operational costs and hardware requirements.
The solution is currently being developed as a new validator client, Firedancer, which is being developed by Jump Crypto and is expected to be implemented on the mainnet in 2025.
The concept of Firedancer was born out of the need to address the challenges faced by Solana, including frequent network outages and limited scalability.In particular, major network outages in 2021 and 2022 raised concerns about Solana's reliability.In response, Jump Crypto began developing a new validator client to improve Solana's performance and reliability, leveraging its experience in high-frequency trading.
There should be no single point of failure anywhere in the quest for the ideal blockchain, and Solana aims to have four separate validators written in four separate programming languages by four separate teams to build multiple redundancies.One of these is Firedancer, which was developed with the following goals
Completely rewrite the Solana validator code
Ensure compatibility with existing protocols and validators
Written in C for high performance
Increased scalability (initial tests show 10- to 100-fold improvement)
Firedancer uses a modular architecture that differs from current Solana Labs clients, allowing each component to operate independently, thus increasing the stability of the overall system.
Modular Architecture
Firedancer is composed of independent Linux processes (tiles), eachtile plays a specific role.This ensures that if a problem occurs with a particular tile, it does not affect other tiles, thus maintaining overall system stability.In addition, each tile can be upgraded or replaced independently, minimizing downtime.Data Parallel Processing
Firedancer processes multiple data elements simultaneously to increase transaction validation speedFiredancer improves transaction validation speed by processing multiple data elements simultaneously.This significantly improves the throughput of signature verification.High Performance Networking Stack
Firedancer efficiently distributes network traffic toFiredancer achieves performance beyond traditional load balancing methods by efficiently distributing network traffic.This ensures stable network performance even under heavy loads.
Using FPGAsUsing FPGAs
Firedancer uses Field Programmable Gate Array (FPGA) to achieve high throughput with low power consumption.Programmable Gate Array (FPGA) for low power consumption and high throughput.This greatly improves real-time processing.Sharding support
Firedancer splits transactions into multiple parts for parallel processing,overall processing capacity.
Although Firedancer is still in its early stages, performance tests have shown that Solana has the potential to consistently outperform Visa and other blockchains in terms of transactions per second (TPS).In fact, "Firedancer" is a technological feat, according to technical expert Paul Baron
"If you think about what technological advances have been made in computing power over the past 20 to 30 years, this technology is definitely in the top 10.What they have accomplished is truly remarkable."
7. can firedancer really solve Solana's problems?
While Firedancer has the potential to significantly improve Solana's scalability and reliability, several concerns exist.First, Firedancer is developed in C and does not have the memory safety guarantees provided by Rust.Naturally, the developers are aware of this, and I am sure that development is being conducted with the utmost care, but there is a certain possibility that memory management vulnerabilities may occur.
In addition, since it is designed differently from the existing Solana Labs client, its behavior must be faithfully reproduced for compatibility.Failure to do so could lead to consensus bugs due to incompatibility, which could lead to security risks.
Furthermore, even if Firedancer is implemented, it is unclear how many validators will switch to Firedancer immediately; while the performance gains of Firedancer are attractive, many validators may be cautious given the risks described above.The risks described may cause many validators to be cautious.
Also, although various sources advertise that Firedancer supports sharding, given the way blockchains work, it should be necessary to implement sharding functionality in the network itself for full-scale use.This is technically very complex for a monolithic blockchain like Solana, and would involve redesigning the entire network, which would require time and resources to implement.In addition, new challenges associated with the implementation of sharding (e.g., maintaining data consistency and optimizing communication between shards) must be considered.
Firedancer, as a new architecture for Solana's validator client, has the potential to contribute significantly to scalability and reliability.However, it is also true that there are some concerns, and it remains to be seen how far Firedancer can contribute to Solana's problem solving until it is implemented.
This is "Solana Complete Review (Part 1).In the second part, which will be updated tomorrow, we will explain "The Road to Decentralization," "Ecosystem Status," "Tokenomics Overview," "Outlook and Considerations," etc. Please look forward to it!
Disclaimer：I carefully examine and write the information that I research, but since it is personally operated and there are many parts with English sources, there may be some paraphrasing or incorrect information. Please understand. Also, there may be introductions of Dapps, NFTs, and tokens in the articles, but there is absolutely no solicitation purpose. Please purchase and use them at your own risk.
About us
🇯🇵🇺🇸🇰🇷🇨🇳🇪🇸 The English version of the web3 newsletter, which is available in 5 languages. Based on the concept of ``Learn more about web3 in 5 minutes a day,'' we deliver research articles five times a week, including explanations of popular web3 trends, project explanations, and introductions to the latest news.
Author
mitsui
A web3 researcher. Operating the newsletter "web3 Research" delivered in five languages around the world.
Contact
The author is a web3 researcher based in Japan. If you have a project that is interested in expanding to Japan, please contact the following:
Telegram：@mitsui0x
*Please note that this newsletter translates articles that are originally in Japanese. There may be translation mistakes such as mistranslations or paraphrasing, so please understand in advance.