Libbitcoin Database - Revision history

Evoskuil: /* See Also */ remove dead link

2021-03-09T01:30:30Z

Evoskuil: /* Dependencies */ change libbitcoin to libbitcoin-system.

2021-03-09T01:27:51Z

Dependencies: change libbitcoin to libbitcoin-system.

← Older revision		Revision as of 01:27, 9 March 2021
Line 55:		Line 55:
	* [http://www.boost.org boost]		* [http://www.boost.org boost]
	* [https://github.com/bitcoin/secp256k1 secp256k1]		* [https://github.com/bitcoin/secp256k1 secp256k1]
	* [[Libbitcoin_Common\|libbitcoin]]		* [[Libbitcoin_Common\|libbitcoin-system]]

	==See Also==		==See Also==

Evoskuil: /* History */

2017-03-08T17:47:45Z

History

← Older revision		Revision as of 17:47, 8 March 2017
Line 50:		Line 50:
	* The original implementation used [http://leveldb.org LevelDB].		* The original implementation used [http://leveldb.org LevelDB].
	* In late 2014 by [[Amir_Taaki\|Amir Taaki]] (genjix) replaced LevelDB with memory-mapped hash table storage and indexing.		* In late 2014 by [[Amir_Taaki\|Amir Taaki]] (genjix) replaced LevelDB with memory-mapped hash table storage and indexing.
	* In early 2016 [[Eric_Voskuil\|Eric Voskuil]] separated the database from [[Libbitcoin_Blockchain\|libbitcoin-blockchain]] into an independent library.		* In early 2016 [[Eric_Voskuil\|Eric Voskuil]] separated the database from [[Libbitcoin_Blockchain\|libbitcoin-blockchain]] into an independent library and used isolated locking to make the concurrent read-while-write safe and to enable concurrent writes.

	==Dependencies==		==Dependencies==

Evoskuil: /* Considerations */

2017-03-08T17:45:22Z

Considerations

← Older revision		Revision as of 17:45, 8 March 2017
Line 43:		Line 43:
	==Considerations==		==Considerations==
	* There is no mechanical hard drive optimization. The implementation is intended for solid state drives ([http://en.wikipedia.org/wiki/Solid-state_drive SSD]).		* There is no mechanical hard drive optimization. The implementation is intended for solid state drives ([http://en.wikipedia.org/wiki/Solid-state_drive SSD]).
	* There is a likelihood of index corruption during hard shutdown. In version 2 there is no means of detecting corruption aside from catastrophic fault. However given that the entire blockchain is a cache this is not considered significant. Repair can be accomplished by re-synchronizing the blockchain. Version 3 provides robust detection of corruption and the option to flush after each write, making corruption very unlikely (and still fully detectable).		* There is a likelihood of index corruption during hard shutdown. In version 2 there is no means of detecting corruption aside from catastrophic fault. However given that the entire blockchain is a cache this is not considered significant. Repair can be accomplished by re-synchronizing the blockchain. Version 3 provides robust detection of corruption and the option to flush after each write, making corruption very unlikely (and still fully detectable) if enabled.
	* Data files are append only, with logical deletions only. Therefore file size is not minimized following blockchain reorganization although the impact is typically small. Defragmentation can be accomplished by re-synchronizing the blockchain.		* Data files are append only, with logical deletions only. Therefore file size is not minimized following blockchain reorganization although the impact is typically small. Defragmentation can be accomplished by re-synchronizing the blockchain.
	* The database is effectively locked during write operations. As these operations are anticipated on a period of approximately ten minutes this is not typically significant. However during a period of catch-up synchronizing a version 2 server can become continuously unresponsive to requests. This is not a problem for version 3.		* The database is effectively locked during write operations. As these operations are anticipated on a period of approximately ten minutes this is not typically significant. However during a period of catch-up synchronizing a version 2 server can become continuously unresponsive to requests. This is not a problem for version 3.

Evoskuil: /* Design */

2017-03-08T17:44:11Z

Design

← Older revision		Revision as of 17:44, 8 March 2017
Line 21:		Line 21:

	==Design==		==Design==
	The database is built directly on the operating system's [https://en.wikipedia.org/wiki/Memory-mapped_file memory-mapped file] system. Logical queries are performed using a set of hash tables. The number of hash buckets is optimized to minimize hash collisions, though collisions are accommodated. These changes resulted in a substantial performance increase, near constant time, for queries against the blockchain. Insert performance was not materially affected. The database uses sequence locking to avoid ~~writer starvation while never~~ blocking the ~~reader~~. This is ideal for a high performance blockchain server as reads are significantly more frequent than writes and yet writes must proceed ~~wtihout~~ delay.		The database is built directly on the operating system's [https://en.wikipedia.org/wiki/Memory-mapped_file memory-mapped file] system. Logical queries are performed using a set of hash tables. The number of hash buckets is optimized to minimize hash collisions, though collisions are accommodated. These changes resulted in a substantial performance increase, near constant time, for queries against the blockchain. Insert performance was not materially affected. The database uses sequence locking to avoid blocking the writer. This is ideal for a high performance blockchain server as reads are significantly more frequent than writes and yet writes must proceed without delay.

	[[Image:Libbitcoin-blockchain-gets.png\|500px]] [[Image:Libbitcoin-blockchain-gets-normalised.png\|500px]]		[[Image:Libbitcoin-blockchain-gets.png\|500px]] [[Image:Libbitcoin-blockchain-gets-normalised.png\|500px]]

Evoskuil: /* Considerations */

2016-12-10T01:17:10Z

Considerations

← Older revision		Revision as of 01:17, 10 December 2016
Line 43:		Line 43:
	==Considerations==		==Considerations==
	* There is no mechanical hard drive optimization. The implementation is intended for solid state drives ([http://en.wikipedia.org/wiki/Solid-state_drive SSD]).		* There is no mechanical hard drive optimization. The implementation is intended for solid state drives ([http://en.wikipedia.org/wiki/Solid-state_drive SSD]).
	* There is a ~~possibility~~ of index corruption during hard shutdown. ~~There~~ is no means of detecting corruption aside from catastrophic fault. However given that the entire blockchain is a cache this is not considered significant. Repair can be accomplished by re-synchronizing the blockchain.		* There is a likelihood of index corruption during hard shutdown. In version 2 there is no means of detecting corruption aside from catastrophic fault. However given that the entire blockchain is a cache this is not considered significant. Repair can be accomplished by re-synchronizing the blockchain. Version 3 provides robust detection of corruption and the option to flush after each write, making corruption very unlikely (and still fully detectable).
	* Data files are append only, with logical deletions only. Therefore file size is not minimized following blockchain reorganization although the impact is typically small. Defragmentation can be accomplished by re-synchronizing the blockchain.		* Data files are append only, with logical deletions only. Therefore file size is not minimized following blockchain reorganization although the impact is typically small. Defragmentation can be accomplished by re-synchronizing the blockchain.
	* The database is effectively locked during write operations. As these operations are anticipated on a period of approximately ten minutes this is not typically significant. However during a period of catch-up synchronizing ~~the~~ server can become continuously unresponsive to requests.		* The database is effectively locked during write operations. As these operations are anticipated on a period of approximately ten minutes this is not typically significant. However during a period of catch-up synchronizing a version 2 server can become continuously unresponsive to requests. This is not a problem for version 3.

	==History==		==History==

Evoskuil: /* Database */

2016-12-10T01:11:47Z

Database

← Older revision		Revision as of 01:11, 10 December 2016
Line 26:		Line 26:

	==Database==		==Database==
	The following files constitute the blockchain database non-volatile storage. As of height ~~350~~,000 ~~the database consumes~~ approximately ~~105~~ GB of disk space.		The following files constitute the blockchain database non-volatile storage for a node. As of height 440,000 these consume approximately 102 GB of disk space.
	* ~~blocks_lookup~~		* block_index
	* ~~blocks_rows~~		* block_table
	* ~~history_lookup~~		* transaction_table

			The following files constitute the non-volatile storage for the server index.
			* history_table
	* history_rows		* history_rows
	* spends
	* stealth_index
	* stealth_rows		* stealth_rows
	* ~~txs~~		* spend_table

			The following files are used for locking.
			* flush_lock
			* exclusive_lock

	==Considerations==		==Considerations==

Evoskuil: add history section

2016-02-22T12:38:23Z

add history section

← Older revision		Revision as of 12:38, 22 February 2016
Line 1:		Line 1:
	The [https://github.com/libbitcoin/libbitcoin-database libbitcoin-database] library is a dependency of [[Libbitcoin_Blockchain\|libbitcoin-blockchain]], [[Libbitcoin_Node\|libbitcoin-node]] and [[Libbitcoin_Server\|libbitcoin-server~~]]. It was originally contained within [[Libbitcoin_Blockchain\|libbitcoin-blockchain~~]].		The [https://github.com/libbitcoin/libbitcoin-database libbitcoin-database] library is a dependency of [[Libbitcoin_Blockchain\|libbitcoin-blockchain]], [[Libbitcoin_Node\|libbitcoin-node]] and [[Libbitcoin_Server\|libbitcoin-server]].

	==Example==		==Example==
Line 21:		Line 21:

	==Design==		==Design==
	The ~~original implementation used [http://leveldb.org LevelDB]. Following a redesign in late 2014 by [[Amir_Taaki\|Amir Taaki]] (genjix)~~ the ~~database was replaced by a~~ [https://en.wikipedia.org/wiki/Memory-mapped_file memory-mapped file] ~~implementation~~. Logical queries are performed using a set of hash tables. The number of hash buckets is optimized to minimize hash collisions, though collisions are accommodated. These changes resulted in a substantial performance increase, near constant time, for queries against the blockchain. Insert performance was not materially affected. The database uses sequence locking to avoid writer starvation while never blocking the reader. This is ideal for a high performance blockchain server as reads are significantly more frequent than writes and yet writes must proceed wtihout delay.		The database is built directly on the operating system's [https://en.wikipedia.org/wiki/Memory-mapped_file memory-mapped file] system. Logical queries are performed using a set of hash tables. The number of hash buckets is optimized to minimize hash collisions, though collisions are accommodated. These changes resulted in a substantial performance increase, near constant time, for queries against the blockchain. Insert performance was not materially affected. The database uses sequence locking to avoid writer starvation while never blocking the reader. This is ideal for a high performance blockchain server as reads are significantly more frequent than writes and yet writes must proceed wtihout delay.

	[[Image:Libbitcoin-blockchain-gets.png\|500px]] [[Image:Libbitcoin-blockchain-gets-normalised.png\|500px]]		[[Image:Libbitcoin-blockchain-gets.png\|500px]] [[Image:Libbitcoin-blockchain-gets-normalised.png\|500px]]
Line 41:		Line 41:
	* Data files are append only, with logical deletions only. Therefore file size is not minimized following blockchain reorganization although the impact is typically small. Defragmentation can be accomplished by re-synchronizing the blockchain.		* Data files are append only, with logical deletions only. Therefore file size is not minimized following blockchain reorganization although the impact is typically small. Defragmentation can be accomplished by re-synchronizing the blockchain.
	* The database is effectively locked during write operations. As these operations are anticipated on a period of approximately ten minutes this is not typically significant. However during a period of catch-up synchronizing the server can become continuously unresponsive to requests.		* The database is effectively locked during write operations. As these operations are anticipated on a period of approximately ten minutes this is not typically significant. However during a period of catch-up synchronizing the server can become continuously unresponsive to requests.

			==History==
			* The original implementation used [http://leveldb.org LevelDB].
			* In late 2014 by [[Amir_Taaki\|Amir Taaki]] (genjix) replaced LevelDB with memory-mapped hash table storage and indexing.
			* In early 2016 [[Eric_Voskuil\|Eric Voskuil]] separated the database from [[Libbitcoin_Blockchain\|libbitcoin-blockchain]] into an independent library.

	==Dependencies==		==Dependencies==

Evoskuil at 12:25, 22 February 2016

2016-02-22T12:25:53Z

← Older revision		Revision as of 12:25, 22 February 2016
Line 1:		Line 1:
	The [https://github.com/libbitcoin/libbitcoin-database libbitcoin-database ] library is a dependency of [[Libbitcoin_Blockchain\|libbitcoin-blockchain]], [[Libbitcoin_Node\|libbitcoin-node]] and [[Libbitcoin_Server\|libbitcoin-server]]. It was originally contained within [[Libbitcoin_Blockchain\|libbitcoin-blockchain]].		The [https://github.com/libbitcoin/libbitcoin-database libbitcoin-database] library is a dependency of [[Libbitcoin_Blockchain\|libbitcoin-blockchain]], [[Libbitcoin_Node\|libbitcoin-node]] and [[Libbitcoin_Server\|libbitcoin-server]]. It was originally contained within [[Libbitcoin_Blockchain\|libbitcoin-blockchain]].

	==Example==		==Example==

Evoskuil: /* Example */

2016-02-22T12:24:55Z

Example

← Older revision		Revision as of 12:24, 22 February 2016
Line 9:		Line 9:
	{		{
	std::string prefix("blockchain");		std::string prefix("blockchain");
	const auto& genesis = bc::chain::block::genesis_mainnet();		const auto genesis = bc::chain::block::genesis_mainnet();

	if (argc > 1)		if (argc > 1)

← Older revision		Revision as of 01:30, 9 March 2021
Line 59:		Line 59:
	==See Also==		==See Also==
	* [[Libbitcoin]]		* [[Libbitcoin]]
	* [https://wiki.unsystem.net/en/index.php/Libbitcoin/Blockchain Libbitcoin blockchain design (Unsystem)]

	==References==		==References==