25,161
edits
Changes
BMI2
,no edit summary
Concerning the application of [[Neural Networks|neural network]] training, in particular [[NNUE #Training|NNUE training]] with millions of [[Chess Position|chess position]] inside a huge file of [[Forsyth-Edwards Notation|FEN]] records,
Lucas Braesch proposed the idea to [https://en.wikipedia.org/wiki/Data_compression compress] a chess position by a sequence of [[#PEXT|PEXT]] instructions,
while the decompression is done by an sequence of corresponding [[#PDEP|PDEP]] instructions <ref>[http://www.talkchess.com/forum3/viewtopic.php?f=7&t=76892&start=3 Re: FEN compression] by lucasart, [[CCC]], March 17, 2021 » [[Forsyth-Edwards Notation]], [[NNUE #Training|NNUE Training]]</ref>.This is intended as possible alternative of the [https://en.wikipedia.org/wiki/Huffman_coding Huffman coding] as applied in the [[Gary Linscott#PyTorch NNUE|PyTorch NNUE]] code by [[Gary Linscott]] <ref>[https://github.com/glinscott/nnue-pytorch/blob/master/lib/nnue_training_data_formats.h#L6405 nnue-pytorch/nnue_training_data_formats.h at master · glinscott/nnue-pytorch · GitHub]</ref>.The table illustrates the layout:{| class="wikitable"! Item! Bits ! Definition|-| occupancy rocc<br/>(remaining occupancy) | style="text-align:right;" | 64| style="text-align:center;" | <nowiki>rocc = white | black </nowiki>|-| pext(white, rocc)| style="text-align:right;" | cnt(rocc)| |-| black = rocc ^ white| style="text-align:right;" | 0||-| pext(pawns, rocc)| style="text-align:right;" | cnt(rocc)| |-| pext(knights, rocc)| style="text-align:right;" | cnt(rocc)| style="text-align:center;" | rocc ^= pawns|-| pext(bishops, rocc)| style="text-align:right;" | cnt(rocc)| style="text-align:center;" | rocc ^= bishops|-| pext(rooks, rocc)| style="text-align:right;" | cnt(rocc)| style="text-align:center;" | rocc ^= rooks |-| pext(queens, rocc)| style="text-align:right;" | cnt(rocc)| style="text-align:center;" | rocc ^= queens|-| kings = rocc| style="text-align:right;" | 0||-| side to move| style="text-align:right;" | 1||-| ep| style="text-align:right;" | 1||-| pext(epSquare, candis)| style="text-align:right;" | cnt(candis)| style="text-align:right;" | candis = ((white & pawns & rank4) << 8)<br/> | ((black & pawns & rank6) >> 8)|-| pext(castler, rooks)| style="text-align:right;" | cnt(rooks)| style="text-align:center;" | castler = rooks with castling rights<br/>(any color)|-| rule50| style="text-align:right;" | 7||}
The uncompressed piece placement bitstream is based on the [[Bitboard Board-Definition#SixTwo|dense bitboard board-definition]]
and consists of the [[Occupancy|occupancy]], one color bitboard (e.g. White pieces),
U64 rocc = pieceBB[nWhite] | pieceBB[nBlack];
stream.write_n_bit(rocc, 64);
stream.write_n_bit(_pext_u64(rocc, pieceBB[nWhite], rocc), popCount(rocc));
for (pt = nPawn; pt <= nQueen; ++pt) {
stream.write_n_bit(_pext_u64(rocc, pieceBB[pt], rocc), popCount(rocc));
rocc ^= pieceBB[pt];
}
void decompress (const BitStream &stream, U64 * pieceBB, ...) {
U64 rocc = stream.read_n_bit(64);
pieceBB[nWhite] = _pdep_u64(rocc, stream.read_n_bit(popCount(rocc)), rocc);
pieceBB[nBlack] = rocc ^ pieceBB[nWhite];
for (pt = nPawn; pt <= nKing; ++pt) {
pieceBB[pt] = _pdep_u64(rocc, stream.read_n_bit(popCount(rocc)), rocc);
rocc ^= pieceBB[pt];
}
