Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

remove the crc-32 library from dependency #1565

Open
wants to merge 2 commits into
base: main
Choose a base branch
from
+61 −8
Open

remove the crc-32 library from dependency #1565

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
d425722
remove the `crc-32` library from dependency
Danil42Russia Jan 24, 2025
2cefcbc
removed the conversion to unsigned by default
Danil42Russia Jan 25, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 0 additions & 1 deletion package.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -66,7 +66,6 @@
"@tonstudio/parser-runtime": "^0.0.1",
"blockstore-core": "1.0.5",
"change-case": "^4.1.2",
"crc-32": "1.2.2",
"ipfs-unixfs-importer": "9.0.10",
"json-bigint": "^1.0.0",
"ohm-js": "^17.1.0",
Expand Down
5 changes: 3 additions & 2 deletions src/optimizer/interpreter.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
import { Address, beginCell, BitString, Cell, toNano } from "@ton/core";
import { paddedBufferToBits } from "@ton/core/dist/boc/utils/paddedBits";
import * as crc32 from "crc-32";
import { crc32 } from "../utils/crc32";
import * as A from "../ast/ast";
import { evalConstantExpression } from "./constEval";
import { CompilerContext } from "../context/context";
Expand Down Expand Up @@ -69,6 +69,7 @@ function throwErrorConstEval(msg: string, source: SrcInfo): never {
source,
);
}

type EvalResult =
| { kind: "ok"; value: A.AstLiteral }
| { kind: "error"; message: string };
Expand Down Expand Up @@ -1342,7 +1343,7 @@ export class Interpreter {
this.interpretExpression(ast.args[0]!),
);
return this.util.makeNumberLiteral(
BigInt(crc32.str(str.value) >>> 0),
BigInt(crc32(str.value) >>> 0),
ast.loc,
); // >>> 0 converts to unsigned
}
Expand Down
58 changes: 58 additions & 0 deletions src/utils/crc32.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,58 @@
const TABLE = new Int32Array([
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please don't add generated code here.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why?

  1. it is faster than creating a table every time;
  2. it is done by analogy with crc16, which has the same table.

Copy link
Contributor

@verytactical verytactical Jan 25, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Which of the two options sounds more correct? :)

  • execute 2048 operations every time compiler starts
  • have a table of questionable provenance that is impossible to review, that also takes 65536 operations every time TS checks types, and also increases size of js bundle

Copy link
Contributor Author

@Danil42Russia Danil42Russia Jan 25, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So typescript and type checking is called only once, when building a project. And if you don't calculate the table, you will have to do it every time you start the compiler.

Regarding the “weird” table:

  1. This table is available in the LLVM project: https://github.com/llvm/llvm-project/blob/6383a12e3b4339fa4743bb97da4d51dea6d2e2ea/llvm/lib/Support/CRC.cpp#L30
  2. There are many references in Microsoft repository https://github.com/search?q=org%3Amicrosoft%20%220xD9D65ADC%22&type=code

If you look at the code of the js-crc32 library, it doesn't get any better, it doesn't even have comments https://github.com/SheetJS/js-crc32/blob/master/crc32c.js#L85

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

To make sure the table is correct, you can copy from anywhere else, diff will be zero.
I can also add its generator to the project.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I realize there are quite a few mentions of this table online, because it's used for something like last half of a century, but I don't think checking numbers by hand is a sound approach to development.

type checking is called only once

There's also tsserver.

0x00000000, 0x77073096, 0xee0e612c, 0x990951ba, 0x076dc419, 0x706af48f,
0xe963a535, 0x9e6495a3, 0x0edb8832, 0x79dcb8a4, 0xe0d5e91e, 0x97d2d988,
0x09b64c2b, 0x7eb17cbd, 0xe7b82d07, 0x90bf1d91, 0x1db71064, 0x6ab020f2,
0xf3b97148, 0x84be41de, 0x1adad47d, 0x6ddde4eb, 0xf4d4b551, 0x83d385c7,
0x136c9856, 0x646ba8c0, 0xfd62f97a, 0x8a65c9ec, 0x14015c4f, 0x63066cd9,
0xfa0f3d63, 0x8d080df5, 0x3b6e20c8, 0x4c69105e, 0xd56041e4, 0xa2677172,
0x3c03e4d1, 0x4b04d447, 0xd20d85fd, 0xa50ab56b, 0x35b5a8fa, 0x42b2986c,
0xdbbbc9d6, 0xacbcf940, 0x32d86ce3, 0x45df5c75, 0xdcd60dcf, 0xabd13d59,
0x26d930ac, 0x51de003a, 0xc8d75180, 0xbfd06116, 0x21b4f4b5, 0x56b3c423,
0xcfba9599, 0xb8bda50f, 0x2802b89e, 0x5f058808, 0xc60cd9b2, 0xb10be924,
0x2f6f7c87, 0x58684c11, 0xc1611dab, 0xb6662d3d, 0x76dc4190, 0x01db7106,
0x98d220bc, 0xefd5102a, 0x71b18589, 0x06b6b51f, 0x9fbfe4a5, 0xe8b8d433,
0x7807c9a2, 0x0f00f934, 0x9609a88e, 0xe10e9818, 0x7f6a0dbb, 0x086d3d2d,
0x91646c97, 0xe6635c01, 0x6b6b51f4, 0x1c6c6162, 0x856530d8, 0xf262004e,
0x6c0695ed, 0x1b01a57b, 0x8208f4c1, 0xf50fc457, 0x65b0d9c6, 0x12b7e950,
0x8bbeb8ea, 0xfcb9887c, 0x62dd1ddf, 0x15da2d49, 0x8cd37cf3, 0xfbd44c65,
0x4db26158, 0x3ab551ce, 0xa3bc0074, 0xd4bb30e2, 0x4adfa541, 0x3dd895d7,
0xa4d1c46d, 0xd3d6f4fb, 0x4369e96a, 0x346ed9fc, 0xad678846, 0xda60b8d0,
0x44042d73, 0x33031de5, 0xaa0a4c5f, 0xdd0d7cc9, 0x5005713c, 0x270241aa,
0xbe0b1010, 0xc90c2086, 0x5768b525, 0x206f85b3, 0xb966d409, 0xce61e49f,
0x5edef90e, 0x29d9c998, 0xb0d09822, 0xc7d7a8b4, 0x59b33d17, 0x2eb40d81,
0xb7bd5c3b, 0xc0ba6cad, 0xedb88320, 0x9abfb3b6, 0x03b6e20c, 0x74b1d29a,
0xead54739, 0x9dd277af, 0x04db2615, 0x73dc1683, 0xe3630b12, 0x94643b84,
0x0d6d6a3e, 0x7a6a5aa8, 0xe40ecf0b, 0x9309ff9d, 0x0a00ae27, 0x7d079eb1,
0xf00f9344, 0x8708a3d2, 0x1e01f268, 0x6906c2fe, 0xf762575d, 0x806567cb,
0x196c3671, 0x6e6b06e7, 0xfed41b76, 0x89d32be0, 0x10da7a5a, 0x67dd4acc,
0xf9b9df6f, 0x8ebeeff9, 0x17b7be43, 0x60b08ed5, 0xd6d6a3e8, 0xa1d1937e,
0x38d8c2c4, 0x4fdff252, 0xd1bb67f1, 0xa6bc5767, 0x3fb506dd, 0x48b2364b,
0xd80d2bda, 0xaf0a1b4c, 0x36034af6, 0x41047a60, 0xdf60efc3, 0xa867df55,
0x316e8eef, 0x4669be79, 0xcb61b38c, 0xbc66831a, 0x256fd2a0, 0x5268e236,
0xcc0c7795, 0xbb0b4703, 0x220216b9, 0x5505262f, 0xc5ba3bbe, 0xb2bd0b28,
0x2bb45a92, 0x5cb36a04, 0xc2d7ffa7, 0xb5d0cf31, 0x2cd99e8b, 0x5bdeae1d,
0x9b64c2b0, 0xec63f226, 0x756aa39c, 0x026d930a, 0x9c0906a9, 0xeb0e363f,
0x72076785, 0x05005713, 0x95bf4a82, 0xe2b87a14, 0x7bb12bae, 0x0cb61b38,
0x92d28e9b, 0xe5d5be0d, 0x7cdcefb7, 0x0bdbdf21, 0x86d3d2d4, 0xf1d4e242,
0x68ddb3f8, 0x1fda836e, 0x81be16cd, 0xf6b9265b, 0x6fb077e1, 0x18b74777,
0x88085ae6, 0xff0f6a70, 0x66063bca, 0x11010b5c, 0x8f659eff, 0xf862ae69,
0x616bffd3, 0x166ccf45, 0xa00ae278, 0xd70dd2ee, 0x4e048354, 0x3903b3c2,
0xa7672661, 0xd06016f7, 0x4969474d, 0x3e6e77db, 0xaed16a4a, 0xd9d65adc,
0x40df0b66, 0x37d83bf0, 0xa9bcae53, 0xdebb9ec5, 0x47b2cf7f, 0x30b5ffe9,
0xbdbdf21c, 0xcabac28a, 0x53b39330, 0x24b4a3a6, 0xbad03605, 0xcdd70693,
0x54de5729, 0x23d967bf, 0xb3667a2e, 0xc4614ab8, 0x5d681b02, 0x2a6f2b94,
0xb40bbe37, 0xc30c8ea1, 0x5a05df1b, 0x2d02ef8d,
]);

export function crc32(data: string | Buffer): number {
if (!(data instanceof Buffer)) {
Copy link
Contributor

@verytactical verytactical Jan 25, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If src/optimizer/interpreter.ts is the only place we call it from, we never pass Buffer here.

Buffer is Node's non-standard API for binary data, while there is platform-agnostic Blob. When we build browser version of @tact-lang/compiler, we have to include a polyfill for Buffer.

While there are other places that use Buffer, we're working hard to remove them. I'd consider removing Buffer support here too.

Copy link
Contributor Author

@Danil42Russia Danil42Russia Jan 27, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

while there is platform-agnostic Blob

Can you elaborate, I didn't see any use of Blob in the code?

data = Buffer.from(data);
}

let crc = 0xffffffff;
for (const byte of data) {
crc = TABLE[(crc ^ byte) & 0xff]! ^ (crc >>> 8);
}

return crc ^ 0xffffffff;
}
5 changes: 0 additions & 5 deletions yarn.lock
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2014,11 +2014,6 @@ core-util-is@^1.0.3:
resolved "https://registry.npmjs.org/core-util-is/-/core-util-is-1.0.3.tgz"
integrity sha512-ZQBvi1DcpJ4GDqanjucZ2Hj3wEO5pZDS89BWbkcrvdxksJorwUDDZamX9ldFkp9aw2lmBDLgkObEA4DWNJ9FYQ==

crc-32@1.2.2:
version "1.2.2"
resolved "https://registry.npmjs.org/crc-32/-/crc-32-1.2.2.tgz#3cad35a934b8bf71f25ca524b6da51fb7eace2ff"
integrity sha512-ROmzCKrTnOwybPcJApAA6WBWij23HVfGVNKqqrZpuyZOHqK2CwHSvpGuyt/UNNvaIjEd8X5IFGp4Mh+Ie1IHJQ==

create-jest@^29.7.0:
version "29.7.0"
resolved "https://registry.npmjs.org/create-jest/-/create-jest-29.7.0.tgz#a355c5b3cb1e1af02ba177fe7afd7feee49a5320"
Expand Down
Loading