Basic USVM TS type system with type coercion #215

zishkaz · 2024-09-30T10:06:24Z

No description provided.

github-advanced-security

detekt found more than 20 potential problems in the proposed changes. Check the Files changed tab for more details.

zishkaz · 2024-10-03T08:24:19Z

This PR indirectly introduces a bug

.github/workflows/build-and-run-tests.yml

usvm-core/src/main/kotlin/org/usvm/StateForker.kt

usvm-ts/src/main/kotlin/org/usvm/TSBinaryOperator.kt

usvm-ts/src/test/kotlin/org/usvm/util/TSMethodTestRunner.kt

CaelmBleidd · 2024-10-04T12:40:06Z

usvm-ts/src/test/kotlin/org/usvm/util/TSTestResolver.kt


-class TSTestResolver {
+class TSTestResolver(
+    private val state: TSState


Why? Resolver should be able to resolve many states independently

TSTestResolver was instantiated for each returned state, and I needed access to state.typeStorage.
So I decided to add state as private constructor param.

usvm-ts/src/test/kotlin/org/usvm/util/TSTestResolver.kt

CaelmBleidd · 2024-10-04T12:42:30Z

usvm-ts/src/test/kotlin/org/usvm/util/TSTestResolver.kt

+    private fun approximateParam(expr: UConcreteHeapRef, idx: Int, model: UModelBase<EtsType>): TSObject =
+        with(expr.ctx as TSContext) {
+            val suggestedType = state.getSuggestedType(idx)
+            return suggestedType?.let { newType ->


Something odd is happening here. You can always access all required information using symbolic values and symbolic values only, otherwise you will encounter path divergence. This map cannot be built from concrete values to type info, only from symbolic ones

I believe different types of ref entities (reg readings, etc.) have unique ways to generate a key. For example, reg readings can use its' idx field as a key, and this key leads to correct list of types from any model.

But you might store it with another key -- if I resolve something from registerReading(1), it doesn't mean that I always accessed this value using this key. Probably, I could do it with registerReading(2).getByIndex(3), i could have accessed it with RR(3) (because of aliasing) and many many other ways

usvm-core/src/main/kotlin/org/usvm/Expressions.kt

usvm-ts/src/main/kotlin/org/usvm/TSBinaryOperator.kt

usvm-ts/src/main/kotlin/org/usvm/TSExprTransformer.kt

usvm-ts/src/main/kotlin/org/usvm/TSUnaryOperator.kt

usvm-ts/src/test/kotlin/org/usvm/util/TSMethodTestRunner.kt

usvm-ts/src/test/kotlin/org/usvm/util/TSTestResolver.kt

usvm-ts/src/test/kotlin/org/usvm/util/TSMethodTestRunner.kt

CaelmBleidd · 2024-10-09T11:38:15Z

usvm-core/src/main/kotlin/org/usvm/Expressions.kt

+    // TODO: draft
+    override fun internEquals(other: Any): Boolean = structurallyEqual(other)


Why is it a draft? Seems like we can implement it

CaelmBleidd · 2024-10-09T12:04:10Z

usvm-ts/src/main/kotlin/org/usvm/TSExprTransformer.kt

+    private fun generateAdditionalExprs(rawExprs: List<UExpr<out USort>>): List<UExpr<out USort>> = with(ctx) {
+        if (!rawExprs.all { it.sort == boolSort }) return emptyList()
+        val newExpr = when (baseExpr.sort) {
+            addressSort -> addedExprCache.putOrNull(mkEq(fpToBoolSort(asFp64()), asBool()))


Why don't we need to add a connection between baseSort expr and them? If it already exists for baseSort <-> fp and bool, it exists for bool and fp by transitivity as well

usvm-ts/src/main/kotlin/org/usvm/TSExpressions.kt

usvm-ts/src/test/kotlin/org/usvm/util/TSMethodTestRunner.kt

CaelmBleidd · 2024-10-09T12:16:52Z

usvm-ts/src/test/kotlin/org/usvm/util/TSTestResolver.kt

+    private fun approximateParam(expr: UConcreteHeapRef, idx: Int, model: UModelBase<EtsType>): TSObject =
+        with(expr.ctx as TSContext) {
+            val suggestedType = state.getSuggestedType(idx)
+            return suggestedType?.let { newType ->


But you might store it with another key -- if I resolve something from registerReading(1), it doesn't mean that I always accessed this value using this key. Probably, I could do it with registerReading(2).getByIndex(3), i could have accessed it with RR(3) (because of aliasing) and many many other ways

usvm-ts/src/main/kotlin/org/usvm/TSExprTransformer.kt

usvm-ts/src/main/kotlin/org/usvm/state/TSState.kt

usvm-ts/src/test/kotlin/org/usvm/util/TSTestResolver.kt

+
+    private fun approximateParam(expr: UConcreteHeapRef, idx: Int, model: UModelBase<EtsType>): TSObject =
+        when (val tr = model.typeStreamOf(expr).take(1)) {
+            is TypesResult.SuccessfulTypesResult -> with (expr.tctx) {


usvm-ts/src/main/kotlin/org/usvm/machine/TSTypeSystem.kt

+        }
+
+        val types = primitiveTypes.toMutableList()
+        return when (val remainingTypes = anyTypeStream.take(n - primitiveTypes.size)) {


usvm-ts/src/main/kotlin/org/usvm/machine/expr/TSExprTransformer.kt

+            // (primitives can't be cast to ref in TypeScript type coercion)
+            addressSort -> {
+                val fpToBoolLink = mkEq(fpToBoolSort(asFp64()), asBool())
+                val boolToRefLink =  mkEq(asBool(), (baseExpr as UExpr<UAddressSort>).neq(mkNullRef()))


usvm-ts/src/main/kotlin/org/usvm/machine/expr/TSExpressions.kt

@@ -0,0 +1,111 @@
+package org.usvm.machine.expr
+
+import io.ksmt.KAst


usvm-ts/src/main/kotlin/org/usvm/machine/expr/TSExpressions.kt

+    private fun coerce(
+        other: UExpr<out USort>,
+        action: CoerceAction,
+    ): UExpr<out USort> = when (other) {


usvm-ts/src/main/kotlin/org/usvm/machine/operator/TSBinaryOperator.kt

@@ -0,0 +1,151 @@
+package org.usvm.machine.operator
+
+import io.ksmt.utils.cast


usvm-ts/src/test/kotlin/org/usvm/util/TSMethodTestRunner.kt

-    discoverProperties(
-        methodIdentifier: MethodDescriptor,
-        vararg analysisResultMatchers: (T1, T2, T3, T4, R?) -> Boolean,
+    protected inline fun <reified T1 : TSObject, reified T2 : TSObject, reified T3 : TSObject, reified T4 : TSObject, reified R : TSObject> discoverProperties(


usvm-ts/src/test/kotlin/org/usvm/util/TSMethodTestRunner.kt

-    discoverProperties(
-        methodIdentifier: MethodDescriptor,
-        vararg analysisResultMatchers: (T1, T2, T3, T4, R?) -> Boolean,
+    protected inline fun <reified T1 : TSObject, reified T2 : TSObject, reified T3 : TSObject, reified T4 : TSObject, reified R : TSObject> discoverProperties(


usvm-ts/src/test/kotlin/org/usvm/util/TSMethodTestRunner.kt

+            only KClass<TSObject> is available to match different objects.
+            However, this method is also used in parent TestRunner class
+            and passes here TSObject instances. So this check on current level is required.
+        */


usvm-ts/src/test/kotlin/org/usvm/util/TSTestResolver.kt

@@ -1,5 +1,6 @@
 package org.usvm.util

+import io.ksmt.utils.cast


usvm-ts/src/test/kotlin/org/usvm/util/TSTestResolver.kt

+        ctx: TSContext,
+        model: UModelBase<EtsType>,
+    ): List<TSObject> = with(ctx) {
+        params.map {  param ->


zishkaz requested a review from CaelmBleidd September 30, 2024 10:06

zishkaz self-assigned this Sep 30, 2024

zishkaz marked this pull request as draft September 30, 2024 11:40

github-advanced-security bot found potential problems Oct 2, 2024

View reviewed changes

zishkaz marked this pull request as ready for review October 3, 2024 08:24

CaelmBleidd requested changes Oct 4, 2024

View reviewed changes

github-advanced-security bot found potential problems Oct 8, 2024

View reviewed changes

usvm-ts/src/test/kotlin/org/usvm/util/TSMethodTestRunner.kt Fixed Show fixed Hide fixed

zishkaz requested a review from CaelmBleidd October 9, 2024 11:24

CaelmBleidd requested changes Oct 9, 2024

View reviewed changes

github-advanced-security bot found potential problems Oct 17, 2024

View reviewed changes

github-advanced-security bot found potential problems Dec 9, 2024

View reviewed changes

CaelmBleidd force-pushed the sergeyl/ts_typecoercion branch from 1eed307 to b58e886 Compare December 20, 2024 16:04

CaelmBleidd changed the base branch from main to caelmbleidd/update_kotlin_version December 20, 2024 16:05

CaelmBleidd force-pushed the caelmbleidd/update_kotlin_version branch from 556fa95 to 703b36c Compare December 20, 2024 18:08

Base automatically changed from caelmbleidd/update_kotlin_version to main December 26, 2024 18:17

CaelmBleidd added 13 commits January 24, 2025 15:47

Upgrade kotlin version

015cbc3

Type coercion

1248a67

Some fixes

beccd93

Tmp commit

ff880f7

Tmp commit

8bbfedc

Govno

2484658

MultiExpr support

c065b19

EQ operator support

21c686b

tmp commit

567c7f6

Remove multiexpr and multivalue

d227391

Add caches for local sorts

0fb485f

Support arguments resolve

80bfa8a

Drochilnya with unresolved objects

b45b6aa

CaelmBleidd and others added 29 commits January 24, 2025 15:49

Continuation of drochilnya with unresolved objects

f672517

Completed test

28c9fa8

Eval expr in model

c323bb5

Format

b80db92

Format

ee705a5

Support any

057b865

Fix indent

86f124d

Cleanup

00cb506

Use fp64Sort

9a1c43b

Cleanup

8d288c1

fix ICE

8001d7d

Remove old comments

fd7bda1

Fix check

8695f20

Add comment about ToBoolean

945323e

Cleanup

0d4ad61

Add EXACTLY_ONCE invokation contract for calcOnState and doWithState

545c334

More tests for And

9047742

More tests

f85c92f

Add class for expr with type guard

3de12b0

Add type constraint in fake type creation

9f61e97

Add example with nan equality

39a4ee5

Remove redundant code with expr transformer

a4cabdb

Add a clarification comment

6e214c4

Remove multiple asserts during one scope call

db90185

Disable test

4cabbfa

Disable tests

0e406b7

Bump jacodb version

8b7f9b1

Rebase fixes

cc43637

Remove non-related changes

aa18660

CaelmBleidd force-pushed the sergeyl/ts_typecoercion branch from bbe7107 to aa18660 Compare January 24, 2025 13:08

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Basic USVM TS type system with type coercion #215

Basic USVM TS type system with type coercion #215

zishkaz commented Sep 30, 2024

github-advanced-security bot left a comment

zishkaz commented Oct 3, 2024

CaelmBleidd Oct 4, 2024

zishkaz Oct 8, 2024

CaelmBleidd Oct 4, 2024

zishkaz Oct 8, 2024

CaelmBleidd Oct 9, 2024

CaelmBleidd Oct 9, 2024

CaelmBleidd Oct 9, 2024

CaelmBleidd Oct 9, 2024

		// TODO: draft
		override fun internEquals(other: Any): Boolean = structurallyEqual(other)

		@@ -0,0 +1,111 @@
		package org.usvm.machine.expr

		import io.ksmt.KAst

		@@ -0,0 +1,151 @@
		package org.usvm.machine.operator

		import io.ksmt.utils.cast

		@@ -1,5 +1,6 @@
		package org.usvm.util

		import io.ksmt.utils.cast

Basic USVM TS type system with type coercion #215

Are you sure you want to change the base?

Basic USVM TS type system with type coercion #215

Conversation

zishkaz commented Sep 30, 2024

github-advanced-security bot left a comment

Choose a reason for hiding this comment

zishkaz commented Oct 3, 2024

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment