Create AISub.md #2365
Create AISub.md #2365
Conversation
|
CLA Assistant Lite bot All contributors have signed the CLA ✍️ ✅ |
github-actions
bot
added
the
admin-review
|
I have read and hereby sign the Contributor License Agreement. |
There was a problem hiding this comment.
Thanks for your interest in our grants program. It feels like there are a lot of features and mechanics that you desire which are not fully thought through and require a lot of work to be fully functional. I have left some comments on some of these features. Most bullet points are very vague. Rarely are any features are described in depth. Furthermore it appears to be the case that you are not experienced in Substrate development yet. This in itself is not a problem but in the combination with the huge scope of this application there seems to be a critical mismatch of skills and aspirations. Simply put I see to much risk of funding an over-ambitious project and team here. I'm by no means trying to invalidate your ideas. I would love to see your project come to life and revolutionize the AI space with this decentralized approach. Yet with all things considered I don't think it's a good fit for our grants program. I will not approve it. Potentially you can convince my colleagues to gather enough votes for a grant. Best wishes trying to realize your project!
|
|
||
| 2. **Review Management**: | ||
| - Users can view their reviews of the AI models they have used through the user interface. Each review is linked to a specific model call record, allowing users to reflect on their feedback for each experience. | ||
| - The platform also supports immediate reviews after using a model, enabling users to rate and leave detailed feedback on their experiences. These reviews will be stored on-chain and used for future model optimization and recommendation algorithms. |
There was a problem hiding this comment.
How will you prevent misuse of the recommendation system? Sounds like everybody who once paid for a model could leave a bad review and influence it's standing within the recommendations.
There was a problem hiding this comment.
Thank you for taking the time to review our application and provide valuable feedback. We greatly appreciate each of your suggestions and will carefully consider these points to refine our plan.
Regarding the development experience issue you mentioned, I would like to clarify that our team has experience with Substrate development and has participated in two Polkadot hackathons. These experiences have provided us with a foundational understanding of the framework and the technical ecosystem. Additionally, we are willing to continue learning and improving in new projects to ensure that we can meet the challenges required by the project.
In the initial phase, we will adopt a manual review mechanism to supervise the evaluation of models. Furthermore, suppliers can lodge appeals on-chain regarding user reviews. This process is conducted on-chain, ensuring transparency and fairness. If a user is found to have a significant amount of malicious comments, they will be restricted from using the platform through a public voting mechanism. Moreover, only users who have actually used the model can leave a review, meaning that each review comes at a cost, further ensuring the authenticity and reliability of the evaluations.
| - The model list supports various filtering criteria, such as model type, provider, and ratings, allowing users to quickly find the most suitable models for their needs. | ||
|
|
||
| 6. **Model Execution and Result Retrieval**: | ||
| - Users can initiate AI model execution requests directly from the user interface by following a specified JSON format. Users only need to input the required parameters, and the platform will automatically send the request to the blockchain and assign the appropriate model provider to handle it. |
There was a problem hiding this comment.
How do you make sure that the user is actually getting execution time on the model he requested? The output that is provided could be generated by another model.
There was a problem hiding this comment.
We have embedded a signature mechanism in the model suppliers' system to ensure that each model execution result comes with a unique signature. This means that each model supplier will sign the generated output using the private key of their own address. Users can verify this signature to confirm that the output indeed originates from the designated model supplier and model. All model execution requests and corresponding execution records will be recorded and stored on the blockchain. This allows users to view and verify which model was executed by the model supplier and when, ensuring that each execution is public, transparent, and tamper-proof. The platform provides a user feedback and rating system, allowing users to evaluate model suppliers based on their experience. By aggregating user evaluations and feedback, the committee can better monitor suppliers' behavior and take necessary actions to ensure the users' interests.
| - All payments and revenue distributions will be handled through decentralized assets on the blockchain, ensuring transparency and security in transactions. | ||
|
|
||
| 4. **Output Model Execution Results**: | ||
| - After receiving a user's AI model execution request, suppliers can directly output the results to the blockchain through the platform. |
There was a problem hiding this comment.
Are you saying the output of a model will be put on-chain?
There was a problem hiding this comment.
Yes, the output results will be stored on-chain. If the output exceeds size limitations, the supplier will use IPFS for storage within the protocol. In such cases, the data stored on the chain will be an IPFS link.
| 3. **Pricing and Charging**: | ||
| - Suppliers can set usage fees for their models. The platform offers various pricing strategies, allowing suppliers to set fees based on usage frequency, time, computational resources consumed, and other factors. | ||
| - Suppliers can choose between fixed pricing or dynamic pricing models. Dynamic pricing allows for adjustments based on model demand and market conditions, enabling more flexible pricing strategies. | ||
| - **When using open-source models**, suppliers must include the corresponding open-source license. This ensures compliance and respects the contributions of the open-source community. Suppliers must clearly specify the type of license used and upload the relevant license files to the platform during the model's publication. |
There was a problem hiding this comment.
Why would you upload license information? I don't think this is necessary information for the user or chain as this seems to be completely trust based. Did you derive this specification from actual open source license restrictions? If so which license led you to this belief?
There was a problem hiding this comment.
We require the upload of license information because sometimes suppliers use open-source models, and it's important to ensure that these models comply with their respective open-source licenses, such as restrictions on commercial use. While we cannot fully verify the compliance of every model used by suppliers, including this information helps mitigate potential copyright and legal risks for the platform. We understand that this may rely on suppliers' honesty, but we aim to provide a safeguard. We are open to making this an optional field in the future.
applications/AISub.md
Outdated
| | 0c. | Testing Guide | Core functions will be fully covered by unit tests to ensure functionality and robustness. In the guide, we will describe how to run these tests. | | ||
| | 0d. | Docker | We will provide a Dockerfile(s) that can be used to test all the functionality delivered with this milestone. | | ||
| | 1. | pallet-member | `create_membership_plan()`: Create new membership plans (admin only)<br>`purchase_membership_plan()`: Allow users to purchase a membership plan<br>`join_membership_plan_as_supplier()`: Suppliers join a membership plan to offer their models<br>`join_membership_plan_as_machine_supplier()`: Machine suppliers join a membership plan to perform calculations for models and receive pool rewards | | ||
| | 2. | pallet-machine-supplier | `create_machine_service()`: Create and register a machine service for running AI models<br>`penalize_slow_performance()`: Penalize machines for slow performance<br>`penalize_failure_to_run()`: Penalize machines that fail to run models as required | |
There was a problem hiding this comment.
How would that work? How do you penalize somebody for slow performance? Who decides what is slow and what isn't?
There was a problem hiding this comment.
For each model execution, we will calculate the time based on the difference between the height of the request and the completion height. For similar types of models, such as text models, we can determine the average execution time after multiple runs. This process helps in maintaining time statistics for suppliers.
Suppliers with slower execution times will be ranked lower in our sorting system. The average processing time of models will be displayed on the front end, allowing users to make informed choices based on processing times. If a supplier’s execution time exceeds 100 times the overall average, their status will be temporarily set to "offline" on the blockchain. The supplier will need to apply for a status change, which will be reviewed and decided by the committee. Additionally, this action will also be performed during heartbeat checks.
applications/AISub.md
Outdated
| | 0b. | Documentation | We will provide both **inline documentation** of the code and a basic **tutorial** that explains how a user can (for example) spin up one of our Substrate nodes and send test transactions, which will show how the new functionality works. | | ||
| | 0c. | Testing Guide | Core functions will be fully covered by unit tests to ensure functionality and robustness. In the guide, we will describe how to run these tests. | | ||
| | 0d. | Docker | We will provide a Dockerfile(s) that can be used to test all the functionality delivered with this milestone. | | ||
| | 1. | pallet-member | `create_membership_plan()`: Create new membership plans (admin only)<br>`purchase_membership_plan()`: Allow users to purchase a membership plan<br>`join_membership_plan_as_supplier()`: Suppliers join a membership plan to offer their models<br>`join_membership_plan_as_machine_supplier()`: Machine suppliers join a membership plan to perform calculations for models and receive pool rewards | |
There was a problem hiding this comment.
One of the main features of memberships are that you are not allowed to share them without restrictions. How would you enforce this feature in a blockchain environment?
There was a problem hiding this comment.
In a blockchain environment, the actual membership duration is represented by the number of blocks. Once a user joins a membership plan, the membership start and end times are stored in the data. The end of the membership is determined based on these block numbers.
To enforce this feature, we use on_finalize to traverse the membership table and check whether the membership has reached the required number of blocks. If it has, the membership expires. Alternatively, each time a membership action is performed, the system checks if the stored membership expiration block number is less than the current block height. If so, membership operations will be restricted.
applications/AISub.md
Outdated
|
|
||
|
|
||
|
|
||
| When the milestones in the grant are completed, we will open source the entire project. |
There was a problem hiding this comment.
This is sadly not in line with our grants program. We do not pay for any milestone deliveries which are closed source.
There was a problem hiding this comment.
This is indeed a mistake on my part, and I have corrected the relevant sections. We will open source the entire project. Our projects will be open source under the GPL-3.0 license.
There was a problem hiding this comment.
Thank you for the application, @kencani, and for diligently answering the questions.
- Are you planning to deploy your own parachain?
- I would categorize your proposal as business-oriented, because I don't think there is a need for more marketplace (user, member, model) pallets. Therefore, I would like to see some form of market analysis to see that there is a need for a project like this (see also our guidelines). Have you done any kind of structured analysis of this kind? Do you perhaps know of a web2-based project like yours, or a similar one in a different blockchain ecosystem that would suggest that this is a viable market? Please note that we have now supported several projects with related concepts, like compute marketplaces and decentralized storage provider infrastructure, none of which has ever gained any significant user base.
- Have you done any calculations on what size output is economical to put on-chain, and which one to put on IPFS? Why would you store the results on-chain in the first place? This seems like a surefire way to bloat your chain, and as far as I can tell, most model queries and outputs do not require indefinite storing.
|
Thank you for your proposal and the effort you've put into it. After reviewing your application, the grant committee has decided not to move forward with it. The main reasons include concerns about the ambitious scope relative to your team's experience with Substrate, a lack of market validation, and significant technical risks—especially regarding on-chain storage and enforcement mechanisms. We appreciate your interest and hope you can secure funding elsewhere, and we wish you all the best for the future of your project. Also, you're welcome to apply to our program in the future with a different project. |
Project Abstract
AISub enables AI Suppliers to connect seamlessly to our platform, publish their models, and receive fees when users select and use their models. The platform manages all payments transparently and securely, providing suppliers with detailed usage statistics for service improvement.
Grant level
Application Checklist
project_name.md).@_______:matrix.org(change the homeserver if you use a different one)