From 0e85b3bd6905f3551215c03ec46fddcc0355b020 Mon Sep 17 00:00:00 2001 From: Tom Foster Date: Sun, 1 Sep 2024 17:48:18 +0100 Subject: [PATCH 01/51] Extended profile fields --- proposals/4133-extended-profiles.md | 351 ++++++++++++++++++++++++++++ 1 file changed, 351 insertions(+) create mode 100644 proposals/4133-extended-profiles.md diff --git a/proposals/4133-extended-profiles.md b/proposals/4133-extended-profiles.md new file mode 100644 index 00000000000..96bd08441f0 --- /dev/null +++ b/proposals/4133-extended-profiles.md @@ -0,0 +1,351 @@ +# MSC4133: Extending User Profile API with Key:Value Pairs + +*This proposal aims to enhance the usability and richness of user profiles within the Matrix +ecosystem by introducing additional, customisable fields. This feature will facilitate the sharing +of more diverse user-defined public information across the federated network. The primary goal is +to enable users to publish a wide variety of possible information, such as preferred communication +languages, pronouns, public-facing organisation roles, or other relevant public details, thereby +enriching the user interaction experience without impacting existing functionalities.* + +## Proposal + +The Matrix protocol's current user profile structure supports very limited fields (`avatar_url` and +`displayname`). This proposal suggests expanding this structure to include custom fields, +allowing for a more versatile user profile. Specifically, it redefines the existing `avatar_url` +and `displayname` endpoints to be more flexible, while attempting to maximise compatibility with +existing clients and servers to help speed adoption. + +Likewise, this proposal is designed to complement rather than replace +[MSC1769](https://github.com/matrix-org/matrix-spec-proposals/pull/1769) (Extensible profiles as +rooms). While [MSC1769](https://github.com/matrix-org/matrix-spec-proposals/pull/1769) offers a more +complex solution for extensible profiles, this proposal focuses on enabling the storage of small, +arbitrary key:value pairs at the global level. + +This proposal does not seek to enforce the exact content or usage of these fields but rather to add +a framework for users to have extra data that can be further clarified and extended in the future as +community usage of these fields grows. + +Homeservers could disable the ability for users to update these fields, or require a specific list +of fields, but the intention of this proposal is that users will be presented with a form to enter +their own free-text fields and values. After using these very flexible fields, the community may +opt to request a further extension to promote one or more fields to be specified per-room via +member events. + +### Client-Server API Changes + +1. **GET `/_matrix/client/v3/profile/{userId}/{key_name}`**: This endpoint will replace the existing + profile endpoints. It will return the value of the specified `key_name`: + + ```json + { + "key_name": "field_value" + } + ``` + + For example, requesting `/_matrix/client/v3/profile/@alice:matrix.org/displayname` would return: + + ```json + { + "displayname": "Alice" + } + ``` + +2. **PUT `/_matrix/client/v3/profile/{userId}/{key_name}`**: This endpoint will set the value of the + specified `key_name`: + + ```json + { + "key_name": "new_value" + } + ``` + + For example, setting `/_matrix/client/v3/profile/@alice:matrix.org/displayname` with: + + ```json + { + "displayname": "Alice Wonderland" + } + ``` + +3. **DELETE `/_matrix/client/v3/profile/{userId}/{key_name}`**: This endpoint will remove the key + (and associated value) from the profile, if permitted by the homeserver. Could be considered a + partial alternative to [MSC3754](https://github.com/matrix-org/matrix-spec-proposals/pull/3754) + which specifies `DELETE` endpoints for specifically `/avatar_url` and `/displayname`. + +4. **GET `/_matrix/client/v3/profile/{userId}`**: This endpoint will retrieve all profile fields: + + ```json + { + "avatar_url": "mxc://matrix.org/MyC00lAvatar", + "displayname": "John Doe", + "u.Custom Field": "value1", + "m.allowed_list": ["value2", "value3"] + } + ``` + +5. **PATCH `/_matrix/client/v3/profile/{userId}`**: This endpoint will accept a complete JSON object + to *merge* into the current profile, updating any changed keys without removing/changing any + absent ones: + + ```json + { + "avatar_url": "mxc://matrix.org/MyNewAvatar", + "displayname": "John Doe", + "u.Custom Field": "new_value1", + "m.allowed_list": ["new_value2", "new_value3"] + } + ``` + +6. **PUT `/_matrix/client/v3/profile/{userId}`**: This endpoint will accept a complete JSON object + to replace the entire profile, not only adding/updating any changed keys, but removing any + absent ones in the process: + + ```json + { + "avatar_url": "mxc://matrix.org/MyNewAvatar", + "displayname": "John Doe", + "u.Custom Field": "new_value1", + "m.allowed_list": ["new_value2", "new_value3"] + } + ``` + + **Note**: User-interactive clients are encouraged to manipulate fields individually to avoid + race conditions, however this `PUT` method allows single-client accounts (such as bots) to + overwrite the entire profile in a single request, which allows bridge bots managing many + accounts to bulk update profiles for their users with minimal requests. + +### Server-Server API Changes + +**GET `/_matrix/federation/v1/query/profile`** will mirror the client-server API changes +to ensure profile information is consistently available across the federated network. + +As there is no method to verify the history of these fields over federation, this endpoint must +only accept requests for local users on the current homeserver, and homeservers must only request +a profile from the homeserver specified in that user's MXID. + +As per the current stable endpoint, it accepts an optional `field` query string parameter to +request a single field. At time of writing, the Matrix specification says: + +> If no `field` was specified, the response should include the fields of the user’s profile that +> can be made public, such as the display name and avatar. + +Given this wording, homeservers currently already have the flexibility to decide whether some +fields are published over federation, and this proposal continues to allow this behaviour. + +### Capabilities + +A new capability `m.profile_fields` will be introduced to control the ability to set custom +profile fields. + +For backwards compatibility purposes, clients should assume these extended endpoints are not +supported when this capability is missing. + +When the capability exists but is set to `false`, clients should expect to read/display extended +fields, but should expect the server to deny creating/updating any custom fields. When this is set +to `true` it should be possible to create/update custom fields, but individual updates may receive +a 400/403 from the homeserver following the error codes listed below. + +Example capability object: + +```json +{ + "capabilities": { + "m.profile_fields": { + "enabled": false + } + } +} +``` + +### Error Handling + +To ensure clear communication of issues, the following error codes and messages will be used: + +- **400 Bad Request**: When the request is malformed, exceeds specified limits, or the profile + JSON object is larger than 64KiB. + - **Error Code for Malformed Request**: `M_BAD_JSON` + + ```json + { + "errcode": "M_BAD_JSON", + "error": "The provided JSON is malformed." + } + ``` + + - **Error Code for Exceeding Size Limit**: `M_TOO_LARGE` + + ```json + { + "errcode": "M_TOO_LARGE", + "error": "The profile data exceeds the maximum allowed size of 64KiB." + } + ``` + + - **Error Code for Invalid Data**: `M_INVALID_PARAM` + + ```json + { + "errcode": "M_INVALID_PARAM", + "error": "The key name exceeds the maximum allowed length of 255 bytes." + } + ``` + +- **403 Forbidden**: When the user does not have permission to take this action on a specific key, + such as when the server policy (e.g. + [MSC4170](https://github.com/matrix-org/matrix-spec-proposals/pull/4170)) restricts such actions: + - **Error Code**: `M_FORBIDDEN` + + ```json + { + "errcode": "M_FORBIDDEN", + "error": "You do not have permission to modify this field." + } + ``` + +- **404 Not Found**: When attempting to `GET` or `DELETE` a profile key that does not exist: + - **Error Code**: `M_NOT_FOUND` + + ```json + { + "errcode": "M_NOT_FOUND", + "error": "The requested profile key does not exist." + } + ``` + +### Propagation of Profile Fields to Membership Events + +The existing fields, `avatar_url` and `displayname`, will continue to trigger state events in each +room. These fields are replicated per-room via member events. Other fields, however, will **not** +trigger state events in rooms. They will exist solely at the global level and are intended for +storing metadata about the user that does not need to be replicated in each room. + +- **avatar_url** and **displayname**: Changes to these fields will generate state events in all + rooms the user is a member of. +- **Custom fields**: These are stored in the user's global profile and do not generate state events + in rooms. + +### Key/Namespace Requirements for Custom Fields + +Homeservers are not expected to enforce these namespaces, as future expansions may be unknown to +the server, but clients are expected to use the correct namespace for field creation/updates. + +The namespace for field names is defined as follows: + +- The namespace `m.*` is reserved for fields defined in the Matrix specification. This field may + have special entry/display requirements that are defined in the Matrix specification. If a client + does not recognise a field in this namespace, it may attempt to display it, but should not + attempt to update the content in case it has special requirements. +- The namespace `u.*` is reserved for user-defined fields. The portion of the string after the `u.` + is defined the display name of this field. These user-defined fields will always be string format. +- Client-specific or unstable fields MUST use the Java package naming convention: `tld.name.*`. + +Following support for this specification change, a user could enter a "My Timezone" field manually +in their client and the client would be expected to add a `u.My Timezone` key in their profile. +However, this would be expected to be a string value, even if the user types "null" or an integer +into the box. + +In contrast, [MSC4175](https://github.com/matrix-org/matrix-spec-proposals/pull/4175) would use the +unstable key `us.cloke.msc4175.tz` and following approval would then support clients using the +`m.tz` key with the values/validation that MSC requires. + +### Size Limit + +The key *must* be a string of *at least* one character, and *must* not exceed 255 bytes. + +To ensure efficient handling and storage of profile data, this proposal requires the entire user +profile JSON object not exceed 64KiB. + +A future MSC may increase this limit or add exceptions, but this current limit has been chosen to +allow both servers and clients to have predictable upper limits, especially for caching. + +Homeservers are allowed to limit the fields (or content) that their local users can set, setting +an upper size limit per field and/or for the entire profile, and may also limit the maximum number +of keys a user may set. + +### Implementation Details + +- This feature will be implemented as optional but recommended, enabling a smooth transition and + minimal disruption to existing deployments. +- The profile data will be public by default, and compliance with GDPR and other privacy + regulations will be enforced, particularly in terms of data deletion and retention policies. +- Custom fields will not trigger state events in rooms, maintaining account-wide metadata without + creating state events or other moderation issues. + +## Potential Issues + +Initial challenges are primarily ensuring uniform support for custom fields across different +servers and clients during the rollout phase: users may come to expect that other users will check +their supported languages before communicating with them, but the sender's server does not support +custom profiles. + +As such, this MSC is designed to be as simple as possible to get initial functionality and data +structures implemented widely in both clients and servers, so further extensions can be debated, +implemented, and tested over time in the future. + +As this data is stored only at the global level, it won't allow users to modify fields per-room, +or track historical changes in profile fields. However, this is for performance and moderation +reasons, as many users will struggle to maintain many fields of personal data between different +rooms, and publishing state events for every field change could become an additional burden on +servers and moderators. + +In a similar vein, this proposal offers no method to "broadcast" to other users or homeservers that +changes have occurred. This is intentional to keep the scope of this change narrow and maximise +compatibility with existing servers. A future proposal may wish to use an EDU (such as Presence) to +notify users and homeservers that these custom fields have been updated. + +## Alternatives + +An alternative could be to allow for more specific endpoint modifications or to introduce a +completely new API specifically for extended information. However, these approaches could lead +to more significant changes to the existing API and higher complexity for client developers. + +## Security Considerations + +Since the profile data is public, there are minimal security risks associated with the transmission +of sensitive information; however, it is critical to ensure that all data handled through these +endpoints complies with GDPR and other privacy regulations, particularly in the event of user data +deletion. + +It is crucial to ensure that clients inform users this data will be public, does not include +sensitive personal information, and complies with legal frameworks such as GDPR. Homeservers will +be encouraged to implement data caching strategies that do not exceed 24 hours to minimise the risk +of unintended data persistence. + +## Unstable prefix + +The [current Matrix specification](https://spec.matrix.org/v1.10/#profiles) technically already +allows extra custom fields to be published in a user's profile, however as this proposal introduces +additional requirements and allows custom user-defined fields, an unstable prefix should be used on +these fields until this proposal has entered the API as stable: + +```json +{ + "avatar_url": "mxc://matrix.org/MyC00lAvatar", + "displayname": "John Doe", + "uk.tcpip.msc4133.u.Custom Field": "field_value", + "uk.tcpip.msc4133.m.allowed_list": ["one value", "another value"] +} +``` + +`/_matrix/client/unstable/uk.tcpip.msc4133/profile/{userId}` would be necessary for the `PATCH` and `PUT` +methods allowed when unstable capability (detailed below) is advertised by the server. + +The existing `GET` method would act as normal and remain on `/_matrix/client/v3/profile/{userId}` +without *need* for an unstable endpoint. + +Likewise, when the unstable capability is advertised by the server, the server should accept the +new key endpoints `/_matrix/client/unstable/uk.tcpip.msc4133/profile/{userId}/{key_name}` which +would then promote to `/_matrix/client/v3/profile/{userId}/{key_name}` when the stable capability +is advertised following this specification change. + +The client capability `m.profile_fields` should use this prefix until stable: + +```json +{ + "capabilities": { + "uk.tcpip.msc4133.profile_fields": { + "enabled": false + } + } +} +``` From 0f373dbdc42b3d4803d2bbd1eda19892bd79b47b Mon Sep 17 00:00:00 2001 From: Tom Foster Date: Sun, 1 Sep 2024 18:26:45 +0100 Subject: [PATCH 02/51] Unstable client features --- proposals/4133-extended-profiles.md | 38 +++++++++++++++++++++++++---- 1 file changed, 33 insertions(+), 5 deletions(-) diff --git a/proposals/4133-extended-profiles.md b/proposals/4133-extended-profiles.md index 96bd08441f0..dc7ae31f621 100644 --- a/proposals/4133-extended-profiles.md +++ b/proposals/4133-extended-profiles.md @@ -311,12 +311,14 @@ sensitive personal information, and complies with legal frameworks such as GDPR. be encouraged to implement data caching strategies that do not exceed 24 hours to minimise the risk of unintended data persistence. -## Unstable prefix +## Unstable Prefixes -The [current Matrix specification](https://spec.matrix.org/v1.10/#profiles) technically already -allows extra custom fields to be published in a user's profile, however as this proposal introduces -additional requirements and allows custom user-defined fields, an unstable prefix should be used on -these fields until this proposal has entered the API as stable: +### Unstable Profile Fields + +The current Matrix specification technically already allows extra custom fields to be published in +a user's profile, however as this proposal introduces additional requirements and allows custom +user-defined fields, an unstable prefix should be used on these fields until this proposal has +entered the API as stable: ```json { @@ -327,6 +329,8 @@ these fields until this proposal has entered the API as stable: } ``` +### Unstable Endpoints + `/_matrix/client/unstable/uk.tcpip.msc4133/profile/{userId}` would be necessary for the `PATCH` and `PUT` methods allowed when unstable capability (detailed below) is advertised by the server. @@ -338,6 +342,8 @@ new key endpoints `/_matrix/client/unstable/uk.tcpip.msc4133/profile/{userId}/{k would then promote to `/_matrix/client/v3/profile/{userId}/{key_name}` when the stable capability is advertised following this specification change. +### Unstable Client Capability + The client capability `m.profile_fields` should use this prefix until stable: ```json @@ -349,3 +355,25 @@ The client capability `m.profile_fields` should use this prefix until stable: } } ``` + +### Unstable Client Features + +The client feature `uk.tcpip.msc4133` should be advertised on the `/_matrix/client/versions` +endpoint when the `PUT` and `PATCH` methods are accepted on the +`/_matrix/client/unstable/uk.tcpip.msc4133/profile/{userId}` endpoint. + +Once this MSC is merged, the client feature `uk.tcpip.msc4133.stable` should be advertised when the +`PUT` and `PATCH` methods are accepted on the `/_matrix/client/v3/profile/{userId}` endpoint until +the next spec version where this endpoint is officially written into the spec, e.g. + +```json +{ + "unstable_features": { + "uk.tcpip.msc4133": true, + "uk.tcpip.msc4133.stable": true + }, + "versions": [ + "v1.11" + ] +} +``` From c5a30157e624fc91148423f7b0b169ab64035e94 Mon Sep 17 00:00:00 2001 From: Tom Foster Date: Thu, 5 Sep 2024 19:34:51 +0100 Subject: [PATCH 03/51] Clarification on limits --- proposals/4133-extended-profiles.md | 11 +++++++---- 1 file changed, 7 insertions(+), 4 deletions(-) diff --git a/proposals/4133-extended-profiles.md b/proposals/4133-extended-profiles.md index dc7ae31f621..971af097c28 100644 --- a/proposals/4133-extended-profiles.md +++ b/proposals/4133-extended-profiles.md @@ -250,7 +250,7 @@ unstable key `us.cloke.msc4175.tz` and following approval would then support cli ### Size Limit -The key *must* be a string of *at least* one character, and *must* not exceed 255 bytes. +The key *must* be a string of *at least* one byte, and *must* not exceed 255 bytes. To ensure efficient handling and storage of profile data, this proposal requires the entire user profile JSON object not exceed 64KiB. @@ -258,9 +258,12 @@ profile JSON object not exceed 64KiB. A future MSC may increase this limit or add exceptions, but this current limit has been chosen to allow both servers and clients to have predictable upper limits, especially for caching. -Homeservers are allowed to limit the fields (or content) that their local users can set, setting -an upper size limit per field and/or for the entire profile, and may also limit the maximum number -of keys a user may set. +Homeservers may limit the fields (or content) that their local users can set, setting a size limit +per field and/or for the entire profile, and may limit the maximum number of keys a user may set. + +For example, if a homeserver implementation finds it more efficient to measure limits per value, it +could limit users to 50 keys with a 1024 byte limit per value, which would allow a user to set the +50 of the maximum key and value lengths without needing to calculate against the 64KiB total limit. ### Implementation Details From c8a5a1a7d2da613addebf4b3b31b03a4a26b8c5a Mon Sep 17 00:00:00 2001 From: Tom Foster Date: Thu, 5 Sep 2024 19:38:07 +0100 Subject: [PATCH 04/51] Warning about avatar_url and displayname limits --- proposals/4133-extended-profiles.md | 3 +++ 1 file changed, 3 insertions(+) diff --git a/proposals/4133-extended-profiles.md b/proposals/4133-extended-profiles.md index 971af097c28..d4b299446ad 100644 --- a/proposals/4133-extended-profiles.md +++ b/proposals/4133-extended-profiles.md @@ -265,6 +265,9 @@ For example, if a homeserver implementation finds it more efficient to measure l could limit users to 50 keys with a 1024 byte limit per value, which would allow a user to set the 50 of the maximum key and value lengths without needing to calculate against the 64KiB total limit. +**Note:** The existing `avatar_url` and `displayname` keys are contained within the profile, so if +an implementation enforces a per-value limit on fields, it must also enforce them upon these fields. + ### Implementation Details - This feature will be implemented as optional but recommended, enabling a smooth transition and From 3157982521a857dc76722ebb82600a0ef4103329 Mon Sep 17 00:00:00 2001 From: Tom Foster Date: Thu, 5 Sep 2024 20:12:09 +0100 Subject: [PATCH 05/51] Error code clarification --- proposals/4133-extended-profiles.md | 18 ++++++++++++++++++ 1 file changed, 18 insertions(+) diff --git a/proposals/4133-extended-profiles.md b/proposals/4133-extended-profiles.md index d4b299446ad..4839a85b227 100644 --- a/proposals/4133-extended-profiles.md +++ b/proposals/4133-extended-profiles.md @@ -181,6 +181,15 @@ To ensure clear communication of issues, the following error codes and messages } ``` + or + + ```json + { + "errcode": "M_TOO_LARGE", + "error": "Profile field u.Bio exceeds maximum allowed size of 1024 bytes." + } + ``` + - **Error Code for Invalid Data**: `M_INVALID_PARAM` ```json @@ -190,6 +199,15 @@ To ensure clear communication of issues, the following error codes and messages } ``` + - **Error Code for Exceeding Resource Limits**: `M_RESOURCE_LIMIT_EXCEEDED` + + ```json + { + "errcode": "M_RESOURCE_LIMIT_EXCEEDED", + "error": "The user has exceeded the maximum number of allowed keys in their profile." + } + ``` + - **403 Forbidden**: When the user does not have permission to take this action on a specific key, such as when the server policy (e.g. [MSC4170](https://github.com/matrix-org/matrix-spec-proposals/pull/4170)) restricts such actions: From b8ed87ada2087e5b7fac65be539e55666032eead Mon Sep 17 00:00:00 2001 From: Tom Foster Date: Fri, 6 Sep 2024 20:15:33 +0100 Subject: [PATCH 06/51] Whitespace fixes --- proposals/4133-extended-profiles.md | 217 ++++++++++++++-------------- 1 file changed, 110 insertions(+), 107 deletions(-) diff --git a/proposals/4133-extended-profiles.md b/proposals/4133-extended-profiles.md index 4839a85b227..759addfa25b 100644 --- a/proposals/4133-extended-profiles.md +++ b/proposals/4133-extended-profiles.md @@ -36,36 +36,36 @@ member events. 1. **GET `/_matrix/client/v3/profile/{userId}/{key_name}`**: This endpoint will replace the existing profile endpoints. It will return the value of the specified `key_name`: - ```json - { - "key_name": "field_value" - } - ``` +```json +{ + "key_name": "field_value" +} +``` - For example, requesting `/_matrix/client/v3/profile/@alice:matrix.org/displayname` would return: +For example, requesting `/_matrix/client/v3/profile/@alice:matrix.org/displayname` would return: - ```json - { - "displayname": "Alice" - } - ``` +```json +{ + "displayname": "Alice" +} +``` 2. **PUT `/_matrix/client/v3/profile/{userId}/{key_name}`**: This endpoint will set the value of the specified `key_name`: - ```json - { - "key_name": "new_value" - } - ``` +```json +{ + "key_name": "new_value" +} +``` - For example, setting `/_matrix/client/v3/profile/@alice:matrix.org/displayname` with: +For example, setting `/_matrix/client/v3/profile/@alice:matrix.org/displayname` with: - ```json - { - "displayname": "Alice Wonderland" - } - ``` +```json +{ + "displayname": "Alice Wonderland" +} +``` 3. **DELETE `/_matrix/client/v3/profile/{userId}/{key_name}`**: This endpoint will remove the key (and associated value) from the profile, if permitted by the homeserver. Could be considered a @@ -74,45 +74,45 @@ member events. 4. **GET `/_matrix/client/v3/profile/{userId}`**: This endpoint will retrieve all profile fields: - ```json - { - "avatar_url": "mxc://matrix.org/MyC00lAvatar", - "displayname": "John Doe", - "u.Custom Field": "value1", - "m.allowed_list": ["value2", "value3"] - } - ``` +```json +{ + "avatar_url": "mxc://matrix.org/MyC00lAvatar", + "displayname": "John Doe", + "u.Custom Field": "value1", + "m.allowed_list": ["value2", "value3"] +} +``` 5. **PATCH `/_matrix/client/v3/profile/{userId}`**: This endpoint will accept a complete JSON object to *merge* into the current profile, updating any changed keys without removing/changing any absent ones: - ```json - { - "avatar_url": "mxc://matrix.org/MyNewAvatar", - "displayname": "John Doe", - "u.Custom Field": "new_value1", - "m.allowed_list": ["new_value2", "new_value3"] - } - ``` +```json +{ + "avatar_url": "mxc://matrix.org/MyNewAvatar", + "displayname": "John Doe", + "u.Custom Field": "new_value1", + "m.allowed_list": ["new_value2", "new_value3"] +} +``` 6. **PUT `/_matrix/client/v3/profile/{userId}`**: This endpoint will accept a complete JSON object to replace the entire profile, not only adding/updating any changed keys, but removing any absent ones in the process: - ```json - { - "avatar_url": "mxc://matrix.org/MyNewAvatar", - "displayname": "John Doe", - "u.Custom Field": "new_value1", - "m.allowed_list": ["new_value2", "new_value3"] - } - ``` +```json +{ + "avatar_url": "mxc://matrix.org/MyNewAvatar", + "displayname": "John Doe", + "u.Custom Field": "new_value1", + "m.allowed_list": ["new_value2", "new_value3"] +} +``` - **Note**: User-interactive clients are encouraged to manipulate fields individually to avoid - race conditions, however this `PUT` method allows single-client accounts (such as bots) to - overwrite the entire profile in a single request, which allows bridge bots managing many - accounts to bulk update profiles for their users with minimal requests. +**Note**: User-interactive clients are encouraged to manipulate fields individually to avoid race +conditions, however this `PUT` method allows single-client accounts (such as bots) to overwrite the +entire profile in a single request, which allows bridge bots managing many accounts to bulk update +profiles for their users with minimal requests. ### Server-Server API Changes @@ -126,7 +126,7 @@ a profile from the homeserver specified in that user's MXID. As per the current stable endpoint, it accepts an optional `field` query string parameter to request a single field. At time of writing, the Matrix specification says: -> If no `field` was specified, the response should include the fields of the user’s profile that +> If no `field` was specified, the response should include the fields of the user's profile that > can be made public, such as the display name and avatar. Given this wording, homeservers currently already have the flexibility to decide whether some @@ -161,74 +161,77 @@ Example capability object: To ensure clear communication of issues, the following error codes and messages will be used: -- **400 Bad Request**: When the request is malformed, exceeds specified limits, or the profile - JSON object is larger than 64KiB. - - **Error Code for Malformed Request**: `M_BAD_JSON` +**400 Bad Request**: When the request is malformed, exceeds specified limits, or the profile JSON +object is larger than 64KiB: - ```json - { - "errcode": "M_BAD_JSON", - "error": "The provided JSON is malformed." - } - ``` +- **Error Code for Malformed Request**: `M_BAD_JSON` - - **Error Code for Exceeding Size Limit**: `M_TOO_LARGE` +```json +{ + "errcode": "M_BAD_JSON", + "error": "The provided JSON is malformed." +} +``` - ```json - { - "errcode": "M_TOO_LARGE", - "error": "The profile data exceeds the maximum allowed size of 64KiB." - } - ``` +- **Error Code for Exceeding Size Limit**: `M_TOO_LARGE` - or +```json +{ + "errcode": "M_TOO_LARGE", + "error": "The profile data exceeds the maximum allowed size of 64KiB." +} +``` - ```json - { - "errcode": "M_TOO_LARGE", - "error": "Profile field u.Bio exceeds maximum allowed size of 1024 bytes." - } - ``` +or - - **Error Code for Invalid Data**: `M_INVALID_PARAM` +```json +{ + "errcode": "M_TOO_LARGE", + "error": "Profile field u.Bio exceeds maximum allowed size of 1024 bytes." +} +``` - ```json - { - "errcode": "M_INVALID_PARAM", - "error": "The key name exceeds the maximum allowed length of 255 bytes." - } - ``` +- **Error Code for Invalid Data**: `M_INVALID_PARAM` + +```json +{ + "errcode": "M_INVALID_PARAM", + "error": "The key name exceeds the maximum allowed length of 255 bytes." +} +``` - - **Error Code for Exceeding Resource Limits**: `M_RESOURCE_LIMIT_EXCEEDED` +- **Error Code for Exceeding Resource Limits**: `M_RESOURCE_LIMIT_EXCEEDED` - ```json - { - "errcode": "M_RESOURCE_LIMIT_EXCEEDED", - "error": "The user has exceeded the maximum number of allowed keys in their profile." - } - ``` +```json +{ + "errcode": "M_RESOURCE_LIMIT_EXCEEDED", + "error": "The user has exceeded the maximum number of allowed keys in their profile." +} +``` -- **403 Forbidden**: When the user does not have permission to take this action on a specific key, - such as when the server policy (e.g. - [MSC4170](https://github.com/matrix-org/matrix-spec-proposals/pull/4170)) restricts such actions: - - **Error Code**: `M_FORBIDDEN` +**403 Forbidden**: When the user does not have permission to take this action on a specific key, +such as when the server policy (e.g. +[MSC4170](https://github.com/matrix-org/matrix-spec-proposals/pull/4170)) restricts such actions: - ```json - { - "errcode": "M_FORBIDDEN", - "error": "You do not have permission to modify this field." - } - ``` +- **Error Code**: `M_FORBIDDEN` -- **404 Not Found**: When attempting to `GET` or `DELETE` a profile key that does not exist: - - **Error Code**: `M_NOT_FOUND` +```json +{ + "errcode": "M_FORBIDDEN", + "error": "You do not have permission to modify this field." +} +``` - ```json - { - "errcode": "M_NOT_FOUND", - "error": "The requested profile key does not exist." - } - ``` +**404 Not Found**: When attempting to `GET` or `DELETE` a profile key that does not exist: + +- **Error Code**: `M_NOT_FOUND` + +```json +{ + "errcode": "M_NOT_FOUND", + "error": "The requested profile key does not exist." +} +``` ### Propagation of Profile Fields to Membership Events @@ -304,7 +307,7 @@ custom profiles. As such, this MSC is designed to be as simple as possible to get initial functionality and data structures implemented widely in both clients and servers, so further extensions can be debated, -implemented, and tested over time in the future. +implemented, and tested over time. As this data is stored only at the global level, it won't allow users to modify fields per-room, or track historical changes in profile fields. However, this is for performance and moderation From a81f21f8eff4023b7acffd7442ee913caed4b6a9 Mon Sep 17 00:00:00 2001 From: Tom Foster Date: Wed, 11 Sep 2024 20:32:39 +0100 Subject: [PATCH 07/51] Adjusted size limits --- proposals/4133-extended-profiles.md | 79 ++++++++++++++--------------- 1 file changed, 38 insertions(+), 41 deletions(-) diff --git a/proposals/4133-extended-profiles.md b/proposals/4133-extended-profiles.md index 759addfa25b..654efa14d2d 100644 --- a/proposals/4133-extended-profiles.md +++ b/proposals/4133-extended-profiles.md @@ -78,8 +78,7 @@ For example, setting `/_matrix/client/v3/profile/@alice:matrix.org/displayname` { "avatar_url": "mxc://matrix.org/MyC00lAvatar", "displayname": "John Doe", - "u.Custom Field": "value1", - "m.allowed_list": ["value2", "value3"] + "u.Custom Field": "value1" } ``` @@ -91,8 +90,7 @@ For example, setting `/_matrix/client/v3/profile/@alice:matrix.org/displayname` { "avatar_url": "mxc://matrix.org/MyNewAvatar", "displayname": "John Doe", - "u.Custom Field": "new_value1", - "m.allowed_list": ["new_value2", "new_value3"] + "u.Custom Field": "new_value1" } ``` @@ -104,8 +102,7 @@ For example, setting `/_matrix/client/v3/profile/@alice:matrix.org/displayname` { "avatar_url": "mxc://matrix.org/MyNewAvatar", "displayname": "John Doe", - "u.Custom Field": "new_value1", - "m.allowed_list": ["new_value2", "new_value3"] + "u.Custom Field": "new_value1" } ``` @@ -164,7 +161,7 @@ To ensure clear communication of issues, the following error codes and messages **400 Bad Request**: When the request is malformed, exceeds specified limits, or the profile JSON object is larger than 64KiB: -- **Error Code for Malformed Request**: `M_BAD_JSON` +- **Error code for malformed request**: `M_BAD_JSON` ```json { @@ -173,7 +170,7 @@ object is larger than 64KiB: } ``` -- **Error Code for Exceeding Size Limit**: `M_TOO_LARGE` +- **Error code for profile/field exceeding size limit**: `M_TOO_LARGE` ```json { @@ -187,33 +184,33 @@ or ```json { "errcode": "M_TOO_LARGE", - "error": "Profile field u.Bio exceeds maximum allowed size of 1024 bytes." + "error": "Profile field u.Bio exceeds maximum allowed size of 512 bytes." } ``` -- **Error Code for Invalid Data**: `M_INVALID_PARAM` +or ```json { - "errcode": "M_INVALID_PARAM", - "error": "The key name exceeds the maximum allowed length of 255 bytes." + "errcode": "M_TOO_LARGE", + "error": "The key name exceeds the maximum allowed length of 128 bytes." } ``` -- **Error Code for Exceeding Resource Limits**: `M_RESOURCE_LIMIT_EXCEEDED` +**403 Forbidden**: When the user does not have permission to take this action on a specific key, +such as when the server policy (e.g. +[MSC4170](https://github.com/matrix-org/matrix-spec-proposals/pull/4170)) restricts such actions: + +- **Error code refusing to create additional keys**: `M_FORBIDDEN` ```json { - "errcode": "M_RESOURCE_LIMIT_EXCEEDED", + "errcode": "M_FORBIDDEN", "error": "The user has exceeded the maximum number of allowed keys in their profile." } ``` -**403 Forbidden**: When the user does not have permission to take this action on a specific key, -such as when the server policy (e.g. -[MSC4170](https://github.com/matrix-org/matrix-spec-proposals/pull/4170)) restricts such actions: - -- **Error Code**: `M_FORBIDDEN` +- **Error code when not allowed to modify key**: `M_FORBIDDEN` ```json { @@ -236,9 +233,9 @@ such as when the server policy (e.g. ### Propagation of Profile Fields to Membership Events The existing fields, `avatar_url` and `displayname`, will continue to trigger state events in each -room. These fields are replicated per-room via member events. Other fields, however, will **not** -trigger state events in rooms. They will exist solely at the global level and are intended for -storing metadata about the user that does not need to be replicated in each room. +room. These fields are replicated per-room via member events. Custom fields, however, will **not** +trigger state events in rooms and will exist solely at the global level for storing metadata about +the user. - **avatar_url** and **displayname**: Changes to these fields will generate state events in all rooms the user is a member of. @@ -269,25 +266,24 @@ In contrast, [MSC4175](https://github.com/matrix-org/matrix-spec-proposals/pull/ unstable key `us.cloke.msc4175.tz` and following approval would then support clients using the `m.tz` key with the values/validation that MSC requires. -### Size Limit - -The key *must* be a string of *at least* one byte, and *must* not exceed 255 bytes. +### Size Limits -To ensure efficient handling and storage of profile data, this proposal requires the entire user -profile JSON object not exceed 64KiB. +Until another MSC specifies otherwise: -A future MSC may increase this limit or add exceptions, but this current limit has been chosen to -allow both servers and clients to have predictable upper limits, especially for caching. +- Each profile *must* be *at most* 64KiB in size +- Each profile may contain *up to* 50 keys (including `avatar_url` and `displayname`) +- Each key *must* be a string of *at least* one byte, and *must* not exceed 128 bytes +- Each value *must* be a string and *must* not exceed 512 bytes -Homeservers may limit the fields (or content) that their local users can set, setting a size limit -per field and/or for the entire profile, and may limit the maximum number of keys a user may set. +These sizes are measured *after* being escaped for JSON, e.g. `🛸` would become `\uD83D\uDEF8`. -For example, if a homeserver implementation finds it more efficient to measure limits per value, it -could limit users to 50 keys with a 1024 byte limit per value, which would allow a user to set the -50 of the maximum key and value lengths without needing to calculate against the 64KiB total limit. +This 64KiB size limit should be calculated on the full canonicalised JSON payload a homeserver +would receive when retrieving the full profile, so a `Content-Length` header on the HTTP response +must not exceed 65536 bytes. -**Note:** The existing `avatar_url` and `displayname` keys are contained within the profile, so if -an implementation enforces a per-value limit on fields, it must also enforce them upon these fields. +Future MSCs may allow fields that are not string values, or add exceptions to the size limits, +but these current limits have been chosen to allow both servers and clients to have predictable +upper limits for performance and caching purposes. ### Implementation Details @@ -297,6 +293,8 @@ an implementation enforces a per-value limit on fields, it must also enforce the regulations will be enforced, particularly in terms of data deletion and retention policies. - Custom fields will not trigger state events in rooms, maintaining account-wide metadata without creating state events or other moderation issues. +- The existing `avatar_url` and `displayname` keys are contained within the profile, so if they are + currently longer than 512 bytes, they will need to be truncated to 512 bytes. ## Potential Issues @@ -343,7 +341,7 @@ of unintended data persistence. ### Unstable Profile Fields The current Matrix specification technically already allows extra custom fields to be published in -a user's profile, however as this proposal introduces additional requirements and allows custom +a user's profile. However, as this proposal introduces additional requirements and allows custom user-defined fields, an unstable prefix should be used on these fields until this proposal has entered the API as stable: @@ -351,15 +349,14 @@ entered the API as stable: { "avatar_url": "mxc://matrix.org/MyC00lAvatar", "displayname": "John Doe", - "uk.tcpip.msc4133.u.Custom Field": "field_value", - "uk.tcpip.msc4133.m.allowed_list": ["one value", "another value"] + "uk.tcpip.msc4133.u.Custom Field": "field_value" } ``` ### Unstable Endpoints -`/_matrix/client/unstable/uk.tcpip.msc4133/profile/{userId}` would be necessary for the `PATCH` and `PUT` -methods allowed when unstable capability (detailed below) is advertised by the server. +`/_matrix/client/unstable/uk.tcpip.msc4133/profile/{userId}` would be necessary for the `PATCH` and +`PUT` methods allowed when the unstable capability (detailed below) is advertised by the server. The existing `GET` method would act as normal and remain on `/_matrix/client/v3/profile/{userId}` without *need* for an unstable endpoint. From e688eb137a7368c0ec71cac1566a22baa9204fdc Mon Sep 17 00:00:00 2001 From: Tom Foster Date: Wed, 11 Sep 2024 20:58:35 +0100 Subject: [PATCH 08/51] Clarified Canonical JSON --- proposals/4133-extended-profiles.md | 16 +++++++--------- 1 file changed, 7 insertions(+), 9 deletions(-) diff --git a/proposals/4133-extended-profiles.md b/proposals/4133-extended-profiles.md index 654efa14d2d..04bed47a58f 100644 --- a/proposals/4133-extended-profiles.md +++ b/proposals/4133-extended-profiles.md @@ -270,20 +270,18 @@ unstable key `us.cloke.msc4175.tz` and following approval would then support cli Until another MSC specifies otherwise: -- Each profile *must* be *at most* 64KiB in size +- Each profile *must* be *at most* 64KiB (65536 bytes) in size - Each profile may contain *up to* 50 keys (including `avatar_url` and `displayname`) - Each key *must* be a string of *at least* one byte, and *must* not exceed 128 bytes - Each value *must* be a string and *must* not exceed 512 bytes -These sizes are measured *after* being escaped for JSON, e.g. `🛸` would become `\uD83D\uDEF8`. +These sizes are measured as encoded in +[Canonical JSON](https://spec.matrix.org/v1.11/appendices/#canonical-json), so `🛸` would consume +two bytes of the limit. -This 64KiB size limit should be calculated on the full canonicalised JSON payload a homeserver -would receive when retrieving the full profile, so a `Content-Length` header on the HTTP response -must not exceed 65536 bytes. - -Future MSCs may allow fields that are not string values, or add exceptions to the size limits, -but these current limits have been chosen to allow both servers and clients to have predictable -upper limits for performance and caching purposes. +Future MSCs may allow fields that are not string values, or add exceptions to the limits, but these +current limits have been chosen to allow servers and clients to have predictable upper limits for +performance and caching purposes. ### Implementation Details From 39d5fa271589824bf0a88778e233710c16f928f7 Mon Sep 17 00:00:00 2001 From: Tom Foster Date: Fri, 13 Sep 2024 18:12:18 +0100 Subject: [PATCH 09/51] Clarifications --- proposals/4133-extended-profiles.md | 39 ++++++++++++++++------------- 1 file changed, 22 insertions(+), 17 deletions(-) diff --git a/proposals/4133-extended-profiles.md b/proposals/4133-extended-profiles.md index 04bed47a58f..70c318c1ae1 100644 --- a/proposals/4133-extended-profiles.md +++ b/proposals/4133-extended-profiles.md @@ -197,19 +197,19 @@ or } ``` -**403 Forbidden**: When the user does not have permission to take this action on a specific key, -such as when the server policy (e.g. -[MSC4170](https://github.com/matrix-org/matrix-spec-proposals/pull/4170)) restricts such actions: - -- **Error code refusing to create additional keys**: `M_FORBIDDEN` +or ```json { - "errcode": "M_FORBIDDEN", + "errcode": "M_TOO_LARGE", "error": "The user has exceeded the maximum number of allowed keys in their profile." } ``` +**403 Forbidden**: When the user does not have permission to take this action on a specific key, +such as when the server policy (e.g. +[MSC4170](https://github.com/matrix-org/matrix-spec-proposals/pull/4170)) restricts such actions: + - **Error code when not allowed to modify key**: `M_FORBIDDEN` ```json @@ -254,7 +254,9 @@ The namespace for field names is defined as follows: does not recognise a field in this namespace, it may attempt to display it, but should not attempt to update the content in case it has special requirements. - The namespace `u.*` is reserved for user-defined fields. The portion of the string after the `u.` - is defined the display name of this field. These user-defined fields will always be string format. + is defined the display name of this field. These user-defined fields will always be string + format, and as measured in UTF-8 byte counts, the key and value must not be longer than 128 bytes + and 512 bytes respectively. - Client-specific or unstable fields MUST use the Java package naming convention: `tld.name.*`. Following support for this specification change, a user could enter a "My Timezone" field manually @@ -268,20 +270,23 @@ unstable key `us.cloke.msc4175.tz` and following approval would then support cli ### Size Limits +Whenever "bytes" are referred to as a limit, this is calculated as UTF-8 bytes, so a two-byte +UTF-16 character consumes two bytes of this limit. + Until another MSC specifies otherwise: -- Each profile *must* be *at most* 64KiB (65536 bytes) in size -- Each profile may contain *up to* 50 keys (including `avatar_url` and `displayname`) -- Each key *must* be a string of *at least* one byte, and *must* not exceed 128 bytes -- Each value *must* be a string and *must* not exceed 512 bytes +- Each profile *must* be *at most* 64KiB (65536 bytes) in size, as measured in Canonical JSON +- Each key *must* be a string of *at least* one byte and *must* not exceed 128 bytes +- Each value in the `u.*` namespace *must* not exceed 512 bytes in length +- Profile size limits include `avatar_url` and `displayname` -These sizes are measured as encoded in -[Canonical JSON](https://spec.matrix.org/v1.11/appendices/#canonical-json), so `🛸` would consume -two bytes of the limit. +Future MSCs may add exceptions to these limits, but these current limits have been chosen to allow +servers and clients to have predictable upper limits for performance and caching purposes, and to +provide a more comfortable UX in clients. -Future MSCs may allow fields that are not string values, or add exceptions to the limits, but these -current limits have been chosen to allow servers and clients to have predictable upper limits for -performance and caching purposes. +As lengths and types are subject to change, implementations are encouraged to accept values of any +type acceptable in [Canonical JSON](https://spec.matrix.org/latest/appendices/#canonical-json), and +treat remote profiles as a single ≤64KiB block of Canonical JSON. ### Implementation Details From e160c7139e619a05e3fbb6378d6768b0f01b94db Mon Sep 17 00:00:00 2001 From: Tom Foster Date: Fri, 13 Sep 2024 18:55:01 +0100 Subject: [PATCH 10/51] Clarifications --- proposals/4133-extended-profiles.md | 155 +++++++++++++--------------- 1 file changed, 73 insertions(+), 82 deletions(-) diff --git a/proposals/4133-extended-profiles.md b/proposals/4133-extended-profiles.md index 70c318c1ae1..0cb900e37d6 100644 --- a/proposals/4133-extended-profiles.md +++ b/proposals/4133-extended-profiles.md @@ -10,22 +10,22 @@ enriching the user interaction experience without impacting existing functionali ## Proposal The Matrix protocol's current user profile structure supports very limited fields (`avatar_url` and -`displayname`). This proposal suggests expanding this structure to include custom fields, -allowing for a more versatile user profile. Specifically, it redefines the existing `avatar_url` -and `displayname` endpoints to be more flexible, while attempting to maximise compatibility with +`displayname`). This proposal suggests expanding this structure to include custom fields, allowing +for a more versatile user profile. Specifically, it redefines the existing `avatar_url` and +`displayname` endpoints to be more flexible, while attempting to maximise compatibility with existing clients and servers to help speed adoption. Likewise, this proposal is designed to complement rather than replace -[MSC1769](https://github.com/matrix-org/matrix-spec-proposals/pull/1769) (Extensible profiles as -rooms). While [MSC1769](https://github.com/matrix-org/matrix-spec-proposals/pull/1769) offers a more -complex solution for extensible profiles, this proposal focuses on enabling the storage of small, -arbitrary key:value pairs at the global level. +[MSC1769](https://github.com/matrix-org/matrix-spec-proposals/pull/1769) (Extensible Profiles as +Rooms). While [MSC1769](https://github.com/matrix-org/matrix-spec-proposals/pull/1769) offers a +more complex solution for extensible profiles, this proposal focuses on enabling the storage of +small, arbitrary key:value pairs at the global level. This proposal does not seek to enforce the exact content or usage of these fields but rather to add -a framework for users to have extra data that can be further clarified and extended in the future as -community usage of these fields grows. +a framework for users to have extra data that can be further clarified and extended in the future +as community usage of these fields grows. -Homeservers could disable the ability for users to update these fields, or require a specific list +Homeservers could disable the ability for users to update these fields or require a specific list of fields, but the intention of this proposal is that users will be presented with a form to enter their own free-text fields and values. After using these very flexible fields, the community may opt to request a further extension to promote one or more fields to be specified per-room via @@ -33,8 +33,8 @@ member events. ### Client-Server API Changes -1. **GET `/_matrix/client/v3/profile/{userId}/{key_name}`**: This endpoint will replace the existing - profile endpoints. It will return the value of the specified `key_name`: +1. **GET `/_matrix/client/v3/profile/{userId}/{key_name}`**: This endpoint extends the existing + profile API to allow clients to retrieve the value of a specified `key_name` in a user's profile: ```json { @@ -50,8 +50,8 @@ For example, requesting `/_matrix/client/v3/profile/@alice:matrix.org/displaynam } ``` -2. **PUT `/_matrix/client/v3/profile/{userId}/{key_name}`**: This endpoint will set the value of the - specified `key_name`: +2. **PUT `/_matrix/client/v3/profile/{userId}/{key_name}`**: This endpoint allows clients to set + the value of a specified `key_name`: ```json { @@ -67,12 +67,12 @@ For example, setting `/_matrix/client/v3/profile/@alice:matrix.org/displayname` } ``` -3. **DELETE `/_matrix/client/v3/profile/{userId}/{key_name}`**: This endpoint will remove the key - (and associated value) from the profile, if permitted by the homeserver. Could be considered a - partial alternative to [MSC3754](https://github.com/matrix-org/matrix-spec-proposals/pull/3754) - which specifies `DELETE` endpoints for specifically `/avatar_url` and `/displayname`. +1. **DELETE `/_matrix/client/v3/profile/{userId}/{key_name}`**: This endpoint removes a key (and + associated value) from the profile, if permitted by the homeserver. This could be considered a + partial alternative to [MSC3754](https://github.com/matrix-org/matrix-spec-proposals/pull/3754), + which specifies `DELETE` endpoints specifically for `/avatar_url` and `/displayname`. -4. **GET `/_matrix/client/v3/profile/{userId}`**: This endpoint will retrieve all profile fields: +2. **GET `/_matrix/client/v3/profile/{userId}`**: This endpoint retrieves all profile fields: ```json { @@ -82,9 +82,8 @@ For example, setting `/_matrix/client/v3/profile/@alice:matrix.org/displayname` } ``` -5. **PATCH `/_matrix/client/v3/profile/{userId}`**: This endpoint will accept a complete JSON object - to *merge* into the current profile, updating any changed keys without removing/changing any - absent ones: +5. **PATCH `/_matrix/client/v3/profile/{userId}`**: This endpoint accepts a complete JSON object to + *merge* into the current profile, updating any changed keys without altering any absent ones: ```json { @@ -94,9 +93,9 @@ For example, setting `/_matrix/client/v3/profile/@alice:matrix.org/displayname` } ``` -6. **PUT `/_matrix/client/v3/profile/{userId}`**: This endpoint will accept a complete JSON object - to replace the entire profile, not only adding/updating any changed keys, but removing any - absent ones in the process: +6. **PUT `/_matrix/client/v3/profile/{userId}`**: This endpoint accepts a complete JSON object to + replace the entire profile, adding or updating any changed keys and removing any keys that are + absent in the provided object: ```json { @@ -113,12 +112,12 @@ profiles for their users with minimal requests. ### Server-Server API Changes -**GET `/_matrix/federation/v1/query/profile`** will mirror the client-server API changes -to ensure profile information is consistently available across the federated network. +**GET `/_matrix/federation/v1/query/profile`** will mirror the client-server API changes to ensure +profile information is consistently available across the federated network. As there is no method to verify the history of these fields over federation, this endpoint must -only accept requests for local users on the current homeserver, and homeservers must only request -a profile from the homeserver specified in that user's MXID. +only accept requests for local users on the current homeserver, and homeservers must only request a +profile from the homeserver specified in that user's MXID. As per the current stable endpoint, it accepts an optional `field` query string parameter to request a single field. At time of writing, the Matrix specification says: @@ -131,35 +130,44 @@ fields are published over federation, and this proposal continues to allow this ### Capabilities -A new capability `m.profile_fields` will be introduced to control the ability to set custom -profile fields. +A new capability `m.profile_fields` is introduced to control the ability to set custom profile +fields. -For backwards compatibility purposes, clients should assume these extended endpoints are not -supported when this capability is missing. +- **Advertising the Capability**: Servers SHOULD advertise this capability in the `capabilities` + object returned by the `GET /_matrix/client/v3/capabilities` endpoint. -When the capability exists but is set to `false`, clients should expect to read/display extended -fields, but should expect the server to deny creating/updating any custom fields. When this is set -to `true` it should be possible to create/update custom fields, but individual updates may receive -a 400/403 from the homeserver following the error codes listed below. +- **Checking the Capability**: Clients SHOULD check for the presence and value of the + `m.profile_fields` capability before attempting to create or update custom profile fields. -Example capability object: +- **Capability Structure**: ```json { - "capabilities": { - "m.profile_fields": { - "enabled": false + "capabilities": { + "m.profile_fields": { + "enabled": true + } } - } } ``` +- **Backward Compatibility**: For backward compatibility, clients SHOULD assume that extended + profile fields are not supported if this capability is missing from the server's advertised + capabilities. + +- **When `enabled` is `false`**: Clients SHOULD expect to read and display extended fields but + SHOULD NOT allow users to create or update custom fields. Any attempt to do so may result in a + `403 Forbidden` error. + +- **When `enabled` is `true`**: Clients MAY allow users to create or update custom fields. However, + individual updates may still receive a `400 Bad Request` or `403 Forbidden` response from the + homeserver if specific server-side policies prevent the action. + ### Error Handling To ensure clear communication of issues, the following error codes and messages will be used: -**400 Bad Request**: When the request is malformed, exceeds specified limits, or the profile JSON -object is larger than 64KiB: +#### 400 Bad Request: When request is malformed, exceeds limits, or the profile larger than 64KiB - **Error code for malformed request**: `M_BAD_JSON` @@ -206,22 +214,13 @@ or } ``` -**403 Forbidden**: When the user does not have permission to take this action on a specific key, -such as when the server policy (e.g. -[MSC4170](https://github.com/matrix-org/matrix-spec-proposals/pull/4170)) restricts such actions: - -- **Error code when not allowed to modify key**: `M_FORBIDDEN` - -```json -{ - "errcode": "M_FORBIDDEN", - "error": "You do not have permission to modify this field." -} -``` +#### 403 Forbidden: User lacks permission to take this action (e.g. restricted by server policy) -**404 Not Found**: When attempting to `GET` or `DELETE` a profile key that does not exist: +**Note:** See [MSC4170](https://github.com/matrix-org/matrix-spec-proposals/pull/4170) for more +discussion on how server policy may result in 403 errors for profile requests. -- **Error Code**: `M_NOT_FOUND` +- **404 Not Found**: When attempting to `GET` or `DELETE` a profile key that does not exist: + - **Error Code**: `M_NOT_FOUND` ```json { @@ -247,14 +246,12 @@ the user. Homeservers are not expected to enforce these namespaces, as future expansions may be unknown to the server, but clients are expected to use the correct namespace for field creation/updates. -The namespace for field names is defined as follows: - - The namespace `m.*` is reserved for fields defined in the Matrix specification. This field may have special entry/display requirements that are defined in the Matrix specification. If a client does not recognise a field in this namespace, it may attempt to display it, but should not attempt to update the content in case it has special requirements. - The namespace `u.*` is reserved for user-defined fields. The portion of the string after the `u.` - is defined the display name of this field. These user-defined fields will always be string + is defined as the display name of this field. These user-defined fields will always be string format, and as measured in UTF-8 byte counts, the key and value must not be longer than 128 bytes and 512 bytes respectively. - Client-specific or unstable fields MUST use the Java package naming convention: `tld.name.*`. @@ -275,18 +272,10 @@ UTF-16 character consumes two bytes of this limit. Until another MSC specifies otherwise: -- Each profile *must* be *at most* 64KiB (65536 bytes) in size, as measured in Canonical JSON -- Each key *must* be a string of *at least* one byte and *must* not exceed 128 bytes -- Each value in the `u.*` namespace *must* not exceed 512 bytes in length -- Profile size limits include `avatar_url` and `displayname` - -Future MSCs may add exceptions to these limits, but these current limits have been chosen to allow -servers and clients to have predictable upper limits for performance and caching purposes, and to -provide a more comfortable UX in clients. - -As lengths and types are subject to change, implementations are encouraged to accept values of any -type acceptable in [Canonical JSON](https://spec.matrix.org/latest/appendices/#canonical-json), and -treat remote profiles as a single ≤64KiB block of Canonical JSON. +- Each profile *must* be *at most* 64KiB (65536 bytes) in size, as measured in Canonical JSON. +- Each key *must* be a string of *at least* one byte and *must* not exceed 128 bytes. +- Each value in the `u.*` namespace *must* not exceed 512 bytes in length. +- The limit on overall profile size includes `avatar_url` and `displayname`. ### Implementation Details @@ -327,17 +316,19 @@ An alternative could be to allow for more specific endpoint modifications or to completely new API specifically for extended information. However, these approaches could lead to more significant changes to the existing API and higher complexity for client developers. -## Security Considerations +### Security Considerations Since the profile data is public, there are minimal security risks associated with the transmission -of sensitive information; however, it is critical to ensure that all data handled through these -endpoints complies with GDPR and other privacy regulations, particularly in the event of user data -deletion. +of sensitive information; however, users should be made aware that any information they add will be +visible to others on the federated network. + +Clients *should* inform users that any custom profile fields they add will be publicly accessible, +and *should* discourage users from adding sensitive personal information to their profiles. -It is crucial to ensure that clients inform users this data will be public, does not include -sensitive personal information, and complies with legal frameworks such as GDPR. Homeservers will -be encouraged to implement data caching strategies that do not exceed 24 hours to minimise the risk -of unintended data persistence. +Homeservers and clients *must* comply with GDPR and other relevant privacy regulations, particularly +regarding data deletion and retention. Profile data *should* be cleared when a user is deactivated. +While homeservers *should* cache remote profiles, they *should* implement strategies that do not +exceed 24 hours to minimize the risk of unintended data persistence. ## Unstable Prefixes @@ -362,7 +353,7 @@ entered the API as stable: `PUT` methods allowed when the unstable capability (detailed below) is advertised by the server. The existing `GET` method would act as normal and remain on `/_matrix/client/v3/profile/{userId}` -without *need* for an unstable endpoint. +without need for an unstable endpoint. Likewise, when the unstable capability is advertised by the server, the server should accept the new key endpoints `/_matrix/client/unstable/uk.tcpip.msc4133/profile/{userId}/{key_name}` which From a9546aa358408a69b10da0f08e07461c3e825868 Mon Sep 17 00:00:00 2001 From: Tom Foster Date: Fri, 13 Sep 2024 18:57:34 +0100 Subject: [PATCH 11/51] Remove UTF-16 --- proposals/4133-extended-profiles.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/proposals/4133-extended-profiles.md b/proposals/4133-extended-profiles.md index 0cb900e37d6..bfc60d6e858 100644 --- a/proposals/4133-extended-profiles.md +++ b/proposals/4133-extended-profiles.md @@ -267,8 +267,8 @@ unstable key `us.cloke.msc4175.tz` and following approval would then support cli ### Size Limits -Whenever "bytes" are referred to as a limit, this is calculated as UTF-8 bytes, so a two-byte -UTF-16 character consumes two bytes of this limit. +Whenever "bytes" are referred to as a limit, this is calculated as UTF-8 bytes, so a two-byte code +point consumes two bytes of this limit. This size would be measured before any JSON encoding. Until another MSC specifies otherwise: From 0c43f50b327aa419c57db4db18bfaf91cbb11f96 Mon Sep 17 00:00:00 2001 From: Tom Foster Date: Fri, 13 Sep 2024 19:01:50 +0100 Subject: [PATCH 12/51] Clarify key persistence --- proposals/4133-extended-profiles.md | 3 +++ 1 file changed, 3 insertions(+) diff --git a/proposals/4133-extended-profiles.md b/proposals/4133-extended-profiles.md index bfc60d6e858..2f9c447647a 100644 --- a/proposals/4133-extended-profiles.md +++ b/proposals/4133-extended-profiles.md @@ -67,6 +67,9 @@ For example, setting `/_matrix/client/v3/profile/@alice:matrix.org/displayname` } ``` +**Note**: As the `DELETE` endpoint exists below, a `PUT` to key using a `null` value should leave +the key in place without deleting the key. + 1. **DELETE `/_matrix/client/v3/profile/{userId}/{key_name}`**: This endpoint removes a key (and associated value) from the profile, if permitted by the homeserver. This could be considered a partial alternative to [MSC3754](https://github.com/matrix-org/matrix-spec-proposals/pull/3754), From 613411a8b05481ff7e9fb5596024451f440911b8 Mon Sep 17 00:00:00 2001 From: Tom Foster Date: Fri, 13 Sep 2024 19:04:10 +0100 Subject: [PATCH 13/51] Out of date line --- proposals/4133-extended-profiles.md | 2 -- 1 file changed, 2 deletions(-) diff --git a/proposals/4133-extended-profiles.md b/proposals/4133-extended-profiles.md index 2f9c447647a..8d524fa9cda 100644 --- a/proposals/4133-extended-profiles.md +++ b/proposals/4133-extended-profiles.md @@ -288,8 +288,6 @@ Until another MSC specifies otherwise: regulations will be enforced, particularly in terms of data deletion and retention policies. - Custom fields will not trigger state events in rooms, maintaining account-wide metadata without creating state events or other moderation issues. -- The existing `avatar_url` and `displayname` keys are contained within the profile, so if they are - currently longer than 512 bytes, they will need to be truncated to 512 bytes. ## Potential Issues From 4a9557f27ba755931dcf497a3c3ee5eeb14005d3 Mon Sep 17 00:00:00 2001 From: Tom Foster Date: Mon, 16 Sep 2024 10:10:26 +0100 Subject: [PATCH 14/51] Clarify only `u.*` namespace is limited to 512 bytes --- proposals/4133-extended-profiles.md | 11 +++++++---- 1 file changed, 7 insertions(+), 4 deletions(-) diff --git a/proposals/4133-extended-profiles.md b/proposals/4133-extended-profiles.md index 8d524fa9cda..7a62ed1492d 100644 --- a/proposals/4133-extended-profiles.md +++ b/proposals/4133-extended-profiles.md @@ -253,10 +253,13 @@ the server, but clients are expected to use the correct namespace for field crea have special entry/display requirements that are defined in the Matrix specification. If a client does not recognise a field in this namespace, it may attempt to display it, but should not attempt to update the content in case it has special requirements. + - The namespace `u.*` is reserved for user-defined fields. The portion of the string after the `u.` is defined as the display name of this field. These user-defined fields will always be string - format, and as measured in UTF-8 byte counts, the key and value must not be longer than 128 bytes - and 512 bytes respectively. + format, and as measured in UTF-8 byte counts: + - The *name* of each key must not exceed 128 bytes. + - The *content* of each value must not exceed 512 bytes. + - Client-specific or unstable fields MUST use the Java package naming convention: `tld.name.*`. Following support for this specification change, a user could enter a "My Timezone" field manually @@ -276,8 +279,8 @@ point consumes two bytes of this limit. This size would be measured before any J Until another MSC specifies otherwise: - Each profile *must* be *at most* 64KiB (65536 bytes) in size, as measured in Canonical JSON. -- Each key *must* be a string of *at least* one byte and *must* not exceed 128 bytes. -- Each value in the `u.*` namespace *must* not exceed 512 bytes in length. +- Each key *must* be a string of *at least* one byte and it's name *must* not exceed 128 bytes. +- Each key's *value* in the `u.*` namespace *must* not exceed 512 bytes in length. - The limit on overall profile size includes `avatar_url` and `displayname`. ### Implementation Details From fbb4e4497b70708862c9b81f62fc11b3204dff6b Mon Sep 17 00:00:00 2001 From: Tom Foster Date: Mon, 16 Sep 2024 10:25:18 +0100 Subject: [PATCH 15/51] Include read-only fields in capability --- proposals/4133-extended-profiles.md | 9 +++++---- 1 file changed, 5 insertions(+), 4 deletions(-) diff --git a/proposals/4133-extended-profiles.md b/proposals/4133-extended-profiles.md index 7a62ed1492d..0687ab5ba5b 100644 --- a/proposals/4133-extended-profiles.md +++ b/proposals/4133-extended-profiles.md @@ -148,7 +148,8 @@ fields. { "capabilities": { "m.profile_fields": { - "enabled": true + "enabled": true, + "disallowed": ["org.example.job_title"] } } } @@ -162,9 +163,9 @@ fields. SHOULD NOT allow users to create or update custom fields. Any attempt to do so may result in a `403 Forbidden` error. -- **When `enabled` is `true`**: Clients MAY allow users to create or update custom fields. However, - individual updates may still receive a `400 Bad Request` or `403 Forbidden` response from the - homeserver if specific server-side policies prevent the action. +- **When `enabled` is `true`**: Clients MAY allow users to create or update custom fields, except + for those listed in the `disallowed` array. Individual requests will receive a `400 Bad Request` + or `403 Forbidden` response from the homeserver if specific server-side policies prevent them. ### Error Handling From 591999fe535f79d4bb131d88bc88ff086aac7305 Mon Sep 17 00:00:00 2001 From: Tom Foster Date: Mon, 16 Sep 2024 12:22:54 +0100 Subject: [PATCH 16/51] Change missing capability behaviour --- proposals/4133-extended-profiles.md | 19 ++++++++++--------- 1 file changed, 10 insertions(+), 9 deletions(-) diff --git a/proposals/4133-extended-profiles.md b/proposals/4133-extended-profiles.md index 0687ab5ba5b..64d61e1c4ac 100644 --- a/proposals/4133-extended-profiles.md +++ b/proposals/4133-extended-profiles.md @@ -155,17 +155,17 @@ fields. } ``` -- **Backward Compatibility**: For backward compatibility, clients SHOULD assume that extended - profile fields are not supported if this capability is missing from the server's advertised - capabilities. +- **When capability missing**: Clients SHOULD assume extended profile fields are supported, and + that they can be created/written to. If a server intends to deny some (or all) changes, it SHOULD + use the capabiltiy to advertise this to improve client experience. - **When `enabled` is `false`**: Clients SHOULD expect to read and display extended fields but SHOULD NOT allow users to create or update custom fields. Any attempt to do so may result in a `403 Forbidden` error. - **When `enabled` is `true`**: Clients MAY allow users to create or update custom fields, except - for those listed in the `disallowed` array. Individual requests will receive a `400 Bad Request` - or `403 Forbidden` response from the homeserver if specific server-side policies prevent them. + those listed in the `disallowed` array. Individual requests will receive a `400 Bad Request` or# + `403 Forbidden` response from the homeserver if server-side policies prevent them. ### Error Handling @@ -371,11 +371,12 @@ The client capability `m.profile_fields` should use this prefix until stable: ```json { - "capabilities": { - "uk.tcpip.msc4133.profile_fields": { - "enabled": false + "capabilities": { + "uk.tcpip.msc4133.profile_fields": { + "enabled": true, + "disallowed": ["org.example.job_title"] + } } - } } ``` From 15a0bd172e1463008cd537ed08d5448f61197a71 Mon Sep 17 00:00:00 2001 From: Tom Foster Date: Tue, 17 Sep 2024 12:08:25 +0100 Subject: [PATCH 17/51] Typo correction --- proposals/4133-extended-profiles.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/proposals/4133-extended-profiles.md b/proposals/4133-extended-profiles.md index 64d61e1c4ac..fa69f129922 100644 --- a/proposals/4133-extended-profiles.md +++ b/proposals/4133-extended-profiles.md @@ -157,14 +157,14 @@ fields. - **When capability missing**: Clients SHOULD assume extended profile fields are supported, and that they can be created/written to. If a server intends to deny some (or all) changes, it SHOULD - use the capabiltiy to advertise this to improve client experience. + use the capability to advertise this to improve client experience. - **When `enabled` is `false`**: Clients SHOULD expect to read and display extended fields but SHOULD NOT allow users to create or update custom fields. Any attempt to do so may result in a `403 Forbidden` error. - **When `enabled` is `true`**: Clients MAY allow users to create or update custom fields, except - those listed in the `disallowed` array. Individual requests will receive a `400 Bad Request` or# + those listed in the `disallowed` array. Individual requests will receive a `400 Bad Request` or `403 Forbidden` response from the homeserver if server-side policies prevent them. ### Error Handling From 26c59f7feb097d6dcf19efbe919e90e7fd53bef6 Mon Sep 17 00:00:00 2001 From: Tom Foster Date: Tue, 24 Sep 2024 17:34:57 +0100 Subject: [PATCH 18/51] Privacy clarification for T&S --- proposals/4133-extended-profiles.md | 15 +++++++++++++-- 1 file changed, 13 insertions(+), 2 deletions(-) diff --git a/proposals/4133-extended-profiles.md b/proposals/4133-extended-profiles.md index fa69f129922..b3c0768e52e 100644 --- a/proposals/4133-extended-profiles.md +++ b/proposals/4133-extended-profiles.md @@ -286,13 +286,20 @@ Until another MSC specifies otherwise: ### Implementation Details -- This feature will be implemented as optional but recommended, enabling a smooth transition and - minimal disruption to existing deployments. - The profile data will be public by default, and compliance with GDPR and other privacy regulations will be enforced, particularly in terms of data deletion and retention policies. + - Custom fields will not trigger state events in rooms, maintaining account-wide metadata without creating state events or other moderation issues. +- Clients are encouraged to provide settings for users to choose the scope of users they present + the freetext (i.e. `u.*` namespaced) fields from (e.g. none, only users on the local server, + users in the current room, any users sharing a room with this user, all users globally) with + defaults sensitive to their implementation's UI and intended audience. + +- Clients should provide a UI for users to enter their own free-text custom fields in the `u.*` + namespace of their own profile. + ## Potential Issues Initial challenges are primarily ensuring uniform support for custom fields across different @@ -335,6 +342,10 @@ regarding data deletion and retention. Profile data *should* be cleared when a u While homeservers *should* cache remote profiles, they *should* implement strategies that do not exceed 24 hours to minimize the risk of unintended data persistence. +To minimise the impact of abuse, clients should offer suitable defaults for the users they will +display the profile fields from. A user may *choose* to display fields from all users globally, +but *by default* profiles should only be shown when the users share a room (or the current one). + ## Unstable Prefixes ### Unstable Profile Fields From 30e82aaa75e692403ad6a947079feefb67010fcf Mon Sep 17 00:00:00 2001 From: Tom Foster Date: Wed, 25 Sep 2024 11:46:34 +0100 Subject: [PATCH 19/51] Consolidation and rewrite --- proposals/4133-extended-profiles.md | 369 +++++++++++++--------------- 1 file changed, 176 insertions(+), 193 deletions(-) diff --git a/proposals/4133-extended-profiles.md b/proposals/4133-extended-profiles.md index b3c0768e52e..0614e6e4eda 100644 --- a/proposals/4133-extended-profiles.md +++ b/proposals/4133-extended-profiles.md @@ -1,126 +1,125 @@ -# MSC4133: Extending User Profile API with Key:Value Pairs +# MSC4133: Extending User Profile API with Custom Key:Value Pairs -*This proposal aims to enhance the usability and richness of user profiles within the Matrix -ecosystem by introducing additional, customisable fields. This feature will facilitate the sharing -of more diverse user-defined public information across the federated network. The primary goal is -to enable users to publish a wide variety of possible information, such as preferred communication -languages, pronouns, public-facing organisation roles, or other relevant public details, thereby -enriching the user interaction experience without impacting existing functionalities.* +*This proposal aims to enhance user profiles within the Matrix ecosystem by introducing +customisable key:value pairs to global profiles. Allowing users to add arbitrary public information +to their profiles (such as preferred languages, organisational roles, or other relevant public +details) should enrich user interactions without impacting existing functionality.* ## Proposal -The Matrix protocol's current user profile structure supports very limited fields (`avatar_url` and -`displayname`). This proposal suggests expanding this structure to include custom fields, allowing -for a more versatile user profile. Specifically, it redefines the existing `avatar_url` and -`displayname` endpoints to be more flexible, while attempting to maximise compatibility with -existing clients and servers to help speed adoption. +Currently, the Matrix protocol supports limited user profile fields: `avatar_url` and +`displayname`. This proposal extends the API to include custom fields, enabling users to add +free-text key:value pairs to their global profiles. This extension provides a flexible framework +for users to share additional public information. -Likewise, this proposal is designed to complement rather than replace +This proposal is designed to be simple and compatible with existing clients and servers, +facilitating quick adoption. It complements, rather than replaces, [MSC1769](https://github.com/matrix-org/matrix-spec-proposals/pull/1769) (Extensible Profiles as -Rooms). While [MSC1769](https://github.com/matrix-org/matrix-spec-proposals/pull/1769) offers a -more complex solution for extensible profiles, this proposal focuses on enabling the storage of -small, arbitrary key:value pairs at the global level. +Rooms), by focusing on global profile data without the complexity of per-room profile management. -This proposal does not seek to enforce the exact content or usage of these fields but rather to add -a framework for users to have extra data that can be further clarified and extended in the future -as community usage of these fields grows. - -Homeservers could disable the ability for users to update these fields or require a specific list -of fields, but the intention of this proposal is that users will be presented with a form to enter -their own free-text fields and values. After using these very flexible fields, the community may -opt to request a further extension to promote one or more fields to be specified per-room via -member events. +After using these custom freetext fields, the community can then propose further extensions to +standardise special fields with specific purposes, such as a user's timezone or languages, and +these proposals may choose to allow selected fields to be specified per-room via member events. ### Client-Server API Changes -1. **GET `/_matrix/client/v3/profile/{userId}/{key_name}`**: This endpoint extends the existing - profile API to allow clients to retrieve the value of a specified `key_name` in a user's profile: +- **Get a Profile Field** + - **Endpoint**: `GET /_matrix/client/v3/profile/{userId}/{key_name}` + - **Description**: Retrieve the value of a specified `key_name` from a user's profile. + - **Response**: ```json { - "key_name": "field_value" + "key_name": "field_value" } ``` -For example, requesting `/_matrix/client/v3/profile/@alice:matrix.org/displayname` would return: +*Example*: Requesting `GET /_matrix/client/v3/profile/@alice:matrix.org/displayname` returns: ```json { - "displayname": "Alice" + "displayname": "Alice" } ``` -2. **PUT `/_matrix/client/v3/profile/{userId}/{key_name}`**: This endpoint allows clients to set - the value of a specified `key_name`: +- **Set a Profile Field** + - **Endpoint**: `PUT /_matrix/client/v3/profile/{userId}/{key_name}` + - **Description**: Set or update the value of a specified `key_name` in the user's profile. + - **Request Body**: ```json { - "key_name": "new_value" + "key_name": "new_value" } ``` -For example, setting `/_matrix/client/v3/profile/@alice:matrix.org/displayname` with: +*Example*: Setting `PUT /_matrix/client/v3/profile/@alice:matrix.org/displayname` with: ```json { - "displayname": "Alice Wonderland" + "displayname": "Alice Wonderland" } ``` -**Note**: As the `DELETE` endpoint exists below, a `PUT` to key using a `null` value should leave -the key in place without deleting the key. +*Note**: Setting a `null` value does not delete the key; the key remains with a `null` value. -1. **DELETE `/_matrix/client/v3/profile/{userId}/{key_name}`**: This endpoint removes a key (and - associated value) from the profile, if permitted by the homeserver. This could be considered a - partial alternative to [MSC3754](https://github.com/matrix-org/matrix-spec-proposals/pull/3754), - which specifies `DELETE` endpoints specifically for `/avatar_url` and `/displayname`. +- **Delete a Profile Field** + - **Endpoint**: `DELETE /_matrix/client/v3/profile/{userId}/{key_name}` + - **Description**: Remove a specified `key_name` (and its value) from the user's profile, + if permitted by the homeserver. -2. **GET `/_matrix/client/v3/profile/{userId}`**: This endpoint retrieves all profile fields: +- **Get All Profile Fields** + - **Endpoint**: `GET /_matrix/client/v3/profile/{userId}` + - **Description**: Retrieve all profile fields for a user. + - **Response**: ```json { - "avatar_url": "mxc://matrix.org/MyC00lAvatar", - "displayname": "John Doe", - "u.Custom Field": "value1" + "avatar_url": "mxc://matrix.org/MyC00lAvatar", + "displayname": "John Doe", + "u.Custom Field": "value1" } ``` -5. **PATCH `/_matrix/client/v3/profile/{userId}`**: This endpoint accepts a complete JSON object to - *merge* into the current profile, updating any changed keys without altering any absent ones: +- **Partially Update Profile Fields** + - **Endpoint**: `PATCH /_matrix/client/v3/profile/{userId}` + - **Description**: Merge the provided JSON object into the user's profile, + updating any specified keys without altering others. + - **Request Body**: ```json { - "avatar_url": "mxc://matrix.org/MyNewAvatar", - "displayname": "John Doe", - "u.Custom Field": "new_value1" + "avatar_url": "mxc://matrix.org/MyNewAvatar", + "displayname": "John Doe", + "u.Custom Field": "new_value1" } ``` -6. **PUT `/_matrix/client/v3/profile/{userId}`**: This endpoint accepts a complete JSON object to - replace the entire profile, adding or updating any changed keys and removing any keys that are - absent in the provided object: +- **Replace Profile Fields** + - **Endpoint**: `PUT /_matrix/client/v3/profile/{userId}` + - **Description**: Replace the entire user's profile with the provided JSON object, + adding or updating keys and removing any absent ones. + - **Request Body**: ```json { - "avatar_url": "mxc://matrix.org/MyNewAvatar", - "displayname": "John Doe", - "u.Custom Field": "new_value1" + "avatar_url": "mxc://matrix.org/MyNewAvatar", + "displayname": "John Doe", + "u.Custom Field": "new_value1" } ``` -**Note**: User-interactive clients are encouraged to manipulate fields individually to avoid race -conditions, however this `PUT` method allows single-client accounts (such as bots) to overwrite the -entire profile in a single request, which allows bridge bots managing many accounts to bulk update -profiles for their users with minimal requests. +**Note**: Clients are encouraged to manipulate fields individually to avoid race conditions. +However, this method allows for bulk updates when needed (e.g., bots managing multiple accounts). ### Server-Server API Changes -**GET `/_matrix/federation/v1/query/profile`** will mirror the client-server API changes to ensure -profile information is consistently available across the federated network. +The federation endpoint `GET /_matrix/federation/v1/query/profile` will mirror the client-server +API changes to ensure profile information is consistent between local and federated users. -As there is no method to verify the history of these fields over federation, this endpoint must -only accept requests for local users on the current homeserver, and homeservers must only request a -profile from the homeserver specified in that user's MXID. +There is no method to verify the history of global profile fields over federation, so this endpoint +must only accept requests for local users on the current homeserver, and homeservers must only +request a profile from the homeserver specified in that user's MXID. As per the current stable endpoint, it accepts an optional `field` query string parameter to request a single field. At time of writing, the Matrix specification says: @@ -133,61 +132,56 @@ fields are published over federation, and this proposal continues to allow this ### Capabilities -A new capability `m.profile_fields` is introduced to control the ability to set custom profile -fields. - -- **Advertising the Capability**: Servers SHOULD advertise this capability in the `capabilities` - object returned by the `GET /_matrix/client/v3/capabilities` endpoint. - -- **Checking the Capability**: Clients SHOULD check for the presence and value of the - `m.profile_fields` capability before attempting to create or update custom profile fields. +A new capability `m.profile_fields` controls the ability to *set* custom profile fields and is +advertised on the `GET /_matrix/client/v3/capabilities` endpoint. Clients should check for this +capability before attempting to create or modify a profile field. - **Capability Structure**: ```json { - "capabilities": { - "m.profile_fields": { - "enabled": true, - "disallowed": ["org.example.job_title"] - } + "capabilities": { + "m.profile_fields": { + "enabled": true, + "disallowed": ["org.example.job_title"] } + } } -``` - -- **When capability missing**: Clients SHOULD assume extended profile fields are supported, and - that they can be created/written to. If a server intends to deny some (or all) changes, it SHOULD - use the capability to advertise this to improve client experience. - -- **When `enabled` is `false`**: Clients SHOULD expect to read and display extended fields but - SHOULD NOT allow users to create or update custom fields. Any attempt to do so may result in a - `403 Forbidden` error. - -- **When `enabled` is `true`**: Clients MAY allow users to create or update custom fields, except - those listed in the `disallowed` array. Individual requests will receive a `400 Bad Request` or - `403 Forbidden` response from the homeserver if server-side policies prevent them. + ``` + +- **Behaviour** + - **When capability missing**: Clients should assume extended profiles are supported, and that + they can be created/written to. If a server intends to deny some (or all) changes, it SHOULD + use the capability to advertise this to improve client experience. + - **When `enabled` is `false`**: Clients should expect to display profiles but NOT create or + update fields. Any attempt to do so should result in a `403 Forbidden` error. This does not + affect `avatar_url` and `displayname` fields, which are allowed for compatibility purposes. + - **When `enabled` is `true`**: Clients should allow users to create or update custom fields, + except those listed in the `disallowed` array. Individual requests will receive a + `400 Bad Request` or `403 Forbidden` response from the homeserver if server-side policies + prevent them. ### Error Handling -To ensure clear communication of issues, the following error codes and messages will be used: +Consistent error codes and messages ensure clear communication of issues: -#### 400 Bad Request: When request is malformed, exceeds limits, or the profile larger than 64KiB +#### **400 Bad Request**: Request exceeds limits or is malformed -- **Error code for malformed request**: `M_BAD_JSON` +- **`M_BAD_JSON`**: Malformed request. ```json { - "errcode": "M_BAD_JSON", - "error": "The provided JSON is malformed." + "errcode": "M_BAD_JSON", + "error": "The provided JSON is malformed." } ``` -- **Error code for profile/field exceeding size limit**: `M_TOO_LARGE` +- **`M_TOO_LARGE`**: Exceeds size limits. ```json { - "errcode": "M_TOO_LARGE", - "error": "The profile data exceeds the maximum allowed size of 64KiB." + "errcode": "M_TOO_LARGE", + "error": "The profile data exceeds the maximum allowed size of 64KiB." } ``` @@ -195,8 +189,8 @@ or ```json { - "errcode": "M_TOO_LARGE", - "error": "Profile field u.Bio exceeds maximum allowed size of 512 bytes." + "errcode": "M_TOO_LARGE", + "error": "The key name exceeds the maximum allowed length of 128 bytes." } ``` @@ -204,82 +198,67 @@ or ```json { - "errcode": "M_TOO_LARGE", - "error": "The key name exceeds the maximum allowed length of 128 bytes." + "errcode": "M_TOO_LARGE", + "error": "The user has exceeded the maximum number of allowed keys in their profile." } ``` -or - -```json -{ - "errcode": "M_TOO_LARGE", - "error": "The user has exceeded the maximum number of allowed keys in their profile." -} -``` - -#### 403 Forbidden: User lacks permission to take this action (e.g. restricted by server policy) +#### **403 Forbidden**: User lacks permission **Note:** See [MSC4170](https://github.com/matrix-org/matrix-spec-proposals/pull/4170) for more discussion on how server policy may result in 403 errors for profile requests. -- **404 Not Found**: When attempting to `GET` or `DELETE` a profile key that does not exist: - - **Error Code**: `M_NOT_FOUND` +#### **404 Not Found**: Target cannot be found + +- **`M_NOT_FOUND`**: Profile key does not exist. ```json { - "errcode": "M_NOT_FOUND", - "error": "The requested profile key does not exist." + "errcode": "M_NOT_FOUND", + "error": "The requested profile key does not exist." } ``` ### Propagation of Profile Fields to Membership Events The existing fields, `avatar_url` and `displayname`, will continue to trigger state events in each -room. These fields are replicated per-room via member events. Custom fields, however, will **not** -trigger state events in rooms and will exist solely at the global level for storing metadata about -the user. +room. These fields are replicated per-room via member events. -- **avatar_url** and **displayname**: Changes to these fields will generate state events in all - rooms the user is a member of. -- **Custom fields**: These are stored in the user's global profile and do not generate state events - in rooms. +All other fields (unless a future proposal specifies otherwise) will **not** trigger state events +in rooms and will exist solely at the global level for storing metadata about the user. ### Key/Namespace Requirements for Custom Fields Homeservers are not expected to enforce these namespaces, as future expansions may be unknown to the server, but clients are expected to use the correct namespace for field creation/updates. -- The namespace `m.*` is reserved for fields defined in the Matrix specification. This field may - have special entry/display requirements that are defined in the Matrix specification. If a client - does not recognise a field in this namespace, it may attempt to display it, but should not - attempt to update the content in case it has special requirements. +The *name* of each key must be a valid UTF-8 string of between one and 128 bytes. -- The namespace `u.*` is reserved for user-defined fields. The portion of the string after the `u.` - is defined as the display name of this field. These user-defined fields will always be string - format, and as measured in UTF-8 byte counts: - - The *name* of each key must not exceed 128 bytes. - - The *content* of each value must not exceed 512 bytes. +- **Namespace `m.*`**: Reserved for fields defined in the Matrix specification. This field may have + special entry/display requirements, so clients that do not recognise a field in this namespace + *may* attempt to display it, but should *not* attempt to update the content. -- Client-specific or unstable fields MUST use the Java package naming convention: `tld.name.*`. +- **Namespace `u.*`**: Reserved for user-defined custom fields. The portion of the key name after + the `u.` defines the display name of this field (e.g. `u.Bio`). The values in this namespace must + always be UTF-8 strings with a content not exceeding 512 bytes. -Following support for this specification change, a user could enter a "My Timezone" field manually -in their client and the client would be expected to add a `u.My Timezone` key in their profile. -However, this would be expected to be a string value, even if the user types "null" or an integer -into the box. +- **Namespace `tld.name.*`**: Client-specific or unstable fields use Java package naming convention. -In contrast, [MSC4175](https://github.com/matrix-org/matrix-spec-proposals/pull/4175) would use the -unstable key `us.cloke.msc4175.tz` and following approval would then support clients using the -`m.tz` key with the values/validation that MSC requires. +Following this change, for example, a user could enter a "My Timezone" field manually in their +client and the client would be expected to add a `u.My Timezone` key in their profile. Clients +would be expected to treat this as a string inside the profile with no special meaning. However, +following [MSC4175](https://github.com/matrix-org/matrix-spec-proposals/pull/4175), clients would +use the unstable key `us.cloke.msc4175.tz` (or stable key `m.tz`) with the validation and +formatting required by that MSC. ### Size Limits Whenever "bytes" are referred to as a limit, this is calculated as UTF-8 bytes, so a two-byte code point consumes two bytes of this limit. This size would be measured before any JSON encoding. -Until another MSC specifies otherwise: +Until a future MSC specifies otherwise: -- Each profile *must* be *at most* 64KiB (65536 bytes) in size, as measured in Canonical JSON. +- Each profile *must* be *at most* 64KiB (65,536 bytes) in size, as measured in Canonical JSON. - Each key *must* be a string of *at least* one byte and it's name *must* not exceed 128 bytes. - Each key's *value* in the `u.*` namespace *must* not exceed 512 bytes in length. - The limit on overall profile size includes `avatar_url` and `displayname`. @@ -292,109 +271,113 @@ Until another MSC specifies otherwise: - Custom fields will not trigger state events in rooms, maintaining account-wide metadata without creating state events or other moderation issues. +- Homeservers should cache remote profiles but implement strategies to minimise unintended data + persistence (e.g. expire caches within 24 hours). + - Clients are encouraged to provide settings for users to choose the scope of users they present the freetext (i.e. `u.*` namespaced) fields from (e.g. none, only users on the local server, users in the current room, any users sharing a room with this user, all users globally) with - defaults sensitive to their implementation's UI and intended audience. + defaults sensitive to their implementation's UI and intended audience. This proposal *recommends* + clients only display profiles of users in the current room whose membership status is `join`, + `invite`, or `knock`, and that if custom fields are always available in the UI, an option should + be provided to hide/minimise them automatically. These recommendations are to moderation concerns + associated with displaying information from banned or departed users. - Clients should provide a UI for users to enter their own free-text custom fields in the `u.*` namespace of their own profile. +- This proposal focuses on introducing custom free-text fields in a controlled manner. Future + extensions, such as fields with special behaviours or administrative controls, will be addressed + in separate MSCs to allow for thorough community discussion and consideration of UI and + moderation impacts. + ## Potential Issues -Initial challenges are primarily ensuring uniform support for custom fields across different -servers and clients during the rollout phase: users may come to expect that other users will check -their supported languages before communicating with them, but the sender's server does not support -custom profiles. +Primarily ensuring uniform support across different servers and clients during the rollout phase: +we don't want users to come to expect other users will check their profile (e.g. languages) before +communicating with them unless most clients and servers support this feature. As such, this MSC is designed to be as simple as possible to get initial functionality and data -structures implemented widely in both clients and servers, so further extensions can be debated, -implemented, and tested over time. +structures implemented widely, so further extensions can be debated, implemented, and tested with +due care over time. As this data is stored only at the global level, it won't allow users to modify fields per-room, -or track historical changes in profile fields. However, this is for performance and moderation -reasons, as many users will struggle to maintain many fields of personal data between different -rooms, and publishing state events for every field change could become an additional burden on -servers and moderators. +or track historical changes in profile fields. However, many users would struggle to track their +own data across many rooms, and publishing state events for every field change could quickly become +a heavy burden on servers and moderators. -In a similar vein, this proposal offers no method to "broadcast" to other users or homeservers that +This proposal recommends future MSCs add certain fields to per-room member events when there is +value in doing so - the current free-text fields added by this proposal are not considered to have +this value. + +This proposal also does not offer a method to "broadcast" to other users or homeservers that changes have occurred. This is intentional to keep the scope of this change narrow and maximise compatibility with existing servers. A future proposal may wish to use an EDU (such as Presence) to notify users and homeservers that these custom fields have been updated. ## Alternatives -An alternative could be to allow for more specific endpoint modifications or to introduce a -completely new API specifically for extended information. However, these approaches could lead -to more significant changes to the existing API and higher complexity for client developers. +An alternative approach could involve introducing a completely new API for extended profile information. However, this may lead to more significant changes and increased complexity for client developers. -### Security Considerations +## Security Considerations -Since the profile data is public, there are minimal security risks associated with the transmission +Since profile fields are public, there are minimal security risks associated with the transmission of sensitive information; however, users should be made aware that any information they add will be -visible to others on the federated network. - -Clients *should* inform users that any custom profile fields they add will be publicly accessible, -and *should* discourage users from adding sensitive personal information to their profiles. +visible to others on the federated network. Clients *should* inform them of this. -Homeservers and clients *must* comply with GDPR and other relevant privacy regulations, particularly -regarding data deletion and retention. Profile data *should* be cleared when a user is deactivated. -While homeservers *should* cache remote profiles, they *should* implement strategies that do not -exceed 24 hours to minimize the risk of unintended data persistence. +Homeservers and clients *must* comply relevant privacy regulations, particularly regarding data +deletion and retention. Profile data *should* be cleared when a user is deactivated, and while +homeservers *should* cache remote profiles, they *should* avoid caching beyond 24 hours to minimise +the risk of unintended data persistence. To minimise the impact of abuse, clients should offer suitable defaults for the users they will display the profile fields from. A user may *choose* to display fields from all users globally, -but *by default* profiles should only be shown when the users share a room (or the current one). +but *by default* profiles should only be shown when the users share the current room and the other +user is in the `join`, `invite`, or `knock` membership states. ## Unstable Prefixes ### Unstable Profile Fields -The current Matrix specification technically already allows extra custom fields to be published in -a user's profile. However, as this proposal introduces additional requirements and allows custom -user-defined fields, an unstable prefix should be used on these fields until this proposal has -entered the API as stable: +Until this proposal is stable, custom fields should use an unstable prefix: ```json { - "avatar_url": "mxc://matrix.org/MyC00lAvatar", - "displayname": "John Doe", - "uk.tcpip.msc4133.u.Custom Field": "field_value" + "avatar_url": "mxc://matrix.org/MyC00lAvatar", + "displayname": "John Doe", + "uk.tcpip.msc4133.u.Custom Field": "field_value" } ``` ### Unstable Endpoints -`/_matrix/client/unstable/uk.tcpip.msc4133/profile/{userId}` would be necessary for the `PATCH` and -`PUT` methods allowed when the unstable capability (detailed below) is advertised by the server. +Use unstable endpoints when the capability is not yet stable: -The existing `GET` method would act as normal and remain on `/_matrix/client/v3/profile/{userId}` -without need for an unstable endpoint. +- **Get/Set/Delete Custom Fields**: + - `/_matrix/client/unstable/uk.tcpip.msc4133/profile/{userId}/{key_name}` -Likewise, when the unstable capability is advertised by the server, the server should accept the -new key endpoints `/_matrix/client/unstable/uk.tcpip.msc4133/profile/{userId}/{key_name}` which -would then promote to `/_matrix/client/v3/profile/{userId}/{key_name}` when the stable capability -is advertised following this specification change. +- **Patch/Put Profile**: + - `/_matrix/client/unstable/uk.tcpip.msc4133/profile/{userId}` -### Unstable Client Capability +### Unstable Capability -The client capability `m.profile_fields` should use this prefix until stable: +Advertise the capability with an unstable prefix: ```json { - "capabilities": { - "uk.tcpip.msc4133.profile_fields": { - "enabled": true, - "disallowed": ["org.example.job_title"] - } + "capabilities": { + "uk.tcpip.msc4133.profile_fields": { + "enabled": true, + "disallowed": ["org.example.job_title"] } + } } ``` ### Unstable Client Features -The client feature `uk.tcpip.msc4133` should be advertised on the `/_matrix/client/versions` -endpoint when the `PUT` and `PATCH` methods are accepted on the +Client feature `uk.tcpip.msc4133` should be advertised on the `/_matrix/client/versions` endpoint +when the `PUT` and `PATCH` methods are accepted on the `/_matrix/client/unstable/uk.tcpip.msc4133/profile/{userId}` endpoint. Once this MSC is merged, the client feature `uk.tcpip.msc4133.stable` should be advertised when the From 2966c85a474eda737375d4fad4dc61ba5528d6bf Mon Sep 17 00:00:00 2001 From: Tom Foster Date: Wed, 25 Sep 2024 12:13:05 +0100 Subject: [PATCH 20/51] Whitespace fix --- proposals/4133-extended-profiles.md | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/proposals/4133-extended-profiles.md b/proposals/4133-extended-profiles.md index 0614e6e4eda..6d201c94cba 100644 --- a/proposals/4133-extended-profiles.md +++ b/proposals/4133-extended-profiles.md @@ -147,7 +147,7 @@ capability before attempting to create or modify a profile field. } } } - ``` +``` - **Behaviour** - **When capability missing**: Clients should assume extended profiles are supported, and that @@ -317,7 +317,8 @@ notify users and homeservers that these custom fields have been updated. ## Alternatives -An alternative approach could involve introducing a completely new API for extended profile information. However, this may lead to more significant changes and increased complexity for client developers. +An alternative approach could involve introducing a completely new API for extended profile +information. However, this may lead to increased complexity for client and server implementations. ## Security Considerations From d97189eccaa475ca3767478c0e977ff2f9ed1458 Mon Sep 17 00:00:00 2001 From: Tom Foster Date: Wed, 25 Sep 2024 15:07:36 +0100 Subject: [PATCH 21/51] Safety and security updates --- proposals/4133-extended-profiles.md | 59 +++++++++++++++++++---------- 1 file changed, 39 insertions(+), 20 deletions(-) diff --git a/proposals/4133-extended-profiles.md b/proposals/4133-extended-profiles.md index 6d201c94cba..1cc788644c9 100644 --- a/proposals/4133-extended-profiles.md +++ b/proposals/4133-extended-profiles.md @@ -21,7 +21,7 @@ After using these custom freetext fields, the community can then propose further standardise special fields with specific purposes, such as a user's timezone or languages, and these proposals may choose to allow selected fields to be specified per-room via member events. -### Client-Server API Changes +## Client-Server API Changes - **Get a Profile Field** - **Endpoint**: `GET /_matrix/client/v3/profile/{userId}/{key_name}` @@ -112,7 +112,7 @@ these proposals may choose to allow selected fields to be specified per-room via **Note**: Clients are encouraged to manipulate fields individually to avoid race conditions. However, this method allows for bulk updates when needed (e.g., bots managing multiple accounts). -### Server-Server API Changes +## Server-Server API Changes The federation endpoint `GET /_matrix/federation/v1/query/profile` will mirror the client-server API changes to ensure profile information is consistent between local and federated users. @@ -130,7 +130,7 @@ request a single field. At time of writing, the Matrix specification says: Given this wording, homeservers currently already have the flexibility to decide whether some fields are published over federation, and this proposal continues to allow this behaviour. -### Capabilities +## Capabilities A new capability `m.profile_fields` controls the ability to *set* custom profile fields and is advertised on the `GET /_matrix/client/v3/capabilities` endpoint. Clients should check for this @@ -143,6 +143,7 @@ capability before attempting to create or modify a profile field. "capabilities": { "m.profile_fields": { "enabled": true, + "allowed": ["u.Custom Field 1", "org.example.job_title"], "disallowed": ["org.example.job_title"] } } @@ -153,19 +154,23 @@ capability before attempting to create or modify a profile field. - **When capability missing**: Clients should assume extended profiles are supported, and that they can be created/written to. If a server intends to deny some (or all) changes, it SHOULD use the capability to advertise this to improve client experience. + - **When `enabled` is `false`**: Clients should expect to display profiles but NOT create or - update fields. Any attempt to do so should result in a `403 Forbidden` error. This does not - affect `avatar_url` and `displayname` fields, which are allowed for compatibility purposes. + update fields. Any attempt to do so should result in a `403 Forbidden` error. This does not + affect `avatar_url` and `displayname` fields, which are allowed for compatibility purposes. + - **When `enabled` is `true`**: Clients should allow users to create or update custom fields, - except those listed in the `disallowed` array. Individual requests will receive a - `400 Bad Request` or `403 Forbidden` response from the homeserver if server-side policies - prevent them. + except those listed in the `disallowed` array, if it exists. Servers may optionally specify the + `allowed` array to allowlist fields that users may set - if the `allowed` key is provided with + empty content (e.g. `"allowed": []`) this also disallows the setting of any fields. Clients + will receive `400 Bad Request` or `403 Forbidden` responses from the homeserver if server-side + policies prevent them. -### Error Handling +## Error Handling Consistent error codes and messages ensure clear communication of issues: -#### **400 Bad Request**: Request exceeds limits or is malformed +### **400 Bad Request**: Request exceeds limits or is malformed - **`M_BAD_JSON`**: Malformed request. @@ -203,12 +208,12 @@ or } ``` -#### **403 Forbidden**: User lacks permission +### **403 Forbidden**: User lacks permission **Note:** See [MSC4170](https://github.com/matrix-org/matrix-spec-proposals/pull/4170) for more discussion on how server policy may result in 403 errors for profile requests. -#### **404 Not Found**: Target cannot be found +### **404 Not Found**: Target cannot be found - **`M_NOT_FOUND`**: Profile key does not exist. @@ -219,7 +224,7 @@ discussion on how server policy may result in 403 errors for profile requests. } ``` -### Propagation of Profile Fields to Membership Events +## Propagation of Profile Fields to Membership Events The existing fields, `avatar_url` and `displayname`, will continue to trigger state events in each room. These fields are replicated per-room via member events. @@ -227,7 +232,7 @@ room. These fields are replicated per-room via member events. All other fields (unless a future proposal specifies otherwise) will **not** trigger state events in rooms and will exist solely at the global level for storing metadata about the user. -### Key/Namespace Requirements for Custom Fields +## Key/Namespace Requirements for Custom Fields Homeservers are not expected to enforce these namespaces, as future expansions may be unknown to the server, but clients are expected to use the correct namespace for field creation/updates. @@ -251,7 +256,7 @@ following [MSC4175](https://github.com/matrix-org/matrix-spec-proposals/pull/417 use the unstable key `us.cloke.msc4175.tz` (or stable key `m.tz`) with the validation and formatting required by that MSC. -### Size Limits +## Size Limits Whenever "bytes" are referred to as a limit, this is calculated as UTF-8 bytes, so a two-byte code point consumes two bytes of this limit. This size would be measured before any JSON encoding. @@ -263,7 +268,7 @@ Until a future MSC specifies otherwise: - Each key's *value* in the `u.*` namespace *must* not exceed 512 bytes in length. - The limit on overall profile size includes `avatar_url` and `displayname`. -### Implementation Details +## Implementation Details - The profile data will be public by default, and compliance with GDPR and other privacy regulations will be enforced, particularly in terms of data deletion and retention policies. @@ -271,7 +276,7 @@ Until a future MSC specifies otherwise: - Custom fields will not trigger state events in rooms, maintaining account-wide metadata without creating state events or other moderation issues. -- Homeservers should cache remote profiles but implement strategies to minimise unintended data +- Servers should cache remote profiles but implement strategies to minimise unintended data persistence (e.g. expire caches within 24 hours). - Clients are encouraged to provide settings for users to choose the scope of users they present @@ -286,6 +291,11 @@ Until a future MSC specifies otherwise: - Clients should provide a UI for users to enter their own free-text custom fields in the `u.*` namespace of their own profile. +- Servers may add/remove/modify fields in their own user's global profile data, whether for + moderation purposes or for other policy reasons, e.g. to automatically populate a job title based + on the user's organisation. Server administrators must make users aware that these fields exist + and that they will be made available to every server that this server federates with, if any. + - This proposal focuses on introducing custom free-text fields in a controlled manner. Future extensions, such as fields with special behaviours or administrative controls, will be addressed in separate MSCs to allow for thorough community discussion and consideration of UI and @@ -320,11 +330,20 @@ notify users and homeservers that these custom fields have been updated. An alternative approach could involve introducing a completely new API for extended profile information. However, this may lead to increased complexity for client and server implementations. +At the time of writing, Extensible Profiles as Rooms +([MSC1769](https://github.com/matrix-org/matrix-spec-proposals/pull/1769)) is under development for +richer and more granular content and privacy controls, which this proposal does not intend to +replace - this proposal focuses on basic global profile data without the complexity of per-room +profile management. + ## Security Considerations -Since profile fields are public, there are minimal security risks associated with the transmission -of sensitive information; however, users should be made aware that any information they add will be -visible to others on the federated network. Clients *should* inform them of this. +Since profile fields are public, the server is not directly responsible for the privacy of the data; +however, clients should make users aware that any information published in their profile will be +visible to others on the federated network. + +Likewise, if a server automatically publishes data in user profile fields (e.g. setting a job title +based on an organisation's internal user database) then they must have consent to do so. Homeservers and clients *must* comply relevant privacy regulations, particularly regarding data deletion and retention. Profile data *should* be cleared when a user is deactivated, and while From ae197255f91d936527285f65ca891757e29859a6 Mon Sep 17 00:00:00 2001 From: Tom Foster Date: Wed, 25 Sep 2024 15:25:14 +0100 Subject: [PATCH 22/51] Clarify servers can hide fields --- proposals/4133-extended-profiles.md | 7 +++++++ 1 file changed, 7 insertions(+) diff --git a/proposals/4133-extended-profiles.md b/proposals/4133-extended-profiles.md index 1cc788644c9..246a3f17d60 100644 --- a/proposals/4133-extended-profiles.md +++ b/proposals/4133-extended-profiles.md @@ -296,6 +296,13 @@ Until a future MSC specifies otherwise: on the user's organisation. Server administrators must make users aware that these fields exist and that they will be made available to every server that this server federates with, if any. +- Servers are *allowed* to suppress some or all local/remote profile fields they do not wish to + share with their local users (e.g. if there are moderation concerns during a go-live phase); + however, server admins *must* disclose to users if they are publishing profile fields on behalf + of a user over federation that they cannot see, as this could be considered a breach of trust. + This feature could cause confusion if some users can see fields that other users cannot, so this + should be used sparingly. + - This proposal focuses on introducing custom free-text fields in a controlled manner. Future extensions, such as fields with special behaviours or administrative controls, will be addressed in separate MSCs to allow for thorough community discussion and consideration of UI and From 23a3a625b13023b09be3f66c872efd7e64701414 Mon Sep 17 00:00:00 2001 From: Tom Foster Date: Thu, 26 Sep 2024 20:08:01 +0100 Subject: [PATCH 23/51] References to MSC4201 and MSC4202 --- proposals/4133-extended-profiles.md | 16 ++++++++++++++-- 1 file changed, 14 insertions(+), 2 deletions(-) diff --git a/proposals/4133-extended-profiles.md b/proposals/4133-extended-profiles.md index 246a3f17d60..638e7ae7fbd 100644 --- a/proposals/4133-extended-profiles.md +++ b/proposals/4133-extended-profiles.md @@ -332,13 +332,20 @@ changes have occurred. This is intentional to keep the scope of this change narr compatibility with existing servers. A future proposal may wish to use an EDU (such as Presence) to notify users and homeservers that these custom fields have been updated. +This proposal does not directly address reporting of user profiles over federation, but +[MSC4202](https://github.com/matrix-org/matrix-spec-proposals/pull/4202) offers a facility for +users to report offensive content to the homeserver that account belongs to. This proposal is not +dependent on [MSC4202](https://github.com/matrix-org/matrix-spec-proposals/pull/4202) but encourages +the use of moderation options to allow users to report offensive content. + ## Alternatives An alternative approach could involve introducing a completely new API for extended profile information. However, this may lead to increased complexity for client and server implementations. At the time of writing, Extensible Profiles as Rooms -([MSC1769](https://github.com/matrix-org/matrix-spec-proposals/pull/1769)) is under development for +([MSC1769](https://github.com/matrix-org/matrix-spec-proposals/pull/1769) and variant +[MSC4201](https://github.com/matrix-org/matrix-spec-proposals/pull/4201)) is under development for richer and more granular content and privacy controls, which this proposal does not intend to replace - this proposal focuses on basic global profile data without the complexity of per-room profile management. @@ -350,7 +357,8 @@ however, clients should make users aware that any information published in their visible to others on the federated network. Likewise, if a server automatically publishes data in user profile fields (e.g. setting a job title -based on an organisation's internal user database) then they must have consent to do so. +based on an organisation's internal user database) then they must have consent to do so, and users +should be made aware that data is published on their behalf. Homeservers and clients *must* comply relevant privacy regulations, particularly regarding data deletion and retention. Profile data *should* be cleared when a user is deactivated, and while @@ -362,6 +370,10 @@ display the profile fields from. A user may *choose* to display fields from all but *by default* profiles should only be shown when the users share the current room and the other user is in the `join`, `invite`, or `knock` membership states. +The proposal [MSC4202](https://github.com/matrix-org/matrix-spec-proposals/pull/4202) adds reporting +of user profiles over federation, which offers a facility for users to report offensive content to +the homeserver that account is registered on. + ## Unstable Prefixes ### Unstable Profile Fields From f32932cf1f45a6afdea9c8a9c622479ae1d54805 Mon Sep 17 00:00:00 2001 From: Tom Foster Date: Mon, 30 Sep 2024 13:39:04 +0100 Subject: [PATCH 24/51] Clarify redacted `m.room.member` events --- proposals/4133-extended-profiles.md | 7 ++++--- 1 file changed, 4 insertions(+), 3 deletions(-) diff --git a/proposals/4133-extended-profiles.md b/proposals/4133-extended-profiles.md index 638e7ae7fbd..4ac3ba49728 100644 --- a/proposals/4133-extended-profiles.md +++ b/proposals/4133-extended-profiles.md @@ -284,9 +284,10 @@ Until a future MSC specifies otherwise: users in the current room, any users sharing a room with this user, all users globally) with defaults sensitive to their implementation's UI and intended audience. This proposal *recommends* clients only display profiles of users in the current room whose membership status is `join`, - `invite`, or `knock`, and that if custom fields are always available in the UI, an option should - be provided to hide/minimise them automatically. These recommendations are to moderation concerns - associated with displaying information from banned or departed users. + `invite`, or `knock`, and whose `m.room.member` event has *not* been redacted. If a client offers + an option for custom fields to always be available in the UI, an option should be provided to + hide/minimise them automatically. These recommendations are to moderation concerns associated + with displaying information from banned or departed users. - Clients should provide a UI for users to enter their own free-text custom fields in the `u.*` namespace of their own profile. From 4afb8b84e2879dbd85affaeb65e468eec9a3710d Mon Sep 17 00:00:00 2001 From: Tom Foster Date: Tue, 1 Oct 2024 16:48:45 +0100 Subject: [PATCH 25/51] Error codes and redundant instructions --- proposals/4133-extended-profiles.md | 118 ++++++++++++---------------- 1 file changed, 52 insertions(+), 66 deletions(-) diff --git a/proposals/4133-extended-profiles.md b/proposals/4133-extended-profiles.md index 4ac3ba49728..834415564fe 100644 --- a/proposals/4133-extended-profiles.md +++ b/proposals/4133-extended-profiles.md @@ -44,7 +44,8 @@ these proposals may choose to allow selected fields to be specified per-room via - **Set a Profile Field** - **Endpoint**: `PUT /_matrix/client/v3/profile/{userId}/{key_name}` - - **Description**: Set or update the value of a specified `key_name` in the user's profile. + - **Description**: Set or update the value of a specified `key_name` in the user's profile, + if permitted by the homeserver. - **Request Body**: ```json @@ -83,8 +84,8 @@ these proposals may choose to allow selected fields to be specified per-room via - **Partially Update Profile Fields** - **Endpoint**: `PATCH /_matrix/client/v3/profile/{userId}` - - **Description**: Merge the provided JSON object into the user's profile, - updating any specified keys without altering others. + - **Description**: Merge the provided JSON object into the user's profile, updating + any specified keys without altering others, if permitted by the homeserver. - **Request Body**: ```json @@ -98,7 +99,7 @@ these proposals may choose to allow selected fields to be specified per-room via - **Replace Profile Fields** - **Endpoint**: `PUT /_matrix/client/v3/profile/{userId}` - **Description**: Replace the entire user's profile with the provided JSON object, - adding or updating keys and removing any absent ones. + adding or updating keys and removing any absent ones, if permitted by the homeserver. - **Request Body**: ```json @@ -110,7 +111,7 @@ these proposals may choose to allow selected fields to be specified per-room via ``` **Note**: Clients are encouraged to manipulate fields individually to avoid race conditions. -However, this method allows for bulk updates when needed (e.g., bots managing multiple accounts). +However, this method allows for bulk updates when needed (e.g. bots managing multiple accounts). ## Server-Server API Changes @@ -162,7 +163,8 @@ capability before attempting to create or modify a profile field. - **When `enabled` is `true`**: Clients should allow users to create or update custom fields, except those listed in the `disallowed` array, if it exists. Servers may optionally specify the `allowed` array to allowlist fields that users may set - if the `allowed` key is provided with - empty content (e.g. `"allowed": []`) this also disallows the setting of any fields. Clients + empty content (e.g. `"allowed": []`) this also disallows the setting of any fields. If both + `allowed` and `disallowed` keys are provided, the `disallowed` one should be ignored. Clients will receive `400 Bad Request` or `403 Forbidden` responses from the homeserver if server-side policies prevent them. @@ -181,29 +183,29 @@ Consistent error codes and messages ensure clear communication of issues: } ``` -- **`M_TOO_LARGE`**: Exceeds size limits. +- **`M_PROFILE_TOO_LARGE`**: Exceeds total profile size limits. ```json { - "errcode": "M_TOO_LARGE", + "errcode": "M_PROFILE_TOO_LARGE", "error": "The profile data exceeds the maximum allowed size of 64KiB." } ``` -or +- **`M_KEY_TOO_LARGE`**: Exceeds individual key length limits. ```json { - "errcode": "M_TOO_LARGE", + "errcode": "M_KEY_TOO_LARGE", "error": "The key name exceeds the maximum allowed length of 128 bytes." } ``` -or +- **`M_TOO_MANY_KEYS`**: Exceeds key limits. ```json { - "errcode": "M_TOO_LARGE", + "errcode": "M_TOO_MANY_KEYS", "error": "The user has exceeded the maximum number of allowed keys in their profile." } ``` @@ -232,12 +234,19 @@ room. These fields are replicated per-room via member events. All other fields (unless a future proposal specifies otherwise) will **not** trigger state events in rooms and will exist solely at the global level for storing metadata about the user. -## Key/Namespace Requirements for Custom Fields +## Key/Namespace Requirements + +(Whenever "bytes" are referred to as a limit, this is calculated as UTF-8 bytes, so a two-byte code +point consumes two bytes of this limit. This size would be measured before any JSON encoding.) + +Each profile *must* be *at most* 64KiB (65,536 bytes) in size, as measured in Canonical JSON, and +includes `avatar_url` and `displayname`. -Homeservers are not expected to enforce these namespaces, as future expansions may be unknown to -the server, but clients are expected to use the correct namespace for field creation/updates. +Homeservers are not expected to enforce namespaces, as future expansions may be unknown to the +server, but clients are expected to use the correct namespace for field creation/updates. -The *name* of each key must be a valid UTF-8 string of between one and 128 bytes. +Key must follow the [Common Namespaced Identifier Grammar](https://spec.matrix.org/unstable/appendices/#common-namespaced-identifier-grammar), +but with an additional `u.*` namespace for user-defined fields: - **Namespace `m.*`**: Reserved for fields defined in the Matrix specification. This field may have special entry/display requirements, so clients that do not recognise a field in this namespace @@ -250,64 +259,41 @@ The *name* of each key must be a valid UTF-8 string of between one and 128 bytes - **Namespace `tld.name.*`**: Client-specific or unstable fields use Java package naming convention. Following this change, for example, a user could enter a "My Timezone" field manually in their -client and the client would be expected to add a `u.My Timezone` key in their profile. Clients -would be expected to treat this as a string inside the profile with no special meaning. However, -following [MSC4175](https://github.com/matrix-org/matrix-spec-proposals/pull/4175), clients would -use the unstable key `us.cloke.msc4175.tz` (or stable key `m.tz`) with the validation and -formatting required by that MSC. - -## Size Limits - -Whenever "bytes" are referred to as a limit, this is calculated as UTF-8 bytes, so a two-byte code -point consumes two bytes of this limit. This size would be measured before any JSON encoding. - -Until a future MSC specifies otherwise: - -- Each profile *must* be *at most* 64KiB (65,536 bytes) in size, as measured in Canonical JSON. -- Each key *must* be a string of *at least* one byte and it's name *must* not exceed 128 bytes. -- Each key's *value* in the `u.*` namespace *must* not exceed 512 bytes in length. -- The limit on overall profile size includes `avatar_url` and `displayname`. +client, and the client would be required to encode this as `u.my_timezone` key in their profile. +Clients would be expected to treat this as a string inside the profile with no special meaning. +However, following [MSC4175](https://github.com/matrix-org/matrix-spec-proposals/pull/4175), +clients would use the unstable key `us.cloke.msc4175.tz` (or stable key `m.tz`) following the +validation and formatting required by that MSC. ## Implementation Details -- The profile data will be public by default, and compliance with GDPR and other privacy - regulations will be enforced, particularly in terms of data deletion and retention policies. - -- Custom fields will not trigger state events in rooms, maintaining account-wide metadata without - creating state events or other moderation issues. - -- Servers should cache remote profiles but implement strategies to minimise unintended data - persistence (e.g. expire caches within 24 hours). +- Custom fields MUST not trigger state events in rooms; their data must not be replicated to + `m.room.member` events unless a future proposal creates exceptions for specific fields. -- Clients are encouraged to provide settings for users to choose the scope of users they present - the freetext (i.e. `u.*` namespaced) fields from (e.g. none, only users on the local server, - users in the current room, any users sharing a room with this user, all users globally) with - defaults sensitive to their implementation's UI and intended audience. This proposal *recommends* - clients only display profiles of users in the current room whose membership status is `join`, - `invite`, or `knock`, and whose `m.room.member` event has *not* been redacted. If a client offers - an option for custom fields to always be available in the UI, an option should be provided to - hide/minimise them automatically. These recommendations are to moderation concerns associated - with displaying information from banned or departed users. +- Servers SHOULD cache remote profiles for *at least* 5 minutes after retrieval; however, until + a future proposal establishes a method for servers to notify each other of updates, it is + *recommended* profiles be cached no longer than 15 minutes to avoid displaying stale data. + Servers MUST not cache for longer than 24 hours to avoid unwanted data persistence. -- Clients should provide a UI for users to enter their own free-text custom fields in the `u.*` +- Clients COULD provide a UI for users to view/enter their own free-text custom fields in the `u.*` namespace of their own profile. -- Servers may add/remove/modify fields in their own user's global profile data, whether for +- If clients present custom fields in profiles to users, clients SHOULD provide settings to choose + the scope of profiles they present the freetext (i.e. `u.*` namespaced) fields from, (e.g. none, + only users on the local server, users in the current room, any users sharing a room with this + user, all users globally) with defaults sensitive to their implementation's UI and intended + audience. This proposal *recommends* clients only display profiles of users in the current room + whose membership status is `join`, `invite`, or `knock`, and whose `m.room.member` event has + *not* been redacted. If a client offers an option for custom fields to always be available in the + UI, an option should be provided to hide/minimise them automatically. + +- Servers MAY add/remove/modify fields in their own user's global profile data, whether for moderation purposes or for other policy reasons, e.g. to automatically populate a job title based - on the user's organisation. Server administrators must make users aware that these fields exist - and that they will be made available to every server that this server federates with, if any. - -- Servers are *allowed* to suppress some or all local/remote profile fields they do not wish to - share with their local users (e.g. if there are moderation concerns during a go-live phase); - however, server admins *must* disclose to users if they are publishing profile fields on behalf - of a user over federation that they cannot see, as this could be considered a breach of trust. - This feature could cause confusion if some users can see fields that other users cannot, so this - should be used sparingly. - -- This proposal focuses on introducing custom free-text fields in a controlled manner. Future - extensions, such as fields with special behaviours or administrative controls, will be addressed - in separate MSCs to allow for thorough community discussion and consideration of UI and - moderation impacts. + on the user's organisation. + +- Servers MAY suppress all extended profile fields to their users (e.g. during a go-live phase); + however, this feature could cause confusion if some users can see fields that other users + cannot, so should be used sparingly. ## Potential Issues From bb4fc7679c50577662169d6dea5e70848a37ea68 Mon Sep 17 00:00:00 2001 From: Tom Foster Date: Wed, 2 Oct 2024 16:40:43 +0100 Subject: [PATCH 26/51] Removing custom fields --- proposals/4133-extended-profiles.md | 150 +++++++++++++--------------- 1 file changed, 70 insertions(+), 80 deletions(-) diff --git a/proposals/4133-extended-profiles.md b/proposals/4133-extended-profiles.md index 834415564fe..84f9dab4242 100644 --- a/proposals/4133-extended-profiles.md +++ b/proposals/4133-extended-profiles.md @@ -9,18 +9,14 @@ details) should enrich user interactions without impacting existing functionalit Currently, the Matrix protocol supports limited user profile fields: `avatar_url` and `displayname`. This proposal extends the API to include custom fields, enabling users to add -free-text key:value pairs to their global profiles. This extension provides a flexible framework -for users to share additional public information. +key:value pairs to their global profiles. This extension provides a flexible framework for users to +share additional public information. This proposal is designed to be simple and compatible with existing clients and servers, facilitating quick adoption. It complements, rather than replaces, [MSC1769](https://github.com/matrix-org/matrix-spec-proposals/pull/1769) (Extensible Profiles as Rooms), by focusing on global profile data without the complexity of per-room profile management. -After using these custom freetext fields, the community can then propose further extensions to -standardise special fields with specific purposes, such as a user's timezone or languages, and -these proposals may choose to allow selected fields to be specified per-room via member events. - ## Client-Server API Changes - **Get a Profile Field** @@ -78,7 +74,8 @@ these proposals may choose to allow selected fields to be specified per-room via { "avatar_url": "mxc://matrix.org/MyC00lAvatar", "displayname": "John Doe", - "u.Custom Field": "value1" + "m.example_field": "value1", + "org.example.job_title": "Software Engineer" } ``` @@ -92,7 +89,7 @@ these proposals may choose to allow selected fields to be specified per-room via { "avatar_url": "mxc://matrix.org/MyNewAvatar", "displayname": "John Doe", - "u.Custom Field": "new_value1" + "m.example_field": "new_value1" } ``` @@ -106,7 +103,7 @@ these proposals may choose to allow selected fields to be specified per-room via { "avatar_url": "mxc://matrix.org/MyNewAvatar", "displayname": "John Doe", - "u.Custom Field": "new_value1" + "m.example_field": "new_value1" } ``` @@ -119,7 +116,7 @@ The federation endpoint `GET /_matrix/federation/v1/query/profile` will mirror t API changes to ensure profile information is consistent between local and federated users. There is no method to verify the history of global profile fields over federation, so this endpoint -must only accept requests for local users on the current homeserver, and homeservers must only +MUST only accept requests for local users on the current homeserver, and homeservers MUST only request a profile from the homeserver specified in that user's MXID. As per the current stable endpoint, it accepts an optional `field` query string parameter to @@ -134,7 +131,7 @@ fields are published over federation, and this proposal continues to allow this ## Capabilities A new capability `m.profile_fields` controls the ability to *set* custom profile fields and is -advertised on the `GET /_matrix/client/v3/capabilities` endpoint. Clients should check for this +advertised on the `GET /_matrix/client/v3/capabilities` endpoint. Clients SHOULD check for this capability before attempting to create or modify a profile field. - **Capability Structure**: @@ -144,29 +141,29 @@ capability before attempting to create or modify a profile field. "capabilities": { "m.profile_fields": { "enabled": true, - "allowed": ["u.Custom Field 1", "org.example.job_title"], + "allowed": ["m.example_field", "org.example.job_title"], "disallowed": ["org.example.job_title"] } } } ``` -- **Behaviour** - - **When capability missing**: Clients should assume extended profiles are supported, and that +- **Behaviour**: + - **When capability is missing**: Clients SHOULD assume extended profiles are supported, and that they can be created/written to. If a server intends to deny some (or all) changes, it SHOULD use the capability to advertise this to improve client experience. - - **When `enabled` is `false`**: Clients should expect to display profiles but NOT create or - update fields. Any attempt to do so should result in a `403 Forbidden` error. This does not + - **When `enabled` is `false`**: Clients SHOULD expect to display profiles but NOT create or + update fields. Any attempt to do so SHOULD result in a `403 Forbidden` error. This does not affect `avatar_url` and `displayname` fields, which are allowed for compatibility purposes. - - **When `enabled` is `true`**: Clients should allow users to create or update custom fields, - except those listed in the `disallowed` array, if it exists. Servers may optionally specify the - `allowed` array to allowlist fields that users may set - if the `allowed` key is provided with - empty content (e.g. `"allowed": []`) this also disallows the setting of any fields. If both - `allowed` and `disallowed` keys are provided, the `disallowed` one should be ignored. Clients - will receive `400 Bad Request` or `403 Forbidden` responses from the homeserver if server-side - policies prevent them. + - **When `enabled` is `true`**: Clients SHOULD allow users to create or update fields, except + those for keys listed in the `disallowed` array, if it exists. Servers MAY optionally specify + the `allowed` array to allowlist fields that users can set - if the `allowed` key is provided + with empty content (e.g. `"allowed": []`), this also disallows the setting of any fields. + If both `allowed` and `disallowed` keys are provided, the `disallowed` one should be ignored. + Clients will receive `400 Bad Request` or `403 Forbidden` responses from the homeserver if + server-side policies prevent them. ## Error Handling @@ -239,35 +236,36 @@ in rooms and will exist solely at the global level for storing metadata about th (Whenever "bytes" are referred to as a limit, this is calculated as UTF-8 bytes, so a two-byte code point consumes two bytes of this limit. This size would be measured before any JSON encoding.) -Each profile *must* be *at most* 64KiB (65,536 bytes) in size, as measured in Canonical JSON, and -includes `avatar_url` and `displayname`. +Each profile MUST be *at most* 64KiB (65,536 bytes) in size, as measured in Canonical JSON, +including the `avatar_url` and `displayname` fields. Homeservers are not expected to enforce namespaces, as future expansions may be unknown to the server, but clients are expected to use the correct namespace for field creation/updates. -Key must follow the [Common Namespaced Identifier Grammar](https://spec.matrix.org/unstable/appendices/#common-namespaced-identifier-grammar), -but with an additional `u.*` namespace for user-defined fields: +Keys MUST follow the [Common Namespaced Identifier Grammar](https://spec.matrix.org/unstable/appendices/#common-namespaced-identifier-grammar), +with the following considerations: -- **Namespace `m.*`**: Reserved for fields defined in the Matrix specification. This field may have - special entry/display requirements, so clients that do not recognise a field in this namespace - *may* attempt to display it, but should *not* attempt to update the content. +- **Namespace `m.*`**: Reserved for fields explicitly defined in the Matrix specification. These + fields may have special entry/display requirements, so clients that do not recognise a field in + this namespace MAY attempt to display it, but SHOULD NOT attempt to update the content unless + they understand its formatting and validation requirements. -- **Namespace `u.*`**: Reserved for user-defined custom fields. The portion of the key name after - the `u.` defines the display name of this field (e.g. `u.Bio`). The values in this namespace must - always be UTF-8 strings with a content not exceeding 512 bytes. +- **Namespace `u.*`**: Reserved for future user-defined custom fields. A future proposal may + justify a need for users to create custom keys within their client, so this namespace has been + reserved for that purpose. Clients and servers SHOULD avoid using this namespace until specified + in a future proposal. -- **Namespace `tld.name.*`**: Client-specific or unstable fields use Java package naming convention. +- **Namespace `tld.name.*`**: For client-specific or unstable fields, using Java package naming + convention (e.g., `com.example.custom_field`). -Following this change, for example, a user could enter a "My Timezone" field manually in their -client, and the client would be required to encode this as `u.my_timezone` key in their profile. -Clients would be expected to treat this as a string inside the profile with no special meaning. -However, following [MSC4175](https://github.com/matrix-org/matrix-spec-proposals/pull/4175), -clients would use the unstable key `us.cloke.msc4175.tz` (or stable key `m.tz`) following the -validation and formatting required by that MSC. +Following this change, for example, clients could use `m.example_field` if that field is defined by +the Matrix specification, or `org.example.job_title` for organisation or client-specific fields. +The proposal [MSC4175](https://github.com/matrix-org/matrix-spec-proposals/pull/4175) demonstrates +the process of definining new fields in the `m.*` namespace. ## Implementation Details -- Custom fields MUST not trigger state events in rooms; their data must not be replicated to +- Custom fields MUST not trigger state events in rooms; their data MUST not be replicated to `m.room.member` events unless a future proposal creates exceptions for specific fields. - Servers SHOULD cache remote profiles for *at least* 5 minutes after retrieval; however, until @@ -275,25 +273,17 @@ validation and formatting required by that MSC. *recommended* profiles be cached no longer than 15 minutes to avoid displaying stale data. Servers MUST not cache for longer than 24 hours to avoid unwanted data persistence. -- Clients COULD provide a UI for users to view/enter their own free-text custom fields in the `u.*` - namespace of their own profile. - -- If clients present custom fields in profiles to users, clients SHOULD provide settings to choose - the scope of profiles they present the freetext (i.e. `u.*` namespaced) fields from, (e.g. none, - only users on the local server, users in the current room, any users sharing a room with this - user, all users globally) with defaults sensitive to their implementation's UI and intended - audience. This proposal *recommends* clients only display profiles of users in the current room - whose membership status is `join`, `invite`, or `knock`, and whose `m.room.member` event has - *not* been redacted. If a client offers an option for custom fields to always be available in the - UI, an option should be provided to hide/minimise them automatically. +- Clients COULD provide a UI for users to view and enter custom fields, respecting the appropriate + namespaces. -- Servers MAY add/remove/modify fields in their own user's global profile data, whether for - moderation purposes or for other policy reasons, e.g. to automatically populate a job title based - on the user's organisation. +- Clients SHOULD only display profiles of users in the current room whose membership status is + `join`, `invite`, or `knock`, and whose `m.room.member` event has *not* been redacted. If a + client offers an option for free-text fields to always be available in the UI, an option SHOULD + be provided to hide/minimise them automatically. -- Servers MAY suppress all extended profile fields to their users (e.g. during a go-live phase); - however, this feature could cause confusion if some users can see fields that other users - cannot, so should be used sparingly. +- Servers MAY add/remove/modify fields in their own users' global profile data, whether for + moderation purposes or for other policy reasons (e.g. to automatically populate a job title based + on the user's organisation). ## Potential Issues @@ -305,13 +295,13 @@ As such, this MSC is designed to be as simple as possible to get initial functio structures implemented widely, so further extensions can be debated, implemented, and tested with due care over time. -As this data is stored only at the global level, it won't allow users to modify fields per-room, -or track historical changes in profile fields. However, many users would struggle to track their -own data across many rooms, and publishing state events for every field change could quickly become -a heavy burden on servers and moderators. +As this data is stored only at the global level, it won't allow users to modify fields per-room or +track historical changes in profile fields. However, many users would struggle to track their own +data across many rooms, and publishing state events for every field change could quickly become a +heavy burden on servers and moderators. -This proposal recommends future MSCs add certain fields to per-room member events when there is -value in doing so - the current free-text fields added by this proposal are not considered to have +This proposal recommends future MSCs only add certain fields to per-room member events when there is +explicit value in doing so — the current fields added by this proposal are not considered to have this value. This proposal also does not offer a method to "broadcast" to other users or homeservers that @@ -340,22 +330,22 @@ profile management. ## Security Considerations Since profile fields are public, the server is not directly responsible for the privacy of the data; -however, clients should make users aware that any information published in their profile will be +however, clients SHOULD make users aware that any information published in their profile will be visible to others on the federated network. Likewise, if a server automatically publishes data in user profile fields (e.g. setting a job title -based on an organisation's internal user database) then they must have consent to do so, and users -should be made aware that data is published on their behalf. +based on an organisation's internal user database), then they SHOULD have consent to do so, and +users SHOULD be made aware that data is published on their behalf. -Homeservers and clients *must* comply relevant privacy regulations, particularly regarding data -deletion and retention. Profile data *should* be cleared when a user is deactivated, and while -homeservers *should* cache remote profiles, they *should* avoid caching beyond 24 hours to minimise +Homeservers and clients SHOULD comply with relevant privacy regulations, particularly regarding data +deletion and retention. Profile data SHOULD be cleared when a user is deactivated, and while +homeservers SHOULD cache remote profiles, they SHOULD avoid caching beyond 24 hours to minimise the risk of unintended data persistence. -To minimise the impact of abuse, clients should offer suitable defaults for the users they will -display the profile fields from. A user may *choose* to display fields from all users globally, -but *by default* profiles should only be shown when the users share the current room and the other -user is in the `join`, `invite`, or `knock` membership states. +To minimise the impact of abuse, clients SHOULD offer suitable defaults for the users they will +display the profile fields from. A user MAY choose to display fields from all users globally, but +*by default* profiles SHOULD only be shown when the users share the current room and the other user +is in the `join`, `invite`, or `knock` membership states. The proposal [MSC4202](https://github.com/matrix-org/matrix-spec-proposals/pull/4202) adds reporting of user profiles over federation, which offers a facility for users to report offensive content to @@ -365,13 +355,13 @@ the homeserver that account is registered on. ### Unstable Profile Fields -Until this proposal is stable, custom fields should use an unstable prefix: +Until this proposal is stable, fields SHOULD use an unstable prefix: ```json { "avatar_url": "mxc://matrix.org/MyC00lAvatar", "displayname": "John Doe", - "uk.tcpip.msc4133.u.Custom Field": "field_value" + "uk.tcpip.msc4133.m.example_field": "field_value" } ``` @@ -379,7 +369,7 @@ Until this proposal is stable, custom fields should use an unstable prefix: Use unstable endpoints when the capability is not yet stable: -- **Get/Set/Delete Custom Fields**: +- **Get/Set/Delete Profile Fields**: - `/_matrix/client/unstable/uk.tcpip.msc4133/profile/{userId}/{key_name}` - **Patch/Put Profile**: @@ -402,13 +392,13 @@ Advertise the capability with an unstable prefix: ### Unstable Client Features -Client feature `uk.tcpip.msc4133` should be advertised on the `/_matrix/client/versions` endpoint +Client feature `uk.tcpip.msc4133` SHOULD be advertised on the `/_matrix/client/versions` endpoint when the `PUT` and `PATCH` methods are accepted on the `/_matrix/client/unstable/uk.tcpip.msc4133/profile/{userId}` endpoint. -Once this MSC is merged, the client feature `uk.tcpip.msc4133.stable` should be advertised when the +Once this MSC is merged, the client feature `uk.tcpip.msc4133.stable` SHOULD be advertised when the `PUT` and `PATCH` methods are accepted on the `/_matrix/client/v3/profile/{userId}` endpoint until -the next spec version where this endpoint is officially written into the spec, e.g. +the next spec version where this endpoint is officially written into the spec, e.g.: ```json { From 09318e80dbaad266f87515d452c390910f724028 Mon Sep 17 00:00:00 2001 From: Tom Foster Date: Fri, 11 Oct 2024 12:26:37 +0100 Subject: [PATCH 27/51] Typo fix --- proposals/4133-extended-profiles.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/proposals/4133-extended-profiles.md b/proposals/4133-extended-profiles.md index 84f9dab4242..76bc3595b67 100644 --- a/proposals/4133-extended-profiles.md +++ b/proposals/4133-extended-profiles.md @@ -261,7 +261,7 @@ with the following considerations: Following this change, for example, clients could use `m.example_field` if that field is defined by the Matrix specification, or `org.example.job_title` for organisation or client-specific fields. The proposal [MSC4175](https://github.com/matrix-org/matrix-spec-proposals/pull/4175) demonstrates -the process of definining new fields in the `m.*` namespace. +the process of defining new fields in the `m.*` namespace. ## Implementation Details From 9b4741e1f19b909173be1f263bbea6b3fa2ebe8c Mon Sep 17 00:00:00 2001 From: Tom Foster Date: Wed, 16 Oct 2024 22:47:07 +0100 Subject: [PATCH 28/51] Clarifications and readability --- proposals/4133-extended-profiles.md | 322 ++++++++++++++-------------- 1 file changed, 164 insertions(+), 158 deletions(-) diff --git a/proposals/4133-extended-profiles.md b/proposals/4133-extended-profiles.md index 76bc3595b67..169c3991f7a 100644 --- a/proposals/4133-extended-profiles.md +++ b/proposals/4133-extended-profiles.md @@ -1,28 +1,30 @@ # MSC4133: Extending User Profile API with Custom Key:Value Pairs -*This proposal aims to enhance user profiles within the Matrix ecosystem by introducing -customisable key:value pairs to global profiles. Allowing users to add arbitrary public information -to their profiles (such as preferred languages, organisational roles, or other relevant public -details) should enrich user interactions without impacting existing functionality.* +This proposal aims to enhance user profiles in the Matrix ecosystem by introducing customisable +key:value pairs to global profiles. By allowing users to add arbitrary public information +(such as preferred languages, organisational roles, or other relevant details) we can enrich +user interactions without impacting existing functionality. -## Proposal +## Proposal Overview -Currently, the Matrix protocol supports limited user profile fields: `avatar_url` and -`displayname`. This proposal extends the API to include custom fields, enabling users to add -key:value pairs to their global profiles. This extension provides a flexible framework for users to -share additional public information. +Currently, the Matrix protocol supports limited user profile fields: `avatar_url` and `displayname`. +This proposal is modelled on the [current API endpoints](https://spec.matrix.org/v1.12/client-server-api/#profiles), +extending the profile API to include extra fields, enabling users and servers to publish key:value +pairs to their global profiles. This extension provides a flexible framework for users to share +additional public information. -This proposal is designed to be simple and compatible with existing clients and servers, -facilitating quick adoption. It complements, rather than replaces, +The proposal is designed to be straightforward and compatible with existing clients and servers, +requiring minimal changes to facilitate quick adoption. It complements, rather than replaces, [MSC1769](https://github.com/matrix-org/matrix-spec-proposals/pull/1769) (Extensible Profiles as -Rooms), by focusing on global profile data without the complexity of per-room profile management. +Rooms) by focusing on global profile data without the complexity of per-room profile management. ## Client-Server API Changes -- **Get a Profile Field** - - **Endpoint**: `GET /_matrix/client/v3/profile/{userId}/{key_name}` - - **Description**: Retrieve the value of a specified `key_name` from a user's profile. - - **Response**: +### Get a Profile Field + +- **Endpoint**: `GET /_matrix/client/v3/profile/{userId}/{key_name}` +- **Description**: Retrieve the value of a specified `key_name` from a user's profile. +- **Response**: ```json { @@ -38,11 +40,12 @@ Rooms), by focusing on global profile data without the complexity of per-room pr } ``` -- **Set a Profile Field** - - **Endpoint**: `PUT /_matrix/client/v3/profile/{userId}/{key_name}` - - **Description**: Set or update the value of a specified `key_name` in the user's profile, - if permitted by the homeserver. - - **Request Body**: +### Set a Profile Field + +- **Endpoint**: `PUT /_matrix/client/v3/profile/{userId}/{key_name}` +- **Description**: Set or update the value of a specified `key_name` in the user's profile, + if permitted by the homeserver. +- **Request Body**: ```json { @@ -58,17 +61,22 @@ Rooms), by focusing on global profile data without the complexity of per-room pr } ``` -*Note**: Setting a `null` value does not delete the key; the key remains with a `null` value. +**Note**: As a `DELETE` endpoint exists to remove a key, setting a `null` value with the `PUT` +method SHOULD NOT delete the key but rather retain it with a `null` value. Servers MAY forbid a +`null` request if their policy does not allow keys to exist with a `null` value. + +### Delete a Profile Field + +- **Endpoint**: `DELETE /_matrix/client/v3/profile/{userId}/{key_name}` +- **Description**: Remove a specified `key_name` (and its value) from the user's profile, + if permitted by the homeserver. -- **Delete a Profile Field** - - **Endpoint**: `DELETE /_matrix/client/v3/profile/{userId}/{key_name}` - - **Description**: Remove a specified `key_name` (and its value) from the user's profile, - if permitted by the homeserver. +### Get All Profile Fields -- **Get All Profile Fields** - - **Endpoint**: `GET /_matrix/client/v3/profile/{userId}` - - **Description**: Retrieve all profile fields for a user. - - **Response**: +- **Endpoint**: `GET /_matrix/client/v3/profile/{userId}` +- **Description**: Retrieve all profile fields for a user, identical to the + [current API](https://spec.matrix.org/latest/client-server-api/#get_matrixclientv3profileuserid). +- **Response**: ```json { @@ -79,11 +87,12 @@ Rooms), by focusing on global profile data without the complexity of per-room pr } ``` -- **Partially Update Profile Fields** - - **Endpoint**: `PATCH /_matrix/client/v3/profile/{userId}` - - **Description**: Merge the provided JSON object into the user's profile, updating - any specified keys without altering others, if permitted by the homeserver. - - **Request Body**: +### Partially Update Profile Fields + +- **Endpoint**: `PATCH /_matrix/client/v3/profile/{userId}` +- **Description**: Merge the provided JSON object into the user's profile, updating any + specified keys without altering others, if permitted by the homeserver. +- **Request Body**: ```json { @@ -93,11 +102,12 @@ Rooms), by focusing on global profile data without the complexity of per-room pr } ``` -- **Replace Profile Fields** - - **Endpoint**: `PUT /_matrix/client/v3/profile/{userId}` - - **Description**: Replace the entire user's profile with the provided JSON object, - adding or updating keys and removing any absent ones, if permitted by the homeserver. - - **Request Body**: +### Replace Profile Fields + +- **Endpoint**: `PUT /_matrix/client/v3/profile/{userId}` +- **Description**: Replace the entire user's profile with the provided JSON object, + adding or updating keys and removing any absent ones, if permitted by the homeserver. +- **Request Body**: ```json { @@ -115,26 +125,26 @@ However, this method allows for bulk updates when needed (e.g. bots managing mul The federation endpoint `GET /_matrix/federation/v1/query/profile` will mirror the client-server API changes to ensure profile information is consistent between local and federated users. -There is no method to verify the history of global profile fields over federation, so this endpoint -MUST only accept requests for local users on the current homeserver, and homeservers MUST only -request a profile from the homeserver specified in that user's MXID. - As per the current stable endpoint, it accepts an optional `field` query string parameter to -request a single field. At time of writing, the Matrix specification says: +request a single field. At the time of writing, the Matrix specification says: > If no `field` was specified, the response should include the fields of the user's profile that > can be made public, such as the display name and avatar. -Given this wording, homeservers currently already have the flexibility to decide whether some -fields are published over federation, and this proposal continues to allow this behaviour. +Given this wording, homeservers currently have the flexibility to decide whether some fields are +published over federation, and this proposal continues to allow this behaviour. ## Capabilities -A new capability `m.profile_fields` controls the ability to *set* custom profile fields and is -advertised on the `GET /_matrix/client/v3/capabilities` endpoint. Clients SHOULD check for this -capability before attempting to create or modify a profile field. +A new capability `m.profile_fields` controls the ability to *set* profile fields and is advertised +on the `GET /_matrix/client/v3/capabilities` endpoint. + +This capability deprecates the use of `m.set_displayname` and `m.set_avatar_url`, which are not +required when this capability is present. -- **Capability Structure**: +Clients SHOULD check for this capability before attempting to create or modify a profile field. + +### Capability Structure ```json { @@ -142,34 +152,54 @@ capability before attempting to create or modify a profile field. "m.profile_fields": { "enabled": true, "allowed": ["m.example_field", "org.example.job_title"], - "disallowed": ["org.example.job_title"] + "disallowed": ["org.example.secret_field"] } } } ``` -- **Behaviour**: - - **When capability is missing**: Clients SHOULD assume extended profiles are supported, and that - they can be created/written to. If a server intends to deny some (or all) changes, it SHOULD - use the capability to advertise this to improve client experience. +### Behaviour - - **When `enabled` is `false`**: Clients SHOULD expect to display profiles but NOT create or - update fields. Any attempt to do so SHOULD result in a `403 Forbidden` error. This does not - affect `avatar_url` and `displayname` fields, which are allowed for compatibility purposes. +- **When capability is missing**: Clients SHOULD assume extended profiles are supported and that + they can be created or modified. If a server intends to deny some (or all) changes, it SHOULD use + the capability to advertise this, improving the client experience. - - **When `enabled` is `true`**: Clients SHOULD allow users to create or update fields, except - those for keys listed in the `disallowed` array, if it exists. Servers MAY optionally specify - the `allowed` array to allowlist fields that users can set - if the `allowed` key is provided - with empty content (e.g. `"allowed": []`), this also disallows the setting of any fields. - If both `allowed` and `disallowed` keys are provided, the `disallowed` one should be ignored. - Clients will receive `400 Bad Request` or `403 Forbidden` responses from the homeserver if - server-side policies prevent them. +- **When `enabled` is `false`**: Clients SHOULD expect to display profiles but NOT create or update + fields. Any attempt to do so SHOULD result in a `403 Forbidden` error. This does not affect + `avatar_url` and `displayname` fields, which are allowed for compatibility purposes. -## Error Handling +- **When `enabled` is `true`**: Clients SHOULD allow users to create or update fields, except those + keys listed in the `disallowed` array. Servers MAY specify an `allowed` array to allowlist fields + that users can set. If both `allowed` and `disallowed` keys are provided, the `disallowed` one + should be ignored. Clients SHOULD receive `400 Bad Request` or `403 Forbidden` responses if + server-side policies prevent them. + +## Key and Namespace Requirements + +Profiles MUST be at most 64 KiB (65,536 bytes) in size, as measured in Canonical JSON, including +the `avatar_url` and `displayname` fields. + +Homeservers SHOULD NOT enforce namespaces, as future expansions may be unknown to the server, but +clients SHOULD use the correct namespace when creating or updating fields. + +Keys MUST follow the [Common Namespaced Identifier Grammar](https://spec.matrix.org/unstable/appendices/#common-namespaced-identifier-grammar), +with the following considerations: + +- **Namespace `m.*`**: Reserved for fields explicitly defined in the Matrix specification. Clients + that do not recognise a field in this namespace MAY attempt to display it but SHOULD NOT attempt + to update the content unless they understand its formatting and validation requirements. + +- **Namespace `tld.name.*`**: For client-specific or unstable fields, using Java package naming + convention (e.g., `com.example.custom_field`). + +Following this change, clients could use `m.example_field` if that field is defined by the Matrix +specification, or `org.example.job_title` for organisation, client-specific fields, or MSC-backed +unstable features. Proposal [MSC4175](https://github.com/matrix-org/matrix-spec-proposals/pull/4175) +demonstrates the process of defining new fields in the `m.*` namespace. -Consistent error codes and messages ensure clear communication of issues: +## Error Handling -### **400 Bad Request**: Request exceeds limits or is malformed +### 400 Bad Request: Request Exceeds Limits or Is Malformed - **`M_BAD_JSON`**: Malformed request. @@ -185,7 +215,7 @@ Consistent error codes and messages ensure clear communication of issues: ```json { "errcode": "M_PROFILE_TOO_LARGE", - "error": "The profile data exceeds the maximum allowed size of 64KiB." + "error": "The profile data exceeds the maximum allowed size of 64 KiB." } ``` @@ -207,12 +237,12 @@ Consistent error codes and messages ensure clear communication of issues: } ``` -### **403 Forbidden**: User lacks permission +### 403 Forbidden: User Lacks Permission -**Note:** See [MSC4170](https://github.com/matrix-org/matrix-spec-proposals/pull/4170) for more +**Note**: See [MSC4170](https://github.com/matrix-org/matrix-spec-proposals/pull/4170) for more discussion on how server policy may result in 403 errors for profile requests. -### **404 Not Found**: Target cannot be found +### 404 Not Found: Target Cannot Be Found - **`M_NOT_FOUND`**: Profile key does not exist. @@ -223,72 +253,48 @@ discussion on how server policy may result in 403 errors for profile requests. } ``` -## Propagation of Profile Fields to Membership Events +## Propagation of Profile Fields The existing fields, `avatar_url` and `displayname`, will continue to trigger state events in each room. These fields are replicated per-room via member events. -All other fields (unless a future proposal specifies otherwise) will **not** trigger state events -in rooms and will exist solely at the global level for storing metadata about the user. - -## Key/Namespace Requirements - -(Whenever "bytes" are referred to as a limit, this is calculated as UTF-8 bytes, so a two-byte code -point consumes two bytes of this limit. This size would be measured before any JSON encoding.) - -Each profile MUST be *at most* 64KiB (65,536 bytes) in size, as measured in Canonical JSON, -including the `avatar_url` and `displayname` fields. - -Homeservers are not expected to enforce namespaces, as future expansions may be unknown to the -server, but clients are expected to use the correct namespace for field creation/updates. - -Keys MUST follow the [Common Namespaced Identifier Grammar](https://spec.matrix.org/unstable/appendices/#common-namespaced-identifier-grammar), -with the following considerations: - -- **Namespace `m.*`**: Reserved for fields explicitly defined in the Matrix specification. These - fields may have special entry/display requirements, so clients that do not recognise a field in - this namespace MAY attempt to display it, but SHOULD NOT attempt to update the content unless - they understand its formatting and validation requirements. +All other fields (unless a future proposal specifies otherwise) WILL NOT trigger state events in +rooms and will exist solely at the global level for storing metadata about the user. -- **Namespace `u.*`**: Reserved for future user-defined custom fields. A future proposal may - justify a need for users to create custom keys within their client, so this namespace has been - reserved for that purpose. Clients and servers SHOULD avoid using this namespace until specified - in a future proposal. - -- **Namespace `tld.name.*`**: For client-specific or unstable fields, using Java package naming - convention (e.g., `com.example.custom_field`). - -Following this change, for example, clients could use `m.example_field` if that field is defined by -the Matrix specification, or `org.example.job_title` for organisation or client-specific fields. -The proposal [MSC4175](https://github.com/matrix-org/matrix-spec-proposals/pull/4175) demonstrates -the process of defining new fields in the `m.*` namespace. +Clients SHOULD consider the increased traffic implications when displaying values (e.g. timezones) +outside of the profile. Servers MAY wish to optimise and relax rate limits on these endpoints in +consideration of this. ## Implementation Details -- Custom fields MUST not trigger state events in rooms; their data MUST not be replicated to +- Custom fields MUST NOT trigger state events in rooms; their data MUST NOT be replicated to `m.room.member` events unless a future proposal creates exceptions for specific fields. -- Servers SHOULD cache remote profiles for *at least* 5 minutes after retrieval; however, until - a future proposal establishes a method for servers to notify each other of updates, it is - *recommended* profiles be cached no longer than 15 minutes to avoid displaying stale data. - Servers MUST not cache for longer than 24 hours to avoid unwanted data persistence. +- Servers SHOULD cache remote profiles for at least 5 minutes after retrieval. Until a method for + servers to notify each other of updates is established, it is recommended that profiles be cached + no longer than an hour to avoid displaying stale data. Servers SHOULD NOT cache profiles for + longer than 24 hours to avoid unwanted data persistence. -- Clients COULD provide a UI for users to view and enter custom fields, respecting the appropriate +- Clients MAY provide a UI for users to view and enter custom fields, respecting the appropriate namespaces. - Clients SHOULD only display profiles of users in the current room whose membership status is - `join`, `invite`, or `knock`, and whose `m.room.member` event has *not* been redacted. If a - client offers an option for free-text fields to always be available in the UI, an option SHOULD - be provided to hide/minimise them automatically. + `join`, `invite`, or `knock`. If a client offers an option for any free-text fields to always be + available in the UI, an option SHOULD be provided to hide or minimise them automatically. -- Servers MAY add/remove/modify fields in their own users' global profile data, whether for - moderation purposes or for other policy reasons (e.g. to automatically populate a job title based - on the user's organisation). +- Servers MAY add, remove, or modify fields in their own users' global profile data, whether for + moderation purposes or for other policy reasons (e.g., to automatically populate a job title + based on the user's organisation). ## Potential Issues -Primarily ensuring uniform support across different servers and clients during the rollout phase: -we don't want users to come to expect other users will check their profile (e.g. languages) before +There is no method to verify the history of global profile fields over federation. This proposal +updates the global profile only, while other more complex proposals, such as +[MSC1769](https://github.com/matrix-org/matrix-spec-proposals/pull/1769) (Extensible Profiles as +Rooms), offer additional mechanisms for users to track changes to their profile data over time. + +Ensuring uniform support across different servers and clients during the rollout phase is crucial. +We do not want users to expect that others will check their profile (e.g. languages) before communicating with them unless most clients and servers support this feature. As such, this MSC is designed to be as simple as possible to get initial functionality and data @@ -296,36 +302,20 @@ structures implemented widely, so further extensions can be debated, implemented due care over time. As this data is stored only at the global level, it won't allow users to modify fields per-room or -track historical changes in profile fields. However, many users would struggle to track their own -data across many rooms, and publishing state events for every field change could quickly become a -heavy burden on servers and moderators. - -This proposal recommends future MSCs only add certain fields to per-room member events when there is -explicit value in doing so — the current fields added by this proposal are not considered to have -this value. +track historical changes in profile fields. This proposal recommends that future MSCs only add +certain fields to per-room member events when there is explicit value in doing so, and the current +functionality added by this proposal is not anticipated to have this value. This proposal also does not offer a method to "broadcast" to other users or homeservers that changes have occurred. This is intentional to keep the scope of this change narrow and maximise -compatibility with existing servers. A future proposal may wish to use an EDU (such as Presence) to -notify users and homeservers that these custom fields have been updated. +compatibility with existing servers. A future proposal may wish to use an EDU (such as Presence) +to notify users and homeservers that these custom fields have been updated. This proposal does not directly address reporting of user profiles over federation, but [MSC4202](https://github.com/matrix-org/matrix-spec-proposals/pull/4202) offers a facility for users to report offensive content to the homeserver that account belongs to. This proposal is not -dependent on [MSC4202](https://github.com/matrix-org/matrix-spec-proposals/pull/4202) but encourages -the use of moderation options to allow users to report offensive content. - -## Alternatives - -An alternative approach could involve introducing a completely new API for extended profile -information. However, this may lead to increased complexity for client and server implementations. - -At the time of writing, Extensible Profiles as Rooms -([MSC1769](https://github.com/matrix-org/matrix-spec-proposals/pull/1769) and variant -[MSC4201](https://github.com/matrix-org/matrix-spec-proposals/pull/4201)) is under development for -richer and more granular content and privacy controls, which this proposal does not intend to -replace - this proposal focuses on basic global profile data without the complexity of per-room -profile management. +dependent on MSC4202 but encourages the use of moderation options to allow users to report +offensive content. ## Security Considerations @@ -337,19 +327,35 @@ Likewise, if a server automatically publishes data in user profile fields (e.g. based on an organisation's internal user database), then they SHOULD have consent to do so, and users SHOULD be made aware that data is published on their behalf. -Homeservers and clients SHOULD comply with relevant privacy regulations, particularly regarding data -deletion and retention. Profile data SHOULD be cleared when a user is deactivated, and while -homeservers SHOULD cache remote profiles, they SHOULD avoid caching beyond 24 hours to minimise -the risk of unintended data persistence. +To minimise the impact of abuse, clients SHOULD offer suitable defaults for displaying user-entered +content. A user MAY choose to display fields from all users globally, but *by default* profiles +SHOULD only be shown when the users share the current room and the other user is in the `join`, +`invite`, or `knock` membership states. + +If future clients (or spec proposals) implement the ability to set custom user-entered text in +profiles, servers MAY require additional moderation and safety tooling on the server side to +provide better visibility of problematic content, e.g., during reports. -To minimise the impact of abuse, clients SHOULD offer suitable defaults for the users they will -display the profile fields from. A user MAY choose to display fields from all users globally, but -*by default* profiles SHOULD only be shown when the users share the current room and the other user -is in the `join`, `invite`, or `knock` membership states. +Proposal [MSC4202](https://github.com/matrix-org/matrix-spec-proposals/pull/4202) adds reporting of +user profiles over federation, which offers a facility for users to report offensive content to the +homeserver that account is registered on. -The proposal [MSC4202](https://github.com/matrix-org/matrix-spec-proposals/pull/4202) adds reporting -of user profiles over federation, which offers a facility for users to report offensive content to -the homeserver that account is registered on. +Homeservers and clients SHOULD comply with relevant privacy regulations, particularly regarding +data deletion and retention. Profile data SHOULD be cleared when a user is deactivated, and while +homeservers SHOULD cache remote profiles, they SHOULD avoid caching beyond 24 hours to minimise the +risk of unintended data persistence. + +## Alternatives + +An alternative approach could involve introducing a completely new API for extended profile +information. However, this may lead to increased complexity for client and server implementations. + +At the time of writing, Extensible Profiles as Rooms +([MSC1769](https://github.com/matrix-org/matrix-spec-proposals/pull/1769) and variant +[MSC4201](https://github.com/matrix-org/matrix-spec-proposals/pull/4201)) is under development for +richer and more granular content and privacy controls, which this proposal does not intend to +replace. This proposal focuses on basic global profile data without the complexity of per-room +profile management. ## Unstable Prefixes @@ -384,7 +390,7 @@ Advertise the capability with an unstable prefix: "capabilities": { "uk.tcpip.msc4133.profile_fields": { "enabled": true, - "disallowed": ["org.example.job_title"] + "disallowed": ["org.example.secret_field"] } } } @@ -392,8 +398,8 @@ Advertise the capability with an unstable prefix: ### Unstable Client Features -Client feature `uk.tcpip.msc4133` SHOULD be advertised on the `/_matrix/client/versions` endpoint -when the `PUT` and `PATCH` methods are accepted on the +The client feature `uk.tcpip.msc4133` SHOULD be advertised on the `/_matrix/client/versions` +endpoint when the `PUT` and `PATCH` methods are accepted on the `/_matrix/client/unstable/uk.tcpip.msc4133/profile/{userId}` endpoint. Once this MSC is merged, the client feature `uk.tcpip.msc4133.stable` SHOULD be advertised when the From 068d44e63909babc62e120d54820b731fad7c8a1 Mon Sep 17 00:00:00 2001 From: Tom Foster Date: Mon, 4 Nov 2024 13:00:32 +0000 Subject: [PATCH 29/51] Correct key length error to match common grammar limit --- proposals/4133-extended-profiles.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/proposals/4133-extended-profiles.md b/proposals/4133-extended-profiles.md index 169c3991f7a..b15a6277b86 100644 --- a/proposals/4133-extended-profiles.md +++ b/proposals/4133-extended-profiles.md @@ -224,7 +224,7 @@ demonstrates the process of defining new fields in the `m.*` namespace. ```json { "errcode": "M_KEY_TOO_LARGE", - "error": "The key name exceeds the maximum allowed length of 128 bytes." + "error": "The key name exceeds the maximum allowed length of 255 bytes." } ``` From fa381dafa0f574ac86cba09b5a86a711149e99de Mon Sep 17 00:00:00 2001 From: Patrick Cloke Date: Fri, 20 Dec 2024 13:20:49 -0500 Subject: [PATCH 30/51] Remove PATCH/PUT. --- proposals/4133-extended-profiles.md | 42 +++-------------------------- 1 file changed, 3 insertions(+), 39 deletions(-) diff --git a/proposals/4133-extended-profiles.md b/proposals/4133-extended-profiles.md index b15a6277b86..046384c2341 100644 --- a/proposals/4133-extended-profiles.md +++ b/proposals/4133-extended-profiles.md @@ -87,39 +87,6 @@ method SHOULD NOT delete the key but rather retain it with a `null` value. Serve } ``` -### Partially Update Profile Fields - -- **Endpoint**: `PATCH /_matrix/client/v3/profile/{userId}` -- **Description**: Merge the provided JSON object into the user's profile, updating any - specified keys without altering others, if permitted by the homeserver. -- **Request Body**: - -```json -{ - "avatar_url": "mxc://matrix.org/MyNewAvatar", - "displayname": "John Doe", - "m.example_field": "new_value1" -} -``` - -### Replace Profile Fields - -- **Endpoint**: `PUT /_matrix/client/v3/profile/{userId}` -- **Description**: Replace the entire user's profile with the provided JSON object, - adding or updating keys and removing any absent ones, if permitted by the homeserver. -- **Request Body**: - -```json -{ - "avatar_url": "mxc://matrix.org/MyNewAvatar", - "displayname": "John Doe", - "m.example_field": "new_value1" -} -``` - -**Note**: Clients are encouraged to manipulate fields individually to avoid race conditions. -However, this method allows for bulk updates when needed (e.g. bots managing multiple accounts). - ## Server-Server API Changes The federation endpoint `GET /_matrix/federation/v1/query/profile` will mirror the client-server @@ -239,12 +206,12 @@ demonstrates the process of defining new fields in the `m.*` namespace. ### 403 Forbidden: User Lacks Permission -**Note**: See [MSC4170](https://github.com/matrix-org/matrix-spec-proposals/pull/4170) for more -discussion on how server policy may result in 403 errors for profile requests. +Unchanged from the [current spec](https://spec.matrix.org/v1.13/client-server-api/#server-behaviour). ### 404 Not Found: Target Cannot Be Found -- **`M_NOT_FOUND`**: Profile key does not exist. +- **`M_NOT_FOUND`**: Profile key does not exist (this is unchanged, just expanded to + apply to arbitrary keys). ```json { @@ -378,9 +345,6 @@ Use unstable endpoints when the capability is not yet stable: - **Get/Set/Delete Profile Fields**: - `/_matrix/client/unstable/uk.tcpip.msc4133/profile/{userId}/{key_name}` -- **Patch/Put Profile**: - - `/_matrix/client/unstable/uk.tcpip.msc4133/profile/{userId}` - ### Unstable Capability Advertise the capability with an unstable prefix: From 334912371bb060d5c57c0bfb6e298b5e1abc6b92 Mon Sep 17 00:00:00 2001 From: Tom Foster Date: Fri, 3 Jan 2025 22:08:14 +0000 Subject: [PATCH 31/51] Removed redundant sections after `PUT` and `PATCH` methods removed --- proposals/4133-extended-profiles.md | 33 +---------------------------- 1 file changed, 1 insertion(+), 32 deletions(-) diff --git a/proposals/4133-extended-profiles.md b/proposals/4133-extended-profiles.md index 046384c2341..7cceef668c5 100644 --- a/proposals/4133-extended-profiles.md +++ b/proposals/4133-extended-profiles.md @@ -195,15 +195,6 @@ demonstrates the process of defining new fields in the `m.*` namespace. } ``` -- **`M_TOO_MANY_KEYS`**: Exceeds key limits. - -```json -{ - "errcode": "M_TOO_MANY_KEYS", - "error": "The user has exceeded the maximum number of allowed keys in their profile." -} -``` - ### 403 Forbidden: User Lacks Permission Unchanged from the [current spec](https://spec.matrix.org/v1.13/client-server-api/#server-behaviour). @@ -358,26 +349,4 @@ Advertise the capability with an unstable prefix: } } } -``` - -### Unstable Client Features - -The client feature `uk.tcpip.msc4133` SHOULD be advertised on the `/_matrix/client/versions` -endpoint when the `PUT` and `PATCH` methods are accepted on the -`/_matrix/client/unstable/uk.tcpip.msc4133/profile/{userId}` endpoint. - -Once this MSC is merged, the client feature `uk.tcpip.msc4133.stable` SHOULD be advertised when the -`PUT` and `PATCH` methods are accepted on the `/_matrix/client/v3/profile/{userId}` endpoint until -the next spec version where this endpoint is officially written into the spec, e.g.: - -```json -{ - "unstable_features": { - "uk.tcpip.msc4133": true, - "uk.tcpip.msc4133.stable": true - }, - "versions": [ - "v1.11" - ] -} -``` +``` \ No newline at end of file From 9fa009607fcec1419454dc120464a4dc0e5abef9 Mon Sep 17 00:00:00 2001 From: Tom Foster Date: Sat, 4 Jan 2025 10:46:07 +0000 Subject: [PATCH 32/51] Re-add client feature for managing profile fields --- proposals/4133-extended-profiles.md | 24 +++++++++++++++++++++++- 1 file changed, 23 insertions(+), 1 deletion(-) diff --git a/proposals/4133-extended-profiles.md b/proposals/4133-extended-profiles.md index 7cceef668c5..205346cc88d 100644 --- a/proposals/4133-extended-profiles.md +++ b/proposals/4133-extended-profiles.md @@ -349,4 +349,26 @@ Advertise the capability with an unstable prefix: } } } -``` \ No newline at end of file +``` + +### Unstable Client Features + +The client feature `uk.tcpip.msc4133` SHOULD be advertised on the `/_matrix/client/versions` +endpoint when the unstable endpoints for managing profile fields are supported at +`/_matrix/client/unstable/uk.tcpip.msc4133/profile/{userId}/{key_name}`. + +Once this MSC is merged, the client feature `uk.tcpip.msc4133.stable` SHOULD be advertised when +these endpoints are accepted at `/_matrix/client/v3/profile/{userId}/{key_name}` until the next +spec version where these endpoints are officially written into the spec, e.g. + +```json +{ + "unstable_features": { + "uk.tcpip.msc4133": true, + "uk.tcpip.msc4133.stable": true + }, + "versions": [ + "v1.11" + ] +} +``` From da0a79181f91a713dffe6978b340685d24fb27a4 Mon Sep 17 00:00:00 2001 From: Tom Foster Date: Sun, 5 Jan 2025 18:04:28 +0000 Subject: [PATCH 33/51] Update proposals/4133-extended-profiles.md Co-authored-by: Travis Ralston --- proposals/4133-extended-profiles.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/proposals/4133-extended-profiles.md b/proposals/4133-extended-profiles.md index 205346cc88d..133a5754ab1 100644 --- a/proposals/4133-extended-profiles.md +++ b/proposals/4133-extended-profiles.md @@ -149,7 +149,7 @@ the `avatar_url` and `displayname` fields. Homeservers SHOULD NOT enforce namespaces, as future expansions may be unknown to the server, but clients SHOULD use the correct namespace when creating or updating fields. -Keys MUST follow the [Common Namespaced Identifier Grammar](https://spec.matrix.org/unstable/appendices/#common-namespaced-identifier-grammar), +Keys MUST follow the [Common Namespaced Identifier Grammar](https://spec.matrix.org/v1.13/appendices/#common-namespaced-identifier-grammar), with the following considerations: - **Namespace `m.*`**: Reserved for fields explicitly defined in the Matrix specification. Clients From a948f5d03a7e25fc6d6ddea6f855a5733b165282 Mon Sep 17 00:00:00 2001 From: Tom Foster Date: Sun, 5 Jan 2025 18:08:53 +0000 Subject: [PATCH 34/51] Clarify 403 error scenarios --- proposals/4133-extended-profiles.md | 15 ++++++++++++++- 1 file changed, 14 insertions(+), 1 deletion(-) diff --git a/proposals/4133-extended-profiles.md b/proposals/4133-extended-profiles.md index 133a5754ab1..4204c11b298 100644 --- a/proposals/4133-extended-profiles.md +++ b/proposals/4133-extended-profiles.md @@ -197,7 +197,20 @@ demonstrates the process of defining new fields in the `m.*` namespace. ### 403 Forbidden: User Lacks Permission -Unchanged from the [current spec](https://spec.matrix.org/v1.13/client-server-api/#server-behaviour). +A server may return this error in several scenarios: + +- When the user lacks permission to modify another user's profile +- When the capability `m.profile_fields` is disabled (`enabled: false`) +- When the server denies setting/creating a specific field value, even if the capability allows it + (for example, due to content policy violations or server-side validation rules) +- When the user is not allowed to modify profiles at all + +```json +{ + "errcode": "M_FORBIDDEN", + "error": "You do not have permission to perform this operation" +} +``` ### 404 Not Found: Target Cannot Be Found From c82dab67d114ded78ef4b6a5a1f9f264002d6666 Mon Sep 17 00:00:00 2001 From: Tom Foster Date: Sun, 5 Jan 2025 18:14:13 +0000 Subject: [PATCH 35/51] Add section on caching behaviour under S-S API --- proposals/4133-extended-profiles.md | 17 ++++++++++++----- 1 file changed, 12 insertions(+), 5 deletions(-) diff --git a/proposals/4133-extended-profiles.md b/proposals/4133-extended-profiles.md index 4204c11b298..8ab04c8e67b 100644 --- a/proposals/4133-extended-profiles.md +++ b/proposals/4133-extended-profiles.md @@ -101,6 +101,18 @@ request a single field. At the time of writing, the Matrix specification says: Given this wording, homeservers currently have the flexibility to decide whether some fields are published over federation, and this proposal continues to allow this behaviour. +### Caching Behaviour + +To reduce federation traffic and improve performance, servers SHOULD implement the following +caching behaviour: + +- Cache remote profiles for at least 5 minutes after retrieval +- Do not cache profiles longer than 1 hour without revalidation, to avoid displaying stale data +- Never cache profiles longer than 24 hours, to avoid unwanted data persistence +- Consider implementing exponential backoff for failed retrievals + +This guidance may be updated when a method for servers to notify each other of profile updates is established. + ## Capabilities A new capability `m.profile_fields` controls the ability to *set* profile fields and is advertised @@ -241,11 +253,6 @@ consideration of this. - Custom fields MUST NOT trigger state events in rooms; their data MUST NOT be replicated to `m.room.member` events unless a future proposal creates exceptions for specific fields. -- Servers SHOULD cache remote profiles for at least 5 minutes after retrieval. Until a method for - servers to notify each other of updates is established, it is recommended that profiles be cached - no longer than an hour to avoid displaying stale data. Servers SHOULD NOT cache profiles for - longer than 24 hours to avoid unwanted data persistence. - - Clients MAY provide a UI for users to view and enter custom fields, respecting the appropriate namespaces. From 21ad83f5dd94f9cff0cecc67f721f247073b472c Mon Sep 17 00:00:00 2001 From: Tom Foster Date: Sun, 5 Jan 2025 18:18:13 +0000 Subject: [PATCH 36/51] Link to Canonical JSON in current spec --- proposals/4133-extended-profiles.md | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/proposals/4133-extended-profiles.md b/proposals/4133-extended-profiles.md index 8ab04c8e67b..5f1aa77481f 100644 --- a/proposals/4133-extended-profiles.md +++ b/proposals/4133-extended-profiles.md @@ -155,8 +155,9 @@ Clients SHOULD check for this capability before attempting to create or modify a ## Key and Namespace Requirements -Profiles MUST be at most 64 KiB (65,536 bytes) in size, as measured in Canonical JSON, including -the `avatar_url` and `displayname` fields. +Profiles MUST be at most 64 KiB (65,536 bytes) in size, as measured in +[Canonical JSON](https://spec.matrix.org/v1.13/appendices/#canonical-json), including the +`avatar_url` and `displayname` fields. Homeservers SHOULD NOT enforce namespaces, as future expansions may be unknown to the server, but clients SHOULD use the correct namespace when creating or updating fields. From 1726ef35ec31d39116608796cfc0c4530aa775bc Mon Sep 17 00:00:00 2001 From: Tom Foster Date: Sun, 5 Jan 2025 18:21:38 +0000 Subject: [PATCH 37/51] Cut down instructions for clients on when to display content from federated users. --- proposals/4133-extended-profiles.md | 11 +++-------- 1 file changed, 3 insertions(+), 8 deletions(-) diff --git a/proposals/4133-extended-profiles.md b/proposals/4133-extended-profiles.md index 5f1aa77481f..c0e3c1f34b3 100644 --- a/proposals/4133-extended-profiles.md +++ b/proposals/4133-extended-profiles.md @@ -306,14 +306,9 @@ Likewise, if a server automatically publishes data in user profile fields (e.g. based on an organisation's internal user database), then they SHOULD have consent to do so, and users SHOULD be made aware that data is published on their behalf. -To minimise the impact of abuse, clients SHOULD offer suitable defaults for displaying user-entered -content. A user MAY choose to display fields from all users globally, but *by default* profiles -SHOULD only be shown when the users share the current room and the other user is in the `join`, -`invite`, or `knock` membership states. - -If future clients (or spec proposals) implement the ability to set custom user-entered text in -profiles, servers MAY require additional moderation and safety tooling on the server side to -provide better visibility of problematic content, e.g., during reports. +To minimise potential abuse risks, clients should consider appropriate defaults for displaying +user-entered content. Profile content is publicly visible to other users on the federated network, +so clients may wish to implement controls around when and how profile fields are displayed. Proposal [MSC4202](https://github.com/matrix-org/matrix-spec-proposals/pull/4202) adds reporting of user profiles over federation, which offers a facility for users to report offensive content to the From 30d203ee6505c4fd1e7ba867addc9d61e6cb4808 Mon Sep 17 00:00:00 2001 From: Tom Foster Date: Sun, 5 Jan 2025 18:27:09 +0000 Subject: [PATCH 38/51] Revert c82dab67d114ded78ef4b6a5a1f9f264002d6666 --- proposals/4133-extended-profiles.md | 17 +++++------------ 1 file changed, 5 insertions(+), 12 deletions(-) diff --git a/proposals/4133-extended-profiles.md b/proposals/4133-extended-profiles.md index c0e3c1f34b3..8f90e215a2f 100644 --- a/proposals/4133-extended-profiles.md +++ b/proposals/4133-extended-profiles.md @@ -101,18 +101,6 @@ request a single field. At the time of writing, the Matrix specification says: Given this wording, homeservers currently have the flexibility to decide whether some fields are published over federation, and this proposal continues to allow this behaviour. -### Caching Behaviour - -To reduce federation traffic and improve performance, servers SHOULD implement the following -caching behaviour: - -- Cache remote profiles for at least 5 minutes after retrieval -- Do not cache profiles longer than 1 hour without revalidation, to avoid displaying stale data -- Never cache profiles longer than 24 hours, to avoid unwanted data persistence -- Consider implementing exponential backoff for failed retrievals - -This guidance may be updated when a method for servers to notify each other of profile updates is established. - ## Capabilities A new capability `m.profile_fields` controls the ability to *set* profile fields and is advertised @@ -254,6 +242,11 @@ consideration of this. - Custom fields MUST NOT trigger state events in rooms; their data MUST NOT be replicated to `m.room.member` events unless a future proposal creates exceptions for specific fields. +- Servers SHOULD cache remote profiles for at least 5 minutes after retrieval. Until a method for + servers to notify each other of updates is established, it is recommended that profiles be cached + no longer than an hour to avoid displaying stale data. Servers SHOULD NOT cache profiles for + longer than 24 hours to avoid unwanted data persistence. + - Clients MAY provide a UI for users to view and enter custom fields, respecting the appropriate namespaces. From 43e1e1a13f8bbd3a58373e0fe8f3a93029ea9914 Mon Sep 17 00:00:00 2001 From: Tom Foster Date: Sun, 5 Jan 2025 18:31:28 +0000 Subject: [PATCH 39/51] Clarify caching and freshness challenges --- proposals/4133-extended-profiles.md | 12 +++++++----- 1 file changed, 7 insertions(+), 5 deletions(-) diff --git a/proposals/4133-extended-profiles.md b/proposals/4133-extended-profiles.md index 8f90e215a2f..e1be41f903a 100644 --- a/proposals/4133-extended-profiles.md +++ b/proposals/4133-extended-profiles.md @@ -242,10 +242,11 @@ consideration of this. - Custom fields MUST NOT trigger state events in rooms; their data MUST NOT be replicated to `m.room.member` events unless a future proposal creates exceptions for specific fields. -- Servers SHOULD cache remote profiles for at least 5 minutes after retrieval. Until a method for - servers to notify each other of updates is established, it is recommended that profiles be cached - no longer than an hour to avoid displaying stale data. Servers SHOULD NOT cache profiles for - longer than 24 hours to avoid unwanted data persistence. +- Servers MAY cache remote profiles to optimise performance. Until a method for servers to notify + each other of updates is established, it is recommended that profiles be cached *no longer* than + an hour to avoid displaying stale data. If caching is implemented, servers SHOULD invalidate their + cache when receiving new profile information. A future MSC may propose a mechanism for servers to + notify each other of profile updates. - Clients MAY provide a UI for users to view and enter custom fields, respecting the appropriate namespaces. @@ -281,7 +282,8 @@ functionality added by this proposal is not anticipated to have this value. This proposal also does not offer a method to "broadcast" to other users or homeservers that changes have occurred. This is intentional to keep the scope of this change narrow and maximise compatibility with existing servers. A future proposal may wish to use an EDU (such as Presence) -to notify users and homeservers that these custom fields have been updated. +to notify users and homeservers that these custom fields have been updated. This would allow +servers to cache profile data more effectively without compromising on user experience. This proposal does not directly address reporting of user profiles over federation, but [MSC4202](https://github.com/matrix-org/matrix-spec-proposals/pull/4202) offers a facility for From af87bbeb26c6d33725d970676009b52056a87f96 Mon Sep 17 00:00:00 2001 From: Tom Foster Date: Sun, 5 Jan 2025 18:37:00 +0000 Subject: [PATCH 40/51] Adjust abuse section --- proposals/4133-extended-profiles.md | 8 +++++--- 1 file changed, 5 insertions(+), 3 deletions(-) diff --git a/proposals/4133-extended-profiles.md b/proposals/4133-extended-profiles.md index e1be41f903a..8276145cbd0 100644 --- a/proposals/4133-extended-profiles.md +++ b/proposals/4133-extended-profiles.md @@ -301,9 +301,11 @@ Likewise, if a server automatically publishes data in user profile fields (e.g. based on an organisation's internal user database), then they SHOULD have consent to do so, and users SHOULD be made aware that data is published on their behalf. -To minimise potential abuse risks, clients should consider appropriate defaults for displaying -user-entered content. Profile content is publicly visible to other users on the federated network, -so clients may wish to implement controls around when and how profile fields are displayed. +To minimise the impact of abuse, clients should carefully consider when and how to display +user-entered profile content. While some clients may choose to show profile fields globally, others +may restrict visibility based on room membership or other trust signals. Clients should be aware +that profile fields may contain abusive content and implement appropriate safety measures based on +their risk assessment. Proposal [MSC4202](https://github.com/matrix-org/matrix-spec-proposals/pull/4202) adds reporting of user profiles over federation, which offers a facility for users to report offensive content to the From f686090d704998ee9dc0de8421047721c3b3208a Mon Sep 17 00:00:00 2001 From: Tom Foster Date: Sun, 5 Jan 2025 18:47:52 +0000 Subject: [PATCH 41/51] Authentication/rate-limiting/guest access requirements --- proposals/4133-extended-profiles.md | 26 +++++++++++++++++++++----- 1 file changed, 21 insertions(+), 5 deletions(-) diff --git a/proposals/4133-extended-profiles.md b/proposals/4133-extended-profiles.md index 8276145cbd0..c2c0309f9b8 100644 --- a/proposals/4133-extended-profiles.md +++ b/proposals/4133-extended-profiles.md @@ -18,12 +18,24 @@ requiring minimal changes to facilitate quick adoption. It complements, rather t [MSC1769](https://github.com/matrix-org/matrix-spec-proposals/pull/1769) (Extensible Profiles as Rooms) by focusing on global profile data without the complexity of per-room profile management. +## Authentication and Rate Limiting + +All endpoints in this proposal follow the standard client-server API authentication rules. Specifically: + +- All endpoints require authentication except for GET requests which may be accessed without + authentication +- Servers MUST verify the access token has permission to modify the requested userId's profile +- Rate limiting SHOULD be applied as per the homeserver's normal profile endpoint limits +- Guest access follows the same rules as existing profile endpoints - guests may view profiles but + not modify them + ## Client-Server API Changes ### Get a Profile Field - **Endpoint**: `GET /_matrix/client/v3/profile/{userId}/{key_name}` - **Description**: Retrieve the value of a specified `key_name` from a user's profile. +- **Pagination**: Not applicable, returns a single bounded key-value pair - **Response**: ```json @@ -76,6 +88,7 @@ method SHOULD NOT delete the key but rather retain it with a `null` value. Serve - **Endpoint**: `GET /_matrix/client/v3/profile/{userId}` - **Description**: Retrieve all profile fields for a user, identical to the [current API](https://spec.matrix.org/latest/client-server-api/#get_matrixclientv3profileuserid). +- **Pagination**: Not applicable, the full profile is bounded at 64 KiB total size - **Response**: ```json @@ -301,11 +314,14 @@ Likewise, if a server automatically publishes data in user profile fields (e.g. based on an organisation's internal user database), then they SHOULD have consent to do so, and users SHOULD be made aware that data is published on their behalf. -To minimise the impact of abuse, clients should carefully consider when and how to display -user-entered profile content. While some clients may choose to show profile fields globally, others -may restrict visibility based on room membership or other trust signals. Clients should be aware -that profile fields may contain abusive content and implement appropriate safety measures based on -their risk assessment. +To minimise the impact of abuse, clients SHOULD offer suitable defaults for displaying user-entered +content. A user MAY choose to display fields from all users globally, but *by default* profiles +SHOULD only be shown when the users share the current room and the other user is in the `join`, +`invite`, or `knock` membership states. + +If future clients (or spec proposals) implement the ability to set custom user-entered text in +profiles, servers MAY require additional moderation and safety tooling on the server side to +provide better visibility of problematic content, e.g., during reports. Proposal [MSC4202](https://github.com/matrix-org/matrix-spec-proposals/pull/4202) adds reporting of user profiles over federation, which offers a facility for users to report offensive content to the From 17cc30bf02b535b645da445d28b396c565a669c0 Mon Sep 17 00:00:00 2001 From: Tom Foster Date: Sun, 5 Jan 2025 18:51:11 +0000 Subject: [PATCH 42/51] Un-revert accidental revert of af87bbeb26c6d33725d970676009b52056a87f96 --- proposals/4133-extended-profiles.md | 13 +++++-------- 1 file changed, 5 insertions(+), 8 deletions(-) diff --git a/proposals/4133-extended-profiles.md b/proposals/4133-extended-profiles.md index c2c0309f9b8..ffee37f434e 100644 --- a/proposals/4133-extended-profiles.md +++ b/proposals/4133-extended-profiles.md @@ -314,14 +314,11 @@ Likewise, if a server automatically publishes data in user profile fields (e.g. based on an organisation's internal user database), then they SHOULD have consent to do so, and users SHOULD be made aware that data is published on their behalf. -To minimise the impact of abuse, clients SHOULD offer suitable defaults for displaying user-entered -content. A user MAY choose to display fields from all users globally, but *by default* profiles -SHOULD only be shown when the users share the current room and the other user is in the `join`, -`invite`, or `knock` membership states. - -If future clients (or spec proposals) implement the ability to set custom user-entered text in -profiles, servers MAY require additional moderation and safety tooling on the server side to -provide better visibility of problematic content, e.g., during reports. +To minimise the impact of abuse, clients should carefully consider when and how to display +user-entered profile content. While some clients may choose to show profile fields globally, others +may restrict visibility based on room membership or other trust signals. Clients should be aware +that profile fields may contain abusive content and implement appropriate safety measures based on +their risk assessment. Proposal [MSC4202](https://github.com/matrix-org/matrix-spec-proposals/pull/4202) adds reporting of user profiles over federation, which offers a facility for users to report offensive content to the From d620259ac07cb38d43fdbb2880ba47a452ed6522 Mon Sep 17 00:00:00 2001 From: Tom Foster Date: Tue, 7 Jan 2025 23:51:14 +0000 Subject: [PATCH 43/51] Simplify caching recommendations --- proposals/4133-extended-profiles.md | 8 +++----- 1 file changed, 3 insertions(+), 5 deletions(-) diff --git a/proposals/4133-extended-profiles.md b/proposals/4133-extended-profiles.md index ffee37f434e..5d0fe4d05b5 100644 --- a/proposals/4133-extended-profiles.md +++ b/proposals/4133-extended-profiles.md @@ -255,11 +255,9 @@ consideration of this. - Custom fields MUST NOT trigger state events in rooms; their data MUST NOT be replicated to `m.room.member` events unless a future proposal creates exceptions for specific fields. -- Servers MAY cache remote profiles to optimise performance. Until a method for servers to notify - each other of updates is established, it is recommended that profiles be cached *no longer* than - an hour to avoid displaying stale data. If caching is implemented, servers SHOULD invalidate their - cache when receiving new profile information. A future MSC may propose a mechanism for servers to - notify each other of profile updates. +- Servers MAY cache remote profiles to optimise performance. Servers which prefer to cache details + should do so for a short period of time to avoid stale data being presented to users. + A future MSC may propose a mechanism for servers to notify each other of profile updates. - Clients MAY provide a UI for users to view and enter custom fields, respecting the appropriate namespaces. From c1b419aa9d92fbf3f36cb44e4586102e2caa2361 Mon Sep 17 00:00:00 2001 From: Tom Foster Date: Fri, 10 Jan 2025 08:17:22 +0000 Subject: [PATCH 44/51] Up to clients whether they check capability Co-authored-by: Richard van der Hoff <1389908+richvdh@users.noreply.github.com> --- proposals/4133-extended-profiles.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/proposals/4133-extended-profiles.md b/proposals/4133-extended-profiles.md index 5d0fe4d05b5..ecf2a92f936 100644 --- a/proposals/4133-extended-profiles.md +++ b/proposals/4133-extended-profiles.md @@ -122,7 +122,7 @@ on the `GET /_matrix/client/v3/capabilities` endpoint. This capability deprecates the use of `m.set_displayname` and `m.set_avatar_url`, which are not required when this capability is present. -Clients SHOULD check for this capability before attempting to create or modify a profile field. +Clients MAY check for this capability before attempting to create or modify a profile field. ### Capability Structure From 4b3a8edefc9aecee2174ea5540bcb35d8a7b594c Mon Sep 17 00:00:00 2001 From: Tom Foster Date: Fri, 10 Jan 2025 08:18:19 +0000 Subject: [PATCH 45/51] Technically correct is the best kind of correct Co-authored-by: Richard van der Hoff <1389908+richvdh@users.noreply.github.com> --- proposals/4133-extended-profiles.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/proposals/4133-extended-profiles.md b/proposals/4133-extended-profiles.md index ecf2a92f936..dc58aea62e5 100644 --- a/proposals/4133-extended-profiles.md +++ b/proposals/4133-extended-profiles.md @@ -205,7 +205,7 @@ demonstrates the process of defining new fields in the `m.*` namespace. ```json { "errcode": "M_KEY_TOO_LARGE", - "error": "The key name exceeds the maximum allowed length of 255 bytes." + "error": "The key name exceeds the maximum allowed length of 255 characters." } ``` From 0f41c8205ef1851ad7656bbd053013ca4484c001 Mon Sep 17 00:00:00 2001 From: Tom Foster Date: Fri, 10 Jan 2025 08:19:29 +0000 Subject: [PATCH 46/51] Link to current federation profile endpoint Co-authored-by: Richard van der Hoff <1389908+richvdh@users.noreply.github.com> --- proposals/4133-extended-profiles.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/proposals/4133-extended-profiles.md b/proposals/4133-extended-profiles.md index dc58aea62e5..099212c6fab 100644 --- a/proposals/4133-extended-profiles.md +++ b/proposals/4133-extended-profiles.md @@ -102,7 +102,7 @@ method SHOULD NOT delete the key but rather retain it with a `null` value. Serve ## Server-Server API Changes -The federation endpoint `GET /_matrix/federation/v1/query/profile` will mirror the client-server +The federation endpoint [`GET /_matrix/federation/v1/query/profile](https://spec.matrix.org/v1.13/server-server-api/#get_matrixfederationv1queryprofile)` will mirror the client-server API changes to ensure profile information is consistent between local and federated users. As per the current stable endpoint, it accepts an optional `field` query string parameter to From 1b98d40c1ec321395d4b6fef51029508b4c91f9a Mon Sep 17 00:00:00 2001 From: Tom Foster Date: Fri, 10 Jan 2025 08:22:36 +0000 Subject: [PATCH 47/51] Mention check for spec version when using profile fields Co-authored-by: Richard van der Hoff <1389908+richvdh@users.noreply.github.com> --- proposals/4133-extended-profiles.md | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/proposals/4133-extended-profiles.md b/proposals/4133-extended-profiles.md index 099212c6fab..8570638c926 100644 --- a/proposals/4133-extended-profiles.md +++ b/proposals/4133-extended-profiles.md @@ -141,7 +141,9 @@ Clients MAY check for this capability before attempting to create or modify a pr ### Behaviour - **When capability is missing**: Clients SHOULD assume extended profiles are supported and that - they can be created or modified. If a server intends to deny some (or all) changes, it SHOULD use + they can be created or modified, provided the response from [`/versions`](https://spec.matrix.org/v1.13/client-server-api/#get_matrixclientversions) + indicates support for a spec version that includes this proposal. + If a server intends to deny some (or all) changes, it SHOULD use the capability to advertise this, improving the client experience. - **When `enabled` is `false`**: Clients SHOULD expect to display profiles but NOT create or update From 9b2918e3735f5aec02f6309fcbc81feaca985804 Mon Sep 17 00:00:00 2001 From: Tom Foster Date: Fri, 10 Jan 2025 08:41:02 +0000 Subject: [PATCH 48/51] Attempt to clarify what servers should not enforce about key naming --- proposals/4133-extended-profiles.md | 16 +++++++--------- 1 file changed, 7 insertions(+), 9 deletions(-) diff --git a/proposals/4133-extended-profiles.md b/proposals/4133-extended-profiles.md index 8570638c926..df532510198 100644 --- a/proposals/4133-extended-profiles.md +++ b/proposals/4133-extended-profiles.md @@ -162,18 +162,16 @@ Profiles MUST be at most 64 KiB (65,536 bytes) in size, as measured in [Canonical JSON](https://spec.matrix.org/v1.13/appendices/#canonical-json), including the `avatar_url` and `displayname` fields. -Homeservers SHOULD NOT enforce namespaces, as future expansions may be unknown to the server, but -clients SHOULD use the correct namespace when creating or updating fields. +Keys MUST follow the [Common Namespaced Identifier Grammar](https://spec.matrix.org/v1.13/appendices/#common-namespaced-identifier-grammar), with the following considerations: -Keys MUST follow the [Common Namespaced Identifier Grammar](https://spec.matrix.org/v1.13/appendices/#common-namespaced-identifier-grammar), -with the following considerations: - -- **Namespace `m.*`**: Reserved for fields explicitly defined in the Matrix specification. Clients - that do not recognise a field in this namespace MAY attempt to display it but SHOULD NOT attempt - to update the content unless they understand its formatting and validation requirements. +- **Namespace `m.*`**: Reserved for fields explicitly defined in the Matrix specification: + - Servers SHOULD NOT check whether a key is known to be in the Matrix specification, as future expansions + may be unknown to it. + - Clients that do not recognise a field in this namespace MAY attempt to display it but SHOULD NOT + attempt to update the content unless they understand its formatting and validation requirements. - **Namespace `tld.name.*`**: For client-specific or unstable fields, using Java package naming - convention (e.g., `com.example.custom_field`). + convention (e.g. `com.example.custom_field`). Following this change, clients could use `m.example_field` if that field is defined by the Matrix specification, or `org.example.job_title` for organisation, client-specific fields, or MSC-backed From e00d2e9a80ac572eda264902a27e2a0bc268c2b1 Mon Sep 17 00:00:00 2001 From: Tom Foster Date: Fri, 10 Jan 2025 08:46:43 +0000 Subject: [PATCH 49/51] Fix line wrapping after 9b2918e3735f5aec02f6309fcbc81feaca985804 --- proposals/4133-extended-profiles.md | 12 +++++++----- 1 file changed, 7 insertions(+), 5 deletions(-) diff --git a/proposals/4133-extended-profiles.md b/proposals/4133-extended-profiles.md index df532510198..8a4794fcebd 100644 --- a/proposals/4133-extended-profiles.md +++ b/proposals/4133-extended-profiles.md @@ -162,13 +162,15 @@ Profiles MUST be at most 64 KiB (65,536 bytes) in size, as measured in [Canonical JSON](https://spec.matrix.org/v1.13/appendices/#canonical-json), including the `avatar_url` and `displayname` fields. -Keys MUST follow the [Common Namespaced Identifier Grammar](https://spec.matrix.org/v1.13/appendices/#common-namespaced-identifier-grammar), with the following considerations: +Keys MUST follow the [Common Namespaced Identifier Grammar](https://spec.matrix.org/v1.13/appendices/#common-namespaced-identifier-grammar), +with the following considerations: - **Namespace `m.*`**: Reserved for fields explicitly defined in the Matrix specification: - - Servers SHOULD NOT check whether a key is known to be in the Matrix specification, as future expansions - may be unknown to it. - - Clients that do not recognise a field in this namespace MAY attempt to display it but SHOULD NOT - attempt to update the content unless they understand its formatting and validation requirements. + - Servers SHOULD NOT check whether a key is known to be in the Matrix specification, as future + expansions may be unknown to it. + - Clients that do not recognise a field in this namespace MAY attempt to display it but + SHOULD NOT attempt to update the content unless they understand its formatting and validation + requirements. - **Namespace `tld.name.*`**: For client-specific or unstable fields, using Java package naming convention (e.g. `com.example.custom_field`). From a11286a5091c0e8ac279265f8295ad819c336379 Mon Sep 17 00:00:00 2001 From: Tom Foster Date: Mon, 13 Jan 2025 19:43:21 +0000 Subject: [PATCH 50/51] Add `M_MISSING_PARAM` error --- proposals/4133-extended-profiles.md | 9 +++++++++ 1 file changed, 9 insertions(+) diff --git a/proposals/4133-extended-profiles.md b/proposals/4133-extended-profiles.md index 8a4794fcebd..3228bfb19cf 100644 --- a/proposals/4133-extended-profiles.md +++ b/proposals/4133-extended-profiles.md @@ -193,6 +193,15 @@ demonstrates the process of defining new fields in the `m.*` namespace. } ``` +- **`M_MISSING_PARAM`**: Required parameter is missing, e.g. if a client attempts to set a profile field, but neglects to include that named field in the request body. + +```json +{ + "errcode": "M_MISSING_PARAM", + "error": "A required parameter is missing: {parameter_name}." +} +``` + - **`M_PROFILE_TOO_LARGE`**: Exceeds total profile size limits. ```json From 2a862350e6847297c420174576d6f87273dc1459 Mon Sep 17 00:00:00 2001 From: Tom Foster Date: Mon, 13 Jan 2025 19:47:00 +0000 Subject: [PATCH 51/51] Clarify where errors apply --- proposals/4133-extended-profiles.md | 20 +++++++++++++++++++- 1 file changed, 19 insertions(+), 1 deletion(-) diff --git a/proposals/4133-extended-profiles.md b/proposals/4133-extended-profiles.md index 3228bfb19cf..263f6e9e22c 100644 --- a/proposals/4133-extended-profiles.md +++ b/proposals/4133-extended-profiles.md @@ -193,7 +193,8 @@ demonstrates the process of defining new fields in the `m.*` namespace. } ``` -- **`M_MISSING_PARAM`**: Required parameter is missing, e.g. if a client attempts to set a profile field, but neglects to include that named field in the request body. +- **`M_MISSING_PARAM`**: Required parameter is missing, e.g. if a client attempts to set a profile + field, but neglects to include that named field in the request body. ```json { @@ -249,6 +250,23 @@ A server may return this error in several scenarios: } ``` +### Applicability of Error Codes + +Unless explicitly stated otherwise, all error codes described in this +section apply to all Client-Server and Server-Server endpoints introduced +by this MSC. For example: + +1. `M_NOT_FOUND` applies to any attempt to retrieve a non-existent profile + field. +2. `M_PROFILE_TOO_LARGE` applies to any attempt to create or update profile + data exceeding the allowed size. + +The Server-Server endpoints introduced in this MSC adhere to the existing +error structure for federation, as the federation access remains read-only +in this proposal. This means no new error codes or status code combinations +are introduced for Server-Server endpoints beyond what is already +documented in the specification. + ## Propagation of Profile Fields The existing fields, `avatar_url` and `displayname`, will continue to trigger state events in each