mirror of https://github.com/zulip/zulip.git
docs: Update `docs/subsystems/` files to use channel.
Updates descriptive text that refer to Zulip channels in the `docs/subsystems` files to use channel instead of stream. Part of the stream to channel rename project.
This commit is contained in:
parent
42efea4e19
commit
76b0025091
|
@ -2,12 +2,12 @@
|
||||||
|
|
||||||
Zulip has a cool analytics system for tracking various useful statistics
|
Zulip has a cool analytics system for tracking various useful statistics
|
||||||
that currently power the `/stats` page, and over time will power other
|
that currently power the `/stats` page, and over time will power other
|
||||||
features, like showing usage statistics for the various streams. It is
|
features, like showing usage statistics for the various channels. It is
|
||||||
designed around the following goals:
|
designed around the following goals:
|
||||||
|
|
||||||
- Minimal impact on scalability and service complexity.
|
- Minimal impact on scalability and service complexity.
|
||||||
- Well-tested so that we can count on the results being correct.
|
- Well-tested so that we can count on the results being correct.
|
||||||
- Efficient to query so that we can display data in-app (e.g. on the streams
|
- Efficient to query so that we can display data in-app (e.g. on the channels
|
||||||
page) with minimum impact on the overall performance of those pages.
|
page) with minimum impact on the overall performance of those pages.
|
||||||
- Storage size smaller than the size of the main Message/UserMessage
|
- Storage size smaller than the size of the main Message/UserMessage
|
||||||
database tables, so that we can store the data in the main PostgreSQL
|
database tables, so that we can store the data in the main PostgreSQL
|
||||||
|
@ -164,7 +164,7 @@ a given realm). The only piece that you can't test here is the "Me"
|
||||||
buttons, which won't have any data. For those, you can instead log in
|
buttons, which won't have any data. For those, you can instead log in
|
||||||
as the `shylock@analytics.ds` in the `analytics` realm and visit
|
as the `shylock@analytics.ds` in the `analytics` realm and visit
|
||||||
`/stats` there (which is only a bit more work). Note that the
|
`/stats` there (which is only a bit more work). Note that the
|
||||||
`analytics` realm is a shell with no streams, so you'll only want to
|
`analytics` realm is a shell with no channels, so you'll only want to
|
||||||
use it for testing the graphs.
|
use it for testing the graphs.
|
||||||
|
|
||||||
If you're adding a new stat/table, you'll want to edit
|
If you're adding a new stat/table, you'll want to edit
|
||||||
|
|
|
@ -47,7 +47,7 @@ work.
|
||||||
As a side note, the policy of using these accessor functions wherever
|
As a side note, the policy of using these accessor functions wherever
|
||||||
possible is a good idea, regardless of caching, because the functions
|
possible is a good idea, regardless of caching, because the functions
|
||||||
also generally take care of details you might not think about
|
also generally take care of details you might not think about
|
||||||
(e.g. case-insensitive matching of stream names or email addresses).
|
(e.g. case-insensitive matching of channel names or email addresses).
|
||||||
It's amazing how slightly tricky logic that's duplicated in several
|
It's amazing how slightly tricky logic that's duplicated in several
|
||||||
places invariably ends up buggy in some of those places, and in
|
places invariably ends up buggy in some of those places, and in
|
||||||
aggregate we call these accessor functions hundreds of times in
|
aggregate we call these accessor functions hundreds of times in
|
||||||
|
@ -252,7 +252,7 @@ multiple servers. We do have a few, however:
|
||||||
|
|
||||||
Zulip makes extensive use of caching of data in the browser and mobile
|
Zulip makes extensive use of caching of data in the browser and mobile
|
||||||
apps; details like which users exist, with metadata like names and
|
apps; details like which users exist, with metadata like names and
|
||||||
avatars, similar details for streams, recent message history, etc.
|
avatars, similar details for channels, recent message history, etc.
|
||||||
|
|
||||||
This data is fetched in the `/register` endpoint (or `page_params`
|
This data is fetched in the `/register` endpoint (or `page_params`
|
||||||
for the web app), and kept correct over time. The key to keeping these
|
for the web app), and kept correct over time. The key to keeping these
|
||||||
|
|
|
@ -15,7 +15,7 @@ to receive updates to the Zulip data. The simplest example is a new
|
||||||
message being sent by one client; other clients must be notified in
|
message being sent by one client; other clients must be notified in
|
||||||
order to display the message. But a complete application like Zulip
|
order to display the message. But a complete application like Zulip
|
||||||
has dozens of different types of data that need to be synced to other
|
has dozens of different types of data that need to be synced to other
|
||||||
clients, whether it be new streams, changes in a user's name or
|
clients, whether it be new channels, changes in a user's name or
|
||||||
avatar, settings changes, etc. In Zulip, we call these updates that
|
avatar, settings changes, etc. In Zulip, we call these updates that
|
||||||
need to be sent to other clients **events**.
|
need to be sent to other clients **events**.
|
||||||
|
|
||||||
|
@ -68,7 +68,7 @@ Usually, this list of users is one of 3 things:
|
||||||
- Everyone in the realm (e.g. for organization-level settings changes,
|
- Everyone in the realm (e.g. for organization-level settings changes,
|
||||||
like new realm emoji).
|
like new realm emoji).
|
||||||
- Everyone who would receive a given message (for messages, emoji
|
- Everyone who would receive a given message (for messages, emoji
|
||||||
reactions, message editing, etc.); i.e. the subscribers to a stream
|
reactions, message editing, etc.); i.e. the subscribers to a channel
|
||||||
or the people on a direct message thread.
|
or the people on a direct message thread.
|
||||||
|
|
||||||
It is the responsibility of the caller of `send_event` to choose the
|
It is the responsibility of the caller of `send_event` to choose the
|
||||||
|
@ -176,7 +176,7 @@ When a client starts up, it usually wants to get 2 things from the
|
||||||
server:
|
server:
|
||||||
|
|
||||||
- The "current state" of various pieces of data, e.g. the current
|
- The "current state" of various pieces of data, e.g. the current
|
||||||
settings, set of users in the organization (for typeahead), stream,
|
settings, set of users in the organization (for typeahead), channel,
|
||||||
messages, etc. (aka the "initial state").
|
messages, etc. (aka the "initial state").
|
||||||
- A subscription to receive updates to those data when they are
|
- A subscription to receive updates to those data when they are
|
||||||
changed by a client (aka an event queue).
|
changed by a client (aka an event queue).
|
||||||
|
|
|
@ -8,13 +8,13 @@ be used to deep-link into the application and allow the browser's
|
||||||
Some examples are:
|
Some examples are:
|
||||||
|
|
||||||
- `/#settings/your-bots`: Bots section of the settings overlay.
|
- `/#settings/your-bots`: Bots section of the settings overlay.
|
||||||
- `/#channels`: Streams overlay, where the user manages streams
|
- `/#channels`: Channels overlay, where the user manages channels
|
||||||
(subscription etc.)
|
(subscription etc.)
|
||||||
- `/#channels/11/announce`: Streams overlay with stream ID 11 (called
|
- `/#channels/11/announce`: Channels overlay with channel ID 11 (called
|
||||||
"announce") selected.
|
"announce") selected.
|
||||||
- `/#narrow/stream/42-android/topic/fun`: Message feed showing stream
|
- `/#narrow/stream/42-android/topic/fun`: Message feed showing channel
|
||||||
"android" and topic "fun". (The `42` represents the id of the
|
"android" and topic "fun". (The `42` represents the id of the
|
||||||
stream.)
|
channel.)
|
||||||
|
|
||||||
The main module in the frontend that manages this all is
|
The main module in the frontend that manages this all is
|
||||||
`web/src/hashchange.js` (plus `hash_util.js` for all the parsing
|
`web/src/hashchange.js` (plus `hash_util.js` for all the parsing
|
||||||
|
@ -23,19 +23,19 @@ the reason that it's thorny is that it needs to support a lot of
|
||||||
different flows:
|
different flows:
|
||||||
|
|
||||||
- The user clicking on an in-app link, which in turn opens an overlay.
|
- The user clicking on an in-app link, which in turn opens an overlay.
|
||||||
For example the streams overlay opens when the user clicks the small
|
For example the channels overlay opens when the user clicks the small
|
||||||
cog symbol on the left sidebar, which is in fact a link to
|
cog symbol on the left sidebar, which is in fact a link to
|
||||||
`/#channels`. This makes it easy to have simple links around the app
|
`/#channels`. This makes it easy to have simple links around the app
|
||||||
without custom click handlers for each one.
|
without custom click handlers for each one.
|
||||||
- The user uses the "back" button in their browser (basically
|
- The user uses the "back" button in their browser (basically
|
||||||
equivalent to the previous one, as a _link_ out of the browser history
|
equivalent to the previous one, as a _link_ out of the browser history
|
||||||
will be visited).
|
will be visited).
|
||||||
- The user clicking some in-app click handler (e.g. "Stream settings"
|
- The user clicking some in-app click handler (e.g. "Channel settings"
|
||||||
for an individual stream), that potentially does
|
for an individual channel), that potentially does
|
||||||
several UI-manipulating things including e.g. loading the streams
|
several UI-manipulating things including e.g. loading the channels
|
||||||
overlay, and needs to update the hash without re-triggering the open
|
overlay, and needs to update the hash without re-triggering the open
|
||||||
animation (etc.).
|
animation (etc.).
|
||||||
- Within an overlay like the streams overlay, the user clicks to
|
- Within an overlay like the channels overlay, the user clicks to
|
||||||
another part of the overlay, which should update the hash but not
|
another part of the overlay, which should update the hash but not
|
||||||
re-trigger loading the overlay (which would result in a confusing
|
re-trigger loading the overlay (which would result in a confusing
|
||||||
animation experience).
|
animation experience).
|
||||||
|
@ -69,7 +69,7 @@ Internally you have these functions:
|
||||||
a hash or using the back button) or triggered internally.
|
a hash or using the back button) or triggered internally.
|
||||||
- `hashchange.do_hashchange_normal` handles most cases, like loading the main
|
- `hashchange.do_hashchange_normal` handles most cases, like loading the main
|
||||||
page (but maybe with a specific URL if you are narrowed to a
|
page (but maybe with a specific URL if you are narrowed to a
|
||||||
stream or topic or direct messages, etc.).
|
channel or topic or direct messages, etc.).
|
||||||
- `hashchange.do_hashchange_overlay` handles overlay cases. Overlays have
|
- `hashchange.do_hashchange_overlay` handles overlay cases. Overlays have
|
||||||
some minor complexity related to remembering the page from
|
some minor complexity related to remembering the page from
|
||||||
which the overlay was launched, as well as optimizing in-page
|
which the overlay was launched, as well as optimizing in-page
|
||||||
|
|
|
@ -46,7 +46,7 @@ styling, clean up now-unused CSS, etc., to keep things maintainable.
|
||||||
|
|
||||||
Opt to write CSS in CSS files. Avoid using the `style=` attribute in
|
Opt to write CSS in CSS files. Avoid using the `style=` attribute in
|
||||||
HTML except for styles that are set dynamically. For example, we set
|
HTML except for styles that are set dynamically. For example, we set
|
||||||
the colors for specific streams (`{{stream_color}}`) on different
|
the colors for specific channels (`{{stream_color}}`) on different
|
||||||
elements dynamically, in files like `user_stream_list_item.hbs`:
|
elements dynamically, in files like `user_stream_list_item.hbs`:
|
||||||
|
|
||||||
```html
|
```html
|
||||||
|
|
|
@ -139,7 +139,7 @@ that depend on realm-specific or user-specific data. For example, the
|
||||||
realm could have
|
realm could have
|
||||||
[linkifiers](https://zulip.com/help/add-a-custom-linkifier)
|
[linkifiers](https://zulip.com/help/add-a-custom-linkifier)
|
||||||
or [custom emoji](https://zulip.com/help/custom-emoji)
|
or [custom emoji](https://zulip.com/help/custom-emoji)
|
||||||
configured, and Zulip supports mentions for streams, users, and user
|
configured, and Zulip supports mentions for channels, users, and user
|
||||||
groups (which depend on data like users' names, IDs, etc.).
|
groups (which depend on data like users' names, IDs, etc.).
|
||||||
|
|
||||||
At a backend code level, these are controlled by the `message_realm`
|
At a backend code level, these are controlled by the `message_realm`
|
||||||
|
@ -151,7 +151,7 @@ processor object via e.g. `_md_engine.zulip_db_data`, and then
|
||||||
individual Markdown rules can access the data from there.
|
individual Markdown rules can access the data from there.
|
||||||
|
|
||||||
For non-message contexts (e.g. an organization's profile (aka the
|
For non-message contexts (e.g. an organization's profile (aka the
|
||||||
thing on the right-hand side of the login page), stream descriptions,
|
thing on the right-hand side of the login page), channel descriptions,
|
||||||
or rendering custom profile fields), one needs to just pass in a
|
or rendering custom profile fields), one needs to just pass in a
|
||||||
`message_realm` (see, for example, `zulip_default_context` for the
|
`message_realm` (see, for example, `zulip_default_context` for the
|
||||||
organization profile code for this). But for messages, we need to
|
organization profile code for this). But for messages, we need to
|
||||||
|
@ -253,7 +253,7 @@ accurate.
|
||||||
- Disable link-by-reference syntax,
|
- Disable link-by-reference syntax,
|
||||||
`[foo][bar]` ... `[bar]: https://google.com`.
|
`[foo][bar]` ... `[bar]: https://google.com`.
|
||||||
|
|
||||||
- Enable linking to other streams using `#**streamName**`.
|
- Enable linking to other channels using `#**channelName**`.
|
||||||
|
|
||||||
### Code
|
### Code
|
||||||
|
|
||||||
|
|
|
@ -42,7 +42,7 @@ as follows:
|
||||||
`enable_online_push_notifications` flag is enabled). This data
|
`enable_online_push_notifications` flag is enabled). This data
|
||||||
structure ignores users for whom the message is not notifiable,
|
structure ignores users for whom the message is not notifiable,
|
||||||
which is important to avoid this being thousands of `user_ids` for
|
which is important to avoid this being thousands of `user_ids` for
|
||||||
messages to large streams with few currently active users.
|
messages to large channels with few currently active users.
|
||||||
- The Tornado [event queue system](events-system.md)
|
- The Tornado [event queue system](events-system.md)
|
||||||
processes that data, as well as data about each user's active event
|
processes that data, as well as data about each user's active event
|
||||||
queues, to (1) push an event to each queue needing that message and
|
queues, to (1) push an event to each queue needing that message and
|
||||||
|
@ -153,7 +153,7 @@ structure of the system, when thinking about changes to it:
|
||||||
- **Future configuration**. Notification settings are an area that we
|
- **Future configuration**. Notification settings are an area that we
|
||||||
expect to only expand with time, with upcoming features like
|
expect to only expand with time, with upcoming features like
|
||||||
following a topic (to get notifications for messages only within
|
following a topic (to get notifications for messages only within
|
||||||
that topic in a stream). There are a lot of different workflows
|
that topic in a channel). There are a lot of different workflows
|
||||||
possible with Zulip's threading, and it's important to make it easy
|
possible with Zulip's threading, and it's important to make it easy
|
||||||
for users to set up Zulip's notification to fit as many of those
|
for users to set up Zulip's notification to fit as many of those
|
||||||
workflows as possible.
|
workflows as possible.
|
||||||
|
|
|
@ -179,7 +179,7 @@ Zulip is somewhat unusual among web apps in sending essentially all of the
|
||||||
data required for the entire Zulip web app in this single request,
|
data required for the entire Zulip web app in this single request,
|
||||||
which is part of why the Zulip web app loads very quickly -- one only
|
which is part of why the Zulip web app loads very quickly -- one only
|
||||||
needs a single round trip aside from cacheable assets (avatars, images, JS,
|
needs a single round trip aside from cacheable assets (avatars, images, JS,
|
||||||
CSS). Data on other users in the organization, streams, supported
|
CSS). Data on other users in the organization, channels, supported
|
||||||
emoji, custom profile fields, etc., is all included. The nice thing
|
emoji, custom profile fields, etc., is all included. The nice thing
|
||||||
about this model is that essentially every UI element in the Zulip
|
about this model is that essentially every UI element in the Zulip
|
||||||
client can be rendered immediately without paying latency to the
|
client can be rendered immediately without paying latency to the
|
||||||
|
@ -211,21 +211,21 @@ typical organization but potentially multiple seconds for large open
|
||||||
organizations with 10,000s of users. There is also smaller
|
organizations with 10,000s of users. There is also smaller
|
||||||
variability based on a individual user's personal data state,
|
variability based on a individual user's personal data state,
|
||||||
primarily in that having 10,000s of unread messages results in a
|
primarily in that having 10,000s of unread messages results in a
|
||||||
somewhat expensive query to find which streams/topics those are in.
|
somewhat expensive query to find which channels/topics those are in.
|
||||||
|
|
||||||
We consider any organization having normal `page_params` fetch times
|
We consider any organization having normal `page_params` fetch times
|
||||||
greater than a second to be a bug, and there is ongoing work to fix that.
|
greater than a second to be a bug, and there is ongoing work to fix that.
|
||||||
|
|
||||||
It can help when thinking about this to imagine `page_params` as what
|
It can help when thinking about this to imagine `page_params` as what
|
||||||
in another web app would have been 25 or so HTTP GET requests, each
|
in another web app would have been 25 or so HTTP GET requests, each
|
||||||
fetching data of a given type (users, streams, custom emoji, etc.); in
|
fetching data of a given type (users, channels, custom emoji, etc.); in
|
||||||
Zulip, we just do all of those in a single API request. In the
|
Zulip, we just do all of those in a single API request. In the
|
||||||
future, we will likely move to a design that does much of the database
|
future, we will likely move to a design that does much of the database
|
||||||
fetching work for different features in parallel to improve latency.
|
fetching work for different features in parallel to improve latency.
|
||||||
|
|
||||||
For organizations with 10K+ users and many default streams, the
|
For organizations with 10K+ users and many default channels, the
|
||||||
majority of time spent constructing `page_params` is spent marshalling
|
majority of time spent constructing `page_params` is spent marshalling
|
||||||
data on which users are subscribed to which streams, which is an area
|
data on which users are subscribed to which channels, which is an area
|
||||||
of active optimization work.
|
of active optimization work.
|
||||||
|
|
||||||
### Fetching message history
|
### Fetching message history
|
||||||
|
@ -238,7 +238,7 @@ it does a large number of these requests:
|
||||||
- Most of these requests are from users clicking into different views
|
- Most of these requests are from users clicking into different views
|
||||||
-- to avoid certain subtle bugs, Zulip's web app currently fetches
|
-- to avoid certain subtle bugs, Zulip's web app currently fetches
|
||||||
content from the server even when it has the history for the
|
content from the server even when it has the history for the
|
||||||
relevant stream/topic cached locally.
|
relevant channel/topic cached locally.
|
||||||
- When a browser opens the Zulip web app, it will eventually fetch and
|
- When a browser opens the Zulip web app, it will eventually fetch and
|
||||||
cache in the browser all messages newer than the oldest unread
|
cache in the browser all messages newer than the oldest unread
|
||||||
message in a non-muted context. This can be in total extremely
|
message in a non-muted context. This can be in total extremely
|
||||||
|
@ -314,7 +314,7 @@ but are also extremely cheap (~3ms).
|
||||||
|
|
||||||
### Other endpoints
|
### Other endpoints
|
||||||
|
|
||||||
Other API actions, like subscribing to a stream, editing settings,
|
Other API actions, like subscribing to a channel, editing settings,
|
||||||
registering an account, etc., are vanishingly rare compared to the
|
registering an account, etc., are vanishingly rare compared to the
|
||||||
requests detailed above, fundamentally because almost nobody changes
|
requests detailed above, fundamentally because almost nobody changes
|
||||||
these things more than a few dozen times over the lifetime of their
|
these things more than a few dozen times over the lifetime of their
|
||||||
|
|
|
@ -1,6 +1,6 @@
|
||||||
# Unread counts and the pointer
|
# Unread counts and the pointer
|
||||||
|
|
||||||
When you're using Zulip and you reload, or narrow to a stream, how
|
When you're using Zulip and you reload, or narrow to a channel, how
|
||||||
does Zulip decide where to place you?
|
does Zulip decide where to place you?
|
||||||
|
|
||||||
Conceptually, Zulip takes you to the place where you left off
|
Conceptually, Zulip takes you to the place where you left off
|
||||||
|
@ -29,7 +29,7 @@ First a bit of terminology:
|
||||||
### Recipient bar: message you clicked
|
### Recipient bar: message you clicked
|
||||||
|
|
||||||
If you enter a narrow by clicking on a message group's _recipient bar_
|
If you enter a narrow by clicking on a message group's _recipient bar_
|
||||||
(stream/topic or direct message recipient list at the top of a group
|
(channel/topic or direct message recipient list at the top of a group
|
||||||
of messages), Zulip will select the message you clicked on. This
|
of messages), Zulip will select the message you clicked on. This
|
||||||
provides a nice user experience where you get to see the stuff near
|
provides a nice user experience where you get to see the stuff near
|
||||||
what you clicked on, and in fact the message you clicked on stays at
|
what you clicked on, and in fact the message you clicked on stays at
|
||||||
|
@ -48,8 +48,8 @@ This provides the nice user experience of taking you to the start of
|
||||||
the new stuff (with enough messages you've seen before still in view
|
the new stuff (with enough messages you've seen before still in view
|
||||||
at the top to provide you with context), which is usually what you
|
at the top to provide you with context), which is usually what you
|
||||||
want. (When finding the "first unread message", Zulip ignores unread
|
want. (When finding the "first unread message", Zulip ignores unread
|
||||||
messages in muted streams or in muted topics within non-muted
|
messages in muted channels or in muted topics within non-muted
|
||||||
streams.)
|
channels.)
|
||||||
|
|
||||||
### Unnarrow: previous sequence
|
### Unnarrow: previous sequence
|
||||||
|
|
||||||
|
|
|
@ -37,10 +37,10 @@ and narrowing).
|
||||||
The compose box does a lot of fancy things that are out of scope for
|
The compose box does a lot of fancy things that are out of scope for
|
||||||
this article. But it also does a decent amount of client-side
|
this article. But it also does a decent amount of client-side
|
||||||
validation before sending a message off to the server, especially
|
validation before sending a message off to the server, especially
|
||||||
around mentions (E.g. checking the stream name is a valid stream,
|
around mentions (E.g. checking the channel name is a valid channel,
|
||||||
displaying a warning about the number of recipients before a user can
|
displaying a warning about the number of recipients before a user can
|
||||||
use `@**all**` or mention a user who is not subscribed to the current
|
use `@**all**` or mention a user who is not subscribed to the current
|
||||||
stream, etc.).
|
channel, etc.).
|
||||||
|
|
||||||
## Backend implementation
|
## Backend implementation
|
||||||
|
|
||||||
|
@ -257,7 +257,7 @@ For background, Zulip’s threading model requires tracking which
|
||||||
individual messages each user has received and read (in other chat
|
individual messages each user has received and read (in other chat
|
||||||
products, the system either doesn’t track what the user has read at
|
products, the system either doesn’t track what the user has read at
|
||||||
all, or just needs to store a pointer for “how far the user has read”
|
all, or just needs to store a pointer for “how far the user has read”
|
||||||
in each room, channel, or stream).
|
in each room or channel).
|
||||||
|
|
||||||
We track these data in the backend in the `UserMessage` table, storing
|
We track these data in the backend in the `UserMessage` table, storing
|
||||||
rows `(message_id, user_id, flags)`, where `flags` is 32 bits of space
|
rows `(message_id, user_id, flags)`, where `flags` is 32 bits of space
|
||||||
|
@ -268,7 +268,7 @@ the database indexes on this table (with joins to the `Message` table
|
||||||
containing the actual message content where required).
|
containing the actual message content where required).
|
||||||
|
|
||||||
The downside of this design is that when a new message is sent to a
|
The downside of this design is that when a new message is sent to a
|
||||||
stream with `N` recipients, we need to write `N` rows to the
|
channel with `N` recipients, we need to write `N` rows to the
|
||||||
`UserMessage` table to record those users receiving those messages.
|
`UserMessage` table to record those users receiving those messages.
|
||||||
Each row is just 3 integers in size, but even with modern databases
|
Each row is just 3 integers in size, but even with modern databases
|
||||||
and SSDs, writing thousands of rows to a database starts to take a few
|
and SSDs, writing thousands of rows to a database starts to take a few
|
||||||
|
@ -278,10 +278,10 @@ This isn’t a problem for most Zulip servers, but is a major problem
|
||||||
for communities like chat.zulip.org, where there might be 10,000s of
|
for communities like chat.zulip.org, where there might be 10,000s of
|
||||||
inactive users who only stopped by briefly to check out the product or
|
inactive users who only stopped by briefly to check out the product or
|
||||||
ask a single question, but are subscribed to whatever the default
|
ask a single question, but are subscribed to whatever the default
|
||||||
streams in the organization are.
|
channels in the organization are.
|
||||||
|
|
||||||
The total amount of work being done here was acceptable (a few seconds
|
The total amount of work being done here was acceptable (a few seconds
|
||||||
of total CPU work per message to large public streams), but the
|
of total CPU work per message to large public channels), but the
|
||||||
latency was unacceptable: The server backend was introducing a latency
|
latency was unacceptable: The server backend was introducing a latency
|
||||||
of about 1 second per 2000 users subscribed to receive the message.
|
of about 1 second per 2000 users subscribed to receive the message.
|
||||||
While these delays may not be immediately obvious to users (Zulip,
|
While these delays may not be immediately obvious to users (Zulip,
|
||||||
|
@ -294,18 +294,18 @@ even simple questions).
|
||||||
|
|
||||||
A key insight for addressing this problem is that there isn’t much of
|
A key insight for addressing this problem is that there isn’t much of
|
||||||
a use case for long chat discussions among 1000s of users who are all
|
a use case for long chat discussions among 1000s of users who are all
|
||||||
continuously online and actively participating. Streams with a very
|
continuously online and actively participating. Channels with a very
|
||||||
large number of active users are likely to only be used for occasional
|
large number of active users are likely to only be used for occasional
|
||||||
announcements, where some latency before everyone sees the message is
|
announcements, where some latency before everyone sees the message is
|
||||||
fine. Even in giant organizations, almost all messages are sent to
|
fine. Even in giant organizations, almost all messages are sent to
|
||||||
smaller streams with dozens or hundreds of active users, representing
|
smaller channels with dozens or hundreds of active users, representing
|
||||||
some organizational unit within the community or company.
|
some organizational unit within the community or company.
|
||||||
|
|
||||||
However, large, active streams are common in open source projects,
|
However, large, active channels are common in open source projects,
|
||||||
standards bodies, professional development groups, and other large
|
standards bodies, professional development groups, and other large
|
||||||
communities with the rough structure of the Zulip development
|
communities with the rough structure of the Zulip development
|
||||||
community. These communities usually have thousands of user accounts
|
community. These communities usually have thousands of user accounts
|
||||||
subscribed to all the default streams, even if they only have dozens
|
subscribed to all the default channels, even if they only have dozens
|
||||||
or hundreds of those users active in any given month. Many of the
|
or hundreds of those users active in any given month. Many of the
|
||||||
other accounts may be from people who signed up just to check the
|
other accounts may be from people who signed up just to check the
|
||||||
community out, or who signed up to ask a few questions and may never
|
community out, or who signed up to ask a few questions and may never
|
||||||
|
@ -330,14 +330,14 @@ organization for a few weeks, they are tagged as soft-deactivated.
|
||||||
The way this works internally is:
|
The way this works internally is:
|
||||||
|
|
||||||
- We (usually) skip creating UserMessage rows for soft-deactivated
|
- We (usually) skip creating UserMessage rows for soft-deactivated
|
||||||
users when a message is sent to a stream where they are subscribed.
|
users when a message is sent to a channel where they are subscribed.
|
||||||
|
|
||||||
- If/when the user ever returns to Zulip, we can at that time
|
- If/when the user ever returns to Zulip, we can at that time
|
||||||
reconstruct the UserMessage rows that they missed, and create the rows
|
reconstruct the UserMessage rows that they missed, and create the rows
|
||||||
at that time (or, to avoid a latency spike if/when the user returns to
|
at that time (or, to avoid a latency spike if/when the user returns to
|
||||||
Zulip, this work can be done in a nightly cron job). We can construct
|
Zulip, this work can be done in a nightly cron job). We can construct
|
||||||
those rows later because we already have the data for when the user
|
those rows later because we already have the data for when the user
|
||||||
might have been subscribed or unsubscribed from streams by other
|
might have been subscribed or unsubscribed from channels by other
|
||||||
users, and, importantly, we also know that the user didn’t interact
|
users, and, importantly, we also know that the user didn’t interact
|
||||||
with the UI since the message was sent (and thus we can safely assume
|
with the UI since the message was sent (and thus we can safely assume
|
||||||
that the messages have not been marked as read by the user). This is
|
that the messages have not been marked as read by the user). This is
|
||||||
|
@ -369,9 +369,9 @@ The end result is the best of both worlds:
|
||||||
with Zulip.
|
with Zulip.
|
||||||
|
|
||||||
Empirically, we've found this technique completely resolved the "send
|
Empirically, we've found this technique completely resolved the "send
|
||||||
latency" scaling problem. The latency of sending a message to a stream
|
latency" scaling problem. The latency of sending a message to a channel
|
||||||
now scales only with the number of active subscribers, so one can send
|
now scales only with the number of active subscribers, so one can send
|
||||||
a message to a stream with 5K subscribers of which 500 are active, and
|
a message to a channel with 5K subscribers of which 500 are active, and
|
||||||
it’ll arrive in the couple hundred milliseconds one would expect if
|
it’ll arrive in the couple hundred milliseconds one would expect if
|
||||||
the extra 4500 inactive subscribers didn’t exist.
|
the extra 4500 inactive subscribers didn’t exist.
|
||||||
|
|
||||||
|
@ -384,7 +384,7 @@ There are a few details that require special care with this system:
|
||||||
assumed a `UserMessage` row would always exist for a message that
|
assumed a `UserMessage` row would always exist for a message that
|
||||||
triggers a notification to a given user.
|
triggers a notification to a given user.
|
||||||
- Digest emails, which use the `UserMessage` table extensively to
|
- Digest emails, which use the `UserMessage` table extensively to
|
||||||
determine what has happened in streams the user can see. We can use
|
determine what has happened in channels the user can see. We can use
|
||||||
the user's subscriptions to construct what messages they should have
|
the user's subscriptions to construct what messages they should have
|
||||||
access to for this feature.
|
access to for this feature.
|
||||||
- Soft-deactivated users experience high loading latency when
|
- Soft-deactivated users experience high loading latency when
|
||||||
|
|
|
@ -170,11 +170,7 @@ not let the notifications become too "sticky" either.
|
||||||
|
|
||||||
## Roadmap
|
## Roadmap
|
||||||
|
|
||||||
The most likely big change to typing indicators is that we will
|
An area for refinement is to tune the timing values a bit.
|
||||||
add them for stream conversations. This will require some thought
|
|
||||||
for large streams, in terms of both usability and performance.
|
|
||||||
|
|
||||||
Another area for refinement is to tune the timing values a bit.
|
|
||||||
Right now, we are possibly too aggressive about sending `stop`
|
Right now, we are possibly too aggressive about sending `stop`
|
||||||
messages when users are just pausing to think. It's possible
|
messages when users are just pausing to think. It's possible
|
||||||
to better account for typing speed or other heuristic things
|
to better account for typing speed or other heuristic things
|
||||||
|
|
|
@ -1,8 +1,8 @@
|
||||||
# Unread message synchronization
|
# Unread message synchronization
|
||||||
|
|
||||||
In general displaying unread counts for all streams and topics may require
|
In general displaying unread counts for all channels and topics may require
|
||||||
downloading an unbounded number of messages. Consider a user who has a muted
|
downloading an unbounded number of messages. Consider a user who has a muted
|
||||||
stream or topic and has not read the backlog in a month; to have an accurate
|
channel or topic and has not read the backlog in a month; to have an accurate
|
||||||
unread count we would need to load all messages this user has received in the
|
unread count we would need to load all messages this user has received in the
|
||||||
past month. This is inefficient for web clients and even more for mobile
|
past month. This is inefficient for web clients and even more for mobile
|
||||||
devices.
|
devices.
|
||||||
|
|
Loading…
Reference in New Issue