mirror of https://github.com/zulip/zulip.git
docs: Update `docs/subsystems/` files to use channel.
Updates descriptive text that refer to Zulip channels in the `docs/subsystems` files to use channel instead of stream. Part of the stream to channel rename project.
This commit is contained in:
parent
42efea4e19
commit
76b0025091
|
@ -2,12 +2,12 @@
|
|||
|
||||
Zulip has a cool analytics system for tracking various useful statistics
|
||||
that currently power the `/stats` page, and over time will power other
|
||||
features, like showing usage statistics for the various streams. It is
|
||||
features, like showing usage statistics for the various channels. It is
|
||||
designed around the following goals:
|
||||
|
||||
- Minimal impact on scalability and service complexity.
|
||||
- Well-tested so that we can count on the results being correct.
|
||||
- Efficient to query so that we can display data in-app (e.g. on the streams
|
||||
- Efficient to query so that we can display data in-app (e.g. on the channels
|
||||
page) with minimum impact on the overall performance of those pages.
|
||||
- Storage size smaller than the size of the main Message/UserMessage
|
||||
database tables, so that we can store the data in the main PostgreSQL
|
||||
|
@ -164,7 +164,7 @@ a given realm). The only piece that you can't test here is the "Me"
|
|||
buttons, which won't have any data. For those, you can instead log in
|
||||
as the `shylock@analytics.ds` in the `analytics` realm and visit
|
||||
`/stats` there (which is only a bit more work). Note that the
|
||||
`analytics` realm is a shell with no streams, so you'll only want to
|
||||
`analytics` realm is a shell with no channels, so you'll only want to
|
||||
use it for testing the graphs.
|
||||
|
||||
If you're adding a new stat/table, you'll want to edit
|
||||
|
|
|
@ -47,7 +47,7 @@ work.
|
|||
As a side note, the policy of using these accessor functions wherever
|
||||
possible is a good idea, regardless of caching, because the functions
|
||||
also generally take care of details you might not think about
|
||||
(e.g. case-insensitive matching of stream names or email addresses).
|
||||
(e.g. case-insensitive matching of channel names or email addresses).
|
||||
It's amazing how slightly tricky logic that's duplicated in several
|
||||
places invariably ends up buggy in some of those places, and in
|
||||
aggregate we call these accessor functions hundreds of times in
|
||||
|
@ -252,7 +252,7 @@ multiple servers. We do have a few, however:
|
|||
|
||||
Zulip makes extensive use of caching of data in the browser and mobile
|
||||
apps; details like which users exist, with metadata like names and
|
||||
avatars, similar details for streams, recent message history, etc.
|
||||
avatars, similar details for channels, recent message history, etc.
|
||||
|
||||
This data is fetched in the `/register` endpoint (or `page_params`
|
||||
for the web app), and kept correct over time. The key to keeping these
|
||||
|
|
|
@ -15,7 +15,7 @@ to receive updates to the Zulip data. The simplest example is a new
|
|||
message being sent by one client; other clients must be notified in
|
||||
order to display the message. But a complete application like Zulip
|
||||
has dozens of different types of data that need to be synced to other
|
||||
clients, whether it be new streams, changes in a user's name or
|
||||
clients, whether it be new channels, changes in a user's name or
|
||||
avatar, settings changes, etc. In Zulip, we call these updates that
|
||||
need to be sent to other clients **events**.
|
||||
|
||||
|
@ -68,7 +68,7 @@ Usually, this list of users is one of 3 things:
|
|||
- Everyone in the realm (e.g. for organization-level settings changes,
|
||||
like new realm emoji).
|
||||
- Everyone who would receive a given message (for messages, emoji
|
||||
reactions, message editing, etc.); i.e. the subscribers to a stream
|
||||
reactions, message editing, etc.); i.e. the subscribers to a channel
|
||||
or the people on a direct message thread.
|
||||
|
||||
It is the responsibility of the caller of `send_event` to choose the
|
||||
|
@ -176,7 +176,7 @@ When a client starts up, it usually wants to get 2 things from the
|
|||
server:
|
||||
|
||||
- The "current state" of various pieces of data, e.g. the current
|
||||
settings, set of users in the organization (for typeahead), stream,
|
||||
settings, set of users in the organization (for typeahead), channel,
|
||||
messages, etc. (aka the "initial state").
|
||||
- A subscription to receive updates to those data when they are
|
||||
changed by a client (aka an event queue).
|
||||
|
|
|
@ -8,13 +8,13 @@ be used to deep-link into the application and allow the browser's
|
|||
Some examples are:
|
||||
|
||||
- `/#settings/your-bots`: Bots section of the settings overlay.
|
||||
- `/#channels`: Streams overlay, where the user manages streams
|
||||
- `/#channels`: Channels overlay, where the user manages channels
|
||||
(subscription etc.)
|
||||
- `/#channels/11/announce`: Streams overlay with stream ID 11 (called
|
||||
- `/#channels/11/announce`: Channels overlay with channel ID 11 (called
|
||||
"announce") selected.
|
||||
- `/#narrow/stream/42-android/topic/fun`: Message feed showing stream
|
||||
- `/#narrow/stream/42-android/topic/fun`: Message feed showing channel
|
||||
"android" and topic "fun". (The `42` represents the id of the
|
||||
stream.)
|
||||
channel.)
|
||||
|
||||
The main module in the frontend that manages this all is
|
||||
`web/src/hashchange.js` (plus `hash_util.js` for all the parsing
|
||||
|
@ -23,19 +23,19 @@ the reason that it's thorny is that it needs to support a lot of
|
|||
different flows:
|
||||
|
||||
- The user clicking on an in-app link, which in turn opens an overlay.
|
||||
For example the streams overlay opens when the user clicks the small
|
||||
For example the channels overlay opens when the user clicks the small
|
||||
cog symbol on the left sidebar, which is in fact a link to
|
||||
`/#channels`. This makes it easy to have simple links around the app
|
||||
without custom click handlers for each one.
|
||||
- The user uses the "back" button in their browser (basically
|
||||
equivalent to the previous one, as a _link_ out of the browser history
|
||||
will be visited).
|
||||
- The user clicking some in-app click handler (e.g. "Stream settings"
|
||||
for an individual stream), that potentially does
|
||||
several UI-manipulating things including e.g. loading the streams
|
||||
- The user clicking some in-app click handler (e.g. "Channel settings"
|
||||
for an individual channel), that potentially does
|
||||
several UI-manipulating things including e.g. loading the channels
|
||||
overlay, and needs to update the hash without re-triggering the open
|
||||
animation (etc.).
|
||||
- Within an overlay like the streams overlay, the user clicks to
|
||||
- Within an overlay like the channels overlay, the user clicks to
|
||||
another part of the overlay, which should update the hash but not
|
||||
re-trigger loading the overlay (which would result in a confusing
|
||||
animation experience).
|
||||
|
@ -69,7 +69,7 @@ Internally you have these functions:
|
|||
a hash or using the back button) or triggered internally.
|
||||
- `hashchange.do_hashchange_normal` handles most cases, like loading the main
|
||||
page (but maybe with a specific URL if you are narrowed to a
|
||||
stream or topic or direct messages, etc.).
|
||||
channel or topic or direct messages, etc.).
|
||||
- `hashchange.do_hashchange_overlay` handles overlay cases. Overlays have
|
||||
some minor complexity related to remembering the page from
|
||||
which the overlay was launched, as well as optimizing in-page
|
||||
|
|
|
@ -46,7 +46,7 @@ styling, clean up now-unused CSS, etc., to keep things maintainable.
|
|||
|
||||
Opt to write CSS in CSS files. Avoid using the `style=` attribute in
|
||||
HTML except for styles that are set dynamically. For example, we set
|
||||
the colors for specific streams (`{{stream_color}}`) on different
|
||||
the colors for specific channels (`{{stream_color}}`) on different
|
||||
elements dynamically, in files like `user_stream_list_item.hbs`:
|
||||
|
||||
```html
|
||||
|
|
|
@ -139,7 +139,7 @@ that depend on realm-specific or user-specific data. For example, the
|
|||
realm could have
|
||||
[linkifiers](https://zulip.com/help/add-a-custom-linkifier)
|
||||
or [custom emoji](https://zulip.com/help/custom-emoji)
|
||||
configured, and Zulip supports mentions for streams, users, and user
|
||||
configured, and Zulip supports mentions for channels, users, and user
|
||||
groups (which depend on data like users' names, IDs, etc.).
|
||||
|
||||
At a backend code level, these are controlled by the `message_realm`
|
||||
|
@ -151,7 +151,7 @@ processor object via e.g. `_md_engine.zulip_db_data`, and then
|
|||
individual Markdown rules can access the data from there.
|
||||
|
||||
For non-message contexts (e.g. an organization's profile (aka the
|
||||
thing on the right-hand side of the login page), stream descriptions,
|
||||
thing on the right-hand side of the login page), channel descriptions,
|
||||
or rendering custom profile fields), one needs to just pass in a
|
||||
`message_realm` (see, for example, `zulip_default_context` for the
|
||||
organization profile code for this). But for messages, we need to
|
||||
|
@ -253,7 +253,7 @@ accurate.
|
|||
- Disable link-by-reference syntax,
|
||||
`[foo][bar]` ... `[bar]: https://google.com`.
|
||||
|
||||
- Enable linking to other streams using `#**streamName**`.
|
||||
- Enable linking to other channels using `#**channelName**`.
|
||||
|
||||
### Code
|
||||
|
||||
|
|
|
@ -42,7 +42,7 @@ as follows:
|
|||
`enable_online_push_notifications` flag is enabled). This data
|
||||
structure ignores users for whom the message is not notifiable,
|
||||
which is important to avoid this being thousands of `user_ids` for
|
||||
messages to large streams with few currently active users.
|
||||
messages to large channels with few currently active users.
|
||||
- The Tornado [event queue system](events-system.md)
|
||||
processes that data, as well as data about each user's active event
|
||||
queues, to (1) push an event to each queue needing that message and
|
||||
|
@ -153,7 +153,7 @@ structure of the system, when thinking about changes to it:
|
|||
- **Future configuration**. Notification settings are an area that we
|
||||
expect to only expand with time, with upcoming features like
|
||||
following a topic (to get notifications for messages only within
|
||||
that topic in a stream). There are a lot of different workflows
|
||||
that topic in a channel). There are a lot of different workflows
|
||||
possible with Zulip's threading, and it's important to make it easy
|
||||
for users to set up Zulip's notification to fit as many of those
|
||||
workflows as possible.
|
||||
|
|
|
@ -179,7 +179,7 @@ Zulip is somewhat unusual among web apps in sending essentially all of the
|
|||
data required for the entire Zulip web app in this single request,
|
||||
which is part of why the Zulip web app loads very quickly -- one only
|
||||
needs a single round trip aside from cacheable assets (avatars, images, JS,
|
||||
CSS). Data on other users in the organization, streams, supported
|
||||
CSS). Data on other users in the organization, channels, supported
|
||||
emoji, custom profile fields, etc., is all included. The nice thing
|
||||
about this model is that essentially every UI element in the Zulip
|
||||
client can be rendered immediately without paying latency to the
|
||||
|
@ -211,21 +211,21 @@ typical organization but potentially multiple seconds for large open
|
|||
organizations with 10,000s of users. There is also smaller
|
||||
variability based on a individual user's personal data state,
|
||||
primarily in that having 10,000s of unread messages results in a
|
||||
somewhat expensive query to find which streams/topics those are in.
|
||||
somewhat expensive query to find which channels/topics those are in.
|
||||
|
||||
We consider any organization having normal `page_params` fetch times
|
||||
greater than a second to be a bug, and there is ongoing work to fix that.
|
||||
|
||||
It can help when thinking about this to imagine `page_params` as what
|
||||
in another web app would have been 25 or so HTTP GET requests, each
|
||||
fetching data of a given type (users, streams, custom emoji, etc.); in
|
||||
fetching data of a given type (users, channels, custom emoji, etc.); in
|
||||
Zulip, we just do all of those in a single API request. In the
|
||||
future, we will likely move to a design that does much of the database
|
||||
fetching work for different features in parallel to improve latency.
|
||||
|
||||
For organizations with 10K+ users and many default streams, the
|
||||
For organizations with 10K+ users and many default channels, the
|
||||
majority of time spent constructing `page_params` is spent marshalling
|
||||
data on which users are subscribed to which streams, which is an area
|
||||
data on which users are subscribed to which channels, which is an area
|
||||
of active optimization work.
|
||||
|
||||
### Fetching message history
|
||||
|
@ -238,7 +238,7 @@ it does a large number of these requests:
|
|||
- Most of these requests are from users clicking into different views
|
||||
-- to avoid certain subtle bugs, Zulip's web app currently fetches
|
||||
content from the server even when it has the history for the
|
||||
relevant stream/topic cached locally.
|
||||
relevant channel/topic cached locally.
|
||||
- When a browser opens the Zulip web app, it will eventually fetch and
|
||||
cache in the browser all messages newer than the oldest unread
|
||||
message in a non-muted context. This can be in total extremely
|
||||
|
@ -314,7 +314,7 @@ but are also extremely cheap (~3ms).
|
|||
|
||||
### Other endpoints
|
||||
|
||||
Other API actions, like subscribing to a stream, editing settings,
|
||||
Other API actions, like subscribing to a channel, editing settings,
|
||||
registering an account, etc., are vanishingly rare compared to the
|
||||
requests detailed above, fundamentally because almost nobody changes
|
||||
these things more than a few dozen times over the lifetime of their
|
||||
|
|
|
@ -1,6 +1,6 @@
|
|||
# Unread counts and the pointer
|
||||
|
||||
When you're using Zulip and you reload, or narrow to a stream, how
|
||||
When you're using Zulip and you reload, or narrow to a channel, how
|
||||
does Zulip decide where to place you?
|
||||
|
||||
Conceptually, Zulip takes you to the place where you left off
|
||||
|
@ -29,7 +29,7 @@ First a bit of terminology:
|
|||
### Recipient bar: message you clicked
|
||||
|
||||
If you enter a narrow by clicking on a message group's _recipient bar_
|
||||
(stream/topic or direct message recipient list at the top of a group
|
||||
(channel/topic or direct message recipient list at the top of a group
|
||||
of messages), Zulip will select the message you clicked on. This
|
||||
provides a nice user experience where you get to see the stuff near
|
||||
what you clicked on, and in fact the message you clicked on stays at
|
||||
|
@ -48,8 +48,8 @@ This provides the nice user experience of taking you to the start of
|
|||
the new stuff (with enough messages you've seen before still in view
|
||||
at the top to provide you with context), which is usually what you
|
||||
want. (When finding the "first unread message", Zulip ignores unread
|
||||
messages in muted streams or in muted topics within non-muted
|
||||
streams.)
|
||||
messages in muted channels or in muted topics within non-muted
|
||||
channels.)
|
||||
|
||||
### Unnarrow: previous sequence
|
||||
|
||||
|
|
|
@ -37,10 +37,10 @@ and narrowing).
|
|||
The compose box does a lot of fancy things that are out of scope for
|
||||
this article. But it also does a decent amount of client-side
|
||||
validation before sending a message off to the server, especially
|
||||
around mentions (E.g. checking the stream name is a valid stream,
|
||||
around mentions (E.g. checking the channel name is a valid channel,
|
||||
displaying a warning about the number of recipients before a user can
|
||||
use `@**all**` or mention a user who is not subscribed to the current
|
||||
stream, etc.).
|
||||
channel, etc.).
|
||||
|
||||
## Backend implementation
|
||||
|
||||
|
@ -257,7 +257,7 @@ For background, Zulip’s threading model requires tracking which
|
|||
individual messages each user has received and read (in other chat
|
||||
products, the system either doesn’t track what the user has read at
|
||||
all, or just needs to store a pointer for “how far the user has read”
|
||||
in each room, channel, or stream).
|
||||
in each room or channel).
|
||||
|
||||
We track these data in the backend in the `UserMessage` table, storing
|
||||
rows `(message_id, user_id, flags)`, where `flags` is 32 bits of space
|
||||
|
@ -268,7 +268,7 @@ the database indexes on this table (with joins to the `Message` table
|
|||
containing the actual message content where required).
|
||||
|
||||
The downside of this design is that when a new message is sent to a
|
||||
stream with `N` recipients, we need to write `N` rows to the
|
||||
channel with `N` recipients, we need to write `N` rows to the
|
||||
`UserMessage` table to record those users receiving those messages.
|
||||
Each row is just 3 integers in size, but even with modern databases
|
||||
and SSDs, writing thousands of rows to a database starts to take a few
|
||||
|
@ -278,10 +278,10 @@ This isn’t a problem for most Zulip servers, but is a major problem
|
|||
for communities like chat.zulip.org, where there might be 10,000s of
|
||||
inactive users who only stopped by briefly to check out the product or
|
||||
ask a single question, but are subscribed to whatever the default
|
||||
streams in the organization are.
|
||||
channels in the organization are.
|
||||
|
||||
The total amount of work being done here was acceptable (a few seconds
|
||||
of total CPU work per message to large public streams), but the
|
||||
of total CPU work per message to large public channels), but the
|
||||
latency was unacceptable: The server backend was introducing a latency
|
||||
of about 1 second per 2000 users subscribed to receive the message.
|
||||
While these delays may not be immediately obvious to users (Zulip,
|
||||
|
@ -294,18 +294,18 @@ even simple questions).
|
|||
|
||||
A key insight for addressing this problem is that there isn’t much of
|
||||
a use case for long chat discussions among 1000s of users who are all
|
||||
continuously online and actively participating. Streams with a very
|
||||
continuously online and actively participating. Channels with a very
|
||||
large number of active users are likely to only be used for occasional
|
||||
announcements, where some latency before everyone sees the message is
|
||||
fine. Even in giant organizations, almost all messages are sent to
|
||||
smaller streams with dozens or hundreds of active users, representing
|
||||
smaller channels with dozens or hundreds of active users, representing
|
||||
some organizational unit within the community or company.
|
||||
|
||||
However, large, active streams are common in open source projects,
|
||||
However, large, active channels are common in open source projects,
|
||||
standards bodies, professional development groups, and other large
|
||||
communities with the rough structure of the Zulip development
|
||||
community. These communities usually have thousands of user accounts
|
||||
subscribed to all the default streams, even if they only have dozens
|
||||
subscribed to all the default channels, even if they only have dozens
|
||||
or hundreds of those users active in any given month. Many of the
|
||||
other accounts may be from people who signed up just to check the
|
||||
community out, or who signed up to ask a few questions and may never
|
||||
|
@ -330,14 +330,14 @@ organization for a few weeks, they are tagged as soft-deactivated.
|
|||
The way this works internally is:
|
||||
|
||||
- We (usually) skip creating UserMessage rows for soft-deactivated
|
||||
users when a message is sent to a stream where they are subscribed.
|
||||
users when a message is sent to a channel where they are subscribed.
|
||||
|
||||
- If/when the user ever returns to Zulip, we can at that time
|
||||
reconstruct the UserMessage rows that they missed, and create the rows
|
||||
at that time (or, to avoid a latency spike if/when the user returns to
|
||||
Zulip, this work can be done in a nightly cron job). We can construct
|
||||
those rows later because we already have the data for when the user
|
||||
might have been subscribed or unsubscribed from streams by other
|
||||
might have been subscribed or unsubscribed from channels by other
|
||||
users, and, importantly, we also know that the user didn’t interact
|
||||
with the UI since the message was sent (and thus we can safely assume
|
||||
that the messages have not been marked as read by the user). This is
|
||||
|
@ -369,9 +369,9 @@ The end result is the best of both worlds:
|
|||
with Zulip.
|
||||
|
||||
Empirically, we've found this technique completely resolved the "send
|
||||
latency" scaling problem. The latency of sending a message to a stream
|
||||
latency" scaling problem. The latency of sending a message to a channel
|
||||
now scales only with the number of active subscribers, so one can send
|
||||
a message to a stream with 5K subscribers of which 500 are active, and
|
||||
a message to a channel with 5K subscribers of which 500 are active, and
|
||||
it’ll arrive in the couple hundred milliseconds one would expect if
|
||||
the extra 4500 inactive subscribers didn’t exist.
|
||||
|
||||
|
@ -384,7 +384,7 @@ There are a few details that require special care with this system:
|
|||
assumed a `UserMessage` row would always exist for a message that
|
||||
triggers a notification to a given user.
|
||||
- Digest emails, which use the `UserMessage` table extensively to
|
||||
determine what has happened in streams the user can see. We can use
|
||||
determine what has happened in channels the user can see. We can use
|
||||
the user's subscriptions to construct what messages they should have
|
||||
access to for this feature.
|
||||
- Soft-deactivated users experience high loading latency when
|
||||
|
|
|
@ -170,11 +170,7 @@ not let the notifications become too "sticky" either.
|
|||
|
||||
## Roadmap
|
||||
|
||||
The most likely big change to typing indicators is that we will
|
||||
add them for stream conversations. This will require some thought
|
||||
for large streams, in terms of both usability and performance.
|
||||
|
||||
Another area for refinement is to tune the timing values a bit.
|
||||
An area for refinement is to tune the timing values a bit.
|
||||
Right now, we are possibly too aggressive about sending `stop`
|
||||
messages when users are just pausing to think. It's possible
|
||||
to better account for typing speed or other heuristic things
|
||||
|
|
|
@ -1,8 +1,8 @@
|
|||
# Unread message synchronization
|
||||
|
||||
In general displaying unread counts for all streams and topics may require
|
||||
In general displaying unread counts for all channels and topics may require
|
||||
downloading an unbounded number of messages. Consider a user who has a muted
|
||||
stream or topic and has not read the backlog in a month; to have an accurate
|
||||
channel or topic and has not read the backlog in a month; to have an accurate
|
||||
unread count we would need to load all messages this user has received in the
|
||||
past month. This is inefficient for web clients and even more for mobile
|
||||
devices.
|
||||
|
|
Loading…
Reference in New Issue