For endpoints with a `type` parameter to indicate whether the message
is a stream or direct message, `POST /typing` and `POST /messages`,
adds support for passing "direct" as the preferred value for direct
messages, group and 1-on-1.
Maintains support for "private" as a deprecated value to indicate
direct messages.
Fixes#24960.
We previously allowed moving messages that have passed the time limit
using "change_all" value for "propagate_mode" parameter. This commit
changes the behavior to not allow moving messages (both stream and
topic edit) that have passed the time limit for non-admin and
non-moderator users.
Adds the user ID to the return values for the `/fetch_api_key` and
`/dev_fetch_api_key` endpoints. This saves clients like mobile a
round trip to the server to get the user's unique ID as it is now
returned as part of the log in flow.
Fixes#24980.
This commit adds a new endpoint, 'POST /user_topics' which
is used to update the personal preferences for a topic.
Currently, it is used to update the visibility policy of
a user-topic row.
Primary goal of library replacement is improving execution speed.
This commit should not affect the functionality of the system
or make any changes to it.
This gives more comprehensive support of new and future CSS features
that can be transpiled for older browsers.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
Previously, `QuerySet` does not support isinstance check since it is
defined to be generic in django-stubs. In a recent update, such check is
possible by using `QuerySetAny`, a non-generic alias of `QuerySet`.
Signed-off-by: Zixuan James Li <p359101898@gmail.com>
This is a dependency of people.js. We include the stubs to prepare it
for a conversion to TypeScript.
Signed-off-by: Zixuan James Li <p359101898@gmail.com>
Currently, there is a checkbox setting for whether to
"Include realm name in subject of message notification emails".
This commit replaces the checkbox setting with a dropdown
having values: Automatic [default], Always, Never.
The Automatic option includes the realm name if, and only if,
there are multiple Zulip realms associated with the user's email.
Tests are added and(or) modified.
Fixes: #19905.
Zulip already has integrations for server-side Sentry integration;
however, it has historically used the Zulip-specific `blueslip`
library for monitoring browser-side errors. However, the latter sends
errors to email, as well optionally to an internal `#errors` stream.
While this is sufficient for low volumes of users, and useful in that
it does not rely on outside services, at higher volumes it is very
difficult to do any analysis or filtering of the errors. Client-side
errors are exceptionally noisy, with many false positives due to
browser extensions or similar, so determining real real errors from a
stream of un-grouped emails or messages in a stream is quite
difficult.
Add a client-side Javascript sentry integration. To provide useful
backtraces, this requires extending the pre-deploy hooks to upload the
source-maps to Sentry. Additional keys are added to the non-public
API of `page_params` to control the DSN, realm identifier, and sample
rates.
Creates `MutableJsonResponse` as a subclass of Django's `HttpResponse`
that we can modify for ignored parameters in the response content.
Updates responses to include `ignored_parameters_unsupported` in
the response data through `has_request_variables`. Creates unit
test for this implementation in `test_decorators.py`.
The `method` parameter processed in `rest_dispatch` is not in the
`REQ` framework, so for any tests that pass that parameter, assert
for the ignored parameter with a comment.
Updates OpenAPI documentation for `ignored_parameters_unsupported`
being returned in the JSON success response for all endpoints.
Adds detailed documentation in the error handling article, and
links to that page in relevant locations throughout the API docs.
For the majority of endpoints, the documentation does not include
the array in any examples of return values, and instead links to
the error handling page. The exceptions are the three endpoints
that had previously supported this return value. The changes note
and example for these endpoints is also used in the error
handling page.
This already became useless in 6e11754642,
as detailed in the API changelog entry here. At this point, we should
eliminate this param and the weird code around it.
This commit also deletes the associated tests added in
6e11754642, since with realm_str removed,
they make no sense anymore (and actually fail with an OpenAPI error due
to using params not used in the API). Hypothetically they could be
translated to use the subdomain= kwarg, but that also doesn't make
sense, since at that point they'd be just testing the case of a user
making an API request on a different subdomain than their current one
and that's just redundant and already tested generally in
test_decorators.
View that handled `PATCH user_groups/<int:user_group_id>` required
both name and description parameters to be passed. Due to this
clients had to pass values for both these parameters even if
one of them was changed.
To resolve this name description parameters to
`PATCH user_groups/<int:user_group_id>` are made optional.
Ever since we started bundling the app with webpack, there’s been less
and less overlap between our ‘static’ directory (files belonging to
the frontend app) and Django’s interpretation of the ‘static’
directory (files served directly to the web).
Split the app out to its own ‘web’ directory outside of ‘static’, and
remove all the custom collectstatic --ignore rules. This makes it
much clearer what’s actually being served to the web, and what’s being
bundled by webpack. It also shrinks the release tarball by 3%.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
Corepack manages multiple per-project version of Yarn and PNPM, which
means we have to maintain less installation code, and could help us
switch away from Yarn 1 without making the system unusable for
development of other Yarn 1 projects.
https://nodejs.org/api/corepack.html
The Unicode spaces in the timerender test resulted from an ICU
upgrade: https://github.com/nodejs/node/pull/45068.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
This commit adds time restriction on moving messages between streams
using the move_messages_between_streams_limit_seconds setting in the
backend. There is no time limit for admins and moderators.
- Updates `.prettierignore` for the new directory.
- Updates any reference to the API documentation directory for
markdown files to be `api_docs/` instead of `zerver/api/`.
- Removes a reference link from `docs/documentation/api.md` that
hasn't referenced anything in the text since commit 0542c60.
- Update rendering of API documentation for new directory.
The `django-sendfile2` module unfortunately only supports a single
`SENDFILE` root path -- an invariant which subsequent commits need to
break. Especially as Zulip only runs with a single webserver, and
thus sendfile backend, the functionality is simple to inline.
It is worth noting that the following headers from the initial Django
response are _preserved_, if present, and sent unmodified to the
client; all other headers are overridden by those supplied by the
internal redirect[^1]:
- Content-Type
- Content-Disposition
- Accept-Ranges
- Set-Cookie
- Cache-Control
- Expires
As such, we explicitly unset the Content-type header to allow nginx to
set it from the static file, but set Content-Disposition and
Cache-Control as we want them to be.
[^1]: https://www.nginx.com/resources/wiki/start/topics/examples/xsendfile/
This commit changes the topic edit permssions to not depend whether the user
editing the message had sent the message or it was sent by someone else.
We only do backend changes in this commit and frontend changes will be done
in further commits.
Previously, we always allowed topic edits when the user themseleves had
sent the message not considering the edit_topic_policy and the 3-day time
limit. But now we consider all messages as same and editing is allowed only
according to edit_topic_policy setting and the time limit of 3 days in
addition for users who are not admins or moderators.
Previously we did not send notification for topic-only edits.
Now, we add backend support for sending notification to topic-only
edits as well.
We would add support for this in webapp in further commits since
message edit UI will be updated as well. We just make sure that no
notifications are sent when editing topic using pencil icon in
message header.
We also change the API default for moving a topic to only notify the
new location, not the old one; this matches the current defaults in
the web UI.
Includes many tests.
We also update the puppeteer tests to test only content edit as
we are going to change the UI to not allow topic editing from
message edit UI. Also fixing the existing tests to pass while
doing topic edits is somewhat complex as notification message
is also sent to new topic by default.
Fixes#21712.
Co-authored-by: Aman Agrawal <amanagr@zulip.com>
Co-authored-by: Tim Abbott <tabbott@zulip.com>
Although Node.js 18 is not the active LTS release for another 3 weeks,
the Node.js 16 end-of-life date was moved forward to September 2023,
(https://nodejs.org/en/blog/announcements/nodejs16-eol/), so it seems
prudent to switch now.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
zerver/migrations/0240_usermessage_migrate_bigint_id_into_id.py needs
to be updated to account for Django 4.1 creating AutoField as an
identity column rather than a serial column.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
Note that django_stubs_ext is required to be placed within common.in
because we need the monkeypatched types in runtime; django-stubs
itself is for type checking only.
In the future, we would like to pin to a release instead of a git
revision, but several patches we've contributed upstream have not
appeared in a release yet.
We also remove the type annotation for RealmAuditLog.event_last_message_id
here instead of earlier because type checking fails otherwise.
Fixes#11560.
Previously, we included all three message edit related settings
("allow_message_editing", "message_content_edit_limit_seconds" and
"edit_topic_policy") in the event data and api response irrespective
of which of these settings were changed. Now, we only include changed
settings and separate events are sent for each setting if more than
one of them is changed.
Note that the previous typed in event_schema.py for
`message_content_edit_limit_seconds` incorrectly did not allow `None`
as a value, which is used to encode no limit.
Unfortunately, doing so requires forking common API documentation
text, since we're not making any changes to other endpoints that don't
allow unauthenticated requests at all.
Follow-up on #21995.
Updates API documentation and changelog for user status `away`
now being a deprecated way to access a user's `presence_enabled`
setting for clients supporting older servers.
Final step in making user status `away` a deprecated way to access
`presence_enabled` for clients supporting older servers.
Part of transitioning from 'unavailable' user status feature to
'invisible mode' user presence feature.
Fixes#22821.
As explained in the comment in the code:
Topics can be large enough that this request will inevitably time out.
In such a case, it's good for some progress to be accomplished, so that
full deletion can be achieved by repeating the request. For that purpose,
we delete messages in atomic batches, committing after each batch.
The additional perk is that the ordering of messages should prevent some
hypothetical deadlocks - ref #19054
To allow `custom_profile_field` to display in user profile popover,
added new boolean field "display_in_profile_summary" in its model class.
In `custom_profile_fields.py`, functions are edited as per conditions,
like currently we can display max 2 `custom_profile_fields` except
`LONG_TEXT` and `USER` type fields.
Default external account custom profile fields made updatable for only
this new field, as previous they were not updatable.
Fixes part of: #21215
This commit adds do_change_can_remove_subscriber_group function for
changing can_remove_subscribers_group field of a stream. We also add
can_remove_subscribers_group_id field to stream and subscription
objects.
This function will be helpful for writing tests in next commit.
We would add API and UI support to change this setting in further
commits.
Added a user_list_style personal user setting to the bottom of
Settings > Display settings > Theme section which controls the look
of the right sidebar user list.
The radio button UI includes a preview of what the styles look like.
The setting is intended to eventually have 3 possible values: COMPACT,
WITH_STATUS and WITH_AVATAR; the final value is not yet implemented.
Co-authored-by: Tim Abbott <tabbott@zulip.com>
Fixes#21037.
This is part of fixing #19371. To bulk-add new emoji regularly,
mobile needs to know which servers support which emoji.
`staticfiles_storage.url` generates a unique URL with a hash
based on the file content, which lets mobile know if it needs
to update its locally stored data.
In Zulip 2.1.0, the `is_muted` stream subscription property was
added and replaced the `in_home_view` property. But the server has
still only been sending subscription update events with the
`in_home_view` property.
Updates `do_change_subscription_property` to send a subscription
update event for both `is_muted` and `in_home_view`, so that
clients can fully migrate away from using `in_home_view` allowing
us to eventually remove it completely.
This commit changes the code to consider zero as an invalid value for
message_content_edit_time_limit_seconds. Now to represent the setting that
user can edit the message anytime, the setting value will be "None" in
database and "unlimited" will be passed to API from clients.
Adds an API endpoint for accessing read receipts for other users, as
well as a modal UI for displaying that information.
Enables the previously merged privacy settings UI for managing whether
a user makes read receipts data available to other users.
Documentation is pending, and we'll likely want to link to the
documentation with help_settings_link once it is complete.
Fixes#3618.
Co-authored-by: Tim Abbott <tabbott@zulip.com>
This commit adds support to change enable_read_receipts
setting through API and also adds the field to response
of "/register" endpoint so that the setting value
is available to clients.
This includes the change from 28fde2ee27.
Only a minor bump is required because it has no effect on type
checking yet before django-stubs gets integrated.
Signed-off-by: Zixuan James Li <p359101898@gmail.com>
We now allow changing access to history of the stream by only passing
"history_public_to_subscribers" parameter. Previously, "is_private"
parameter was also required to change history_public_to_subscribers
otherwise the request was silently ignored.
We also raise error when only history_public_to_subscribers parameter
is passed with value False without "is_private: True" for a public
or web-public stream since we do not allow public streams with
protected history.
We now send a new user_topic event while muting and unmuting topics.
fetch_initial_state_data now returns an additional user_topics array to
the client that will maintain the user-topic relationship data.
This will support any future addition of new features to modify the
relationship between a user-topic pair.
This commit adds the relevent backend code and schema for the new
event.
Now that we can assume Python 3.6+, we can use the
email.headerregistry module to replace hacky manual email address
parsing.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
distutils is deprecated in Python 3.10 and will be removed in Python
3.12. We don’t need a full-powered version parser for this anyway.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
This commit removes "role" field from subscription
objects since we are not moving forward with stream
administrator concept and instead working on new
permssions model as per #19525.
markdown-include is GPL licensed.
Also, rewrite it as a block processor, so that it works correctly
inside indented blocks.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
Set the default_language as cookie and reload the page so that
the spectator can immediately see the language change in effect.
We can reload the page forcefully for spectators since there is
no chance of any work being lost. It is possible that the spectator
may lose the selected message on doing so.
This requires a new dependency, to be able to set cookies from
frontend JavaScript.
Fixes#21961
This commit attempts to add the backend support by extending the
/json/bots/{bot_id}/ url support to accept the role field as a
parameter. This was previously already possible via
`/json/users/{user_id}`, so this change just simplifies client
implementation.
Adds `want_advertise_in_communities_directory` to the realm model
to track organizations that give permission to be listed on such
a site / directory on zulip.com.
Adds a checkbox to the organization profile admin for
organizations to give permission to be advertised in the
Zulip communities directory.
Adds a help center article about the Zulip communities directory
and uses a shared intro documentation file to create sections in
the articles on creating an organization profile and moderating
open organizations.
Co-authored-by: Alya Abbott <alya@zulip.com>
As a consequence:
• Bump minimum supported Python version to 3.8.
• Move Vagrant environment to Ubuntu 20.04, which has Python 3.8.
• Move CI frontend tests to Ubuntu 20.04.
• Move production build test to Ubuntu 20.04.
• Move 3.4 upgrade test to Ubuntu 20.04.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
`org_type` already exists as a field in the Realm model and is
used when organizations are created / updated in Zulip Cloud,
via the `/analytics/support` view.
Extends the `PATCH /realm` view to be able update `org_type` as
other realm / organization settings are updated, but using the
special log / action that was created for the analytics view.
Adds a field to the `realm op: update` / `realm op: update_dict`
events, which also means an event is now sent when and if the
`org_type` is updated via the analytics view. This is similar
to how updates to an organization's `plan_type` trigger events.
Adds `realm_org_type` as a realm setting fetched from the
`POST /register` endpoint.
This commit changes the invite API to accept invitation
expiration time in minutes since we are going to add a
custom option in further commits which would allow a user
to set expiration time in minutes, hours and weeks as well.
Added a setting to the bottom of Settings > Display settings > Theme section
to display the reacting users on a message when numnber of reactions are
small.
This is a preparatory commit for #20980.
Co-authored-by: Steve Howell <showell@zulip.com>
Co-authored-by: Tim Abbott <tabbott@zulip.com>
This commit adds the backend functionality to
mark messages as unread through update_message_flags
with `unread` flag and `remove` operation.
We also manage incoming events in the webapp.
Tweaked by tabbott to simplify the implementation and add an API
feature level update to the documentation.
This commit was originally drafted by showell, and showell
also finalized the changes. Many thanks to Suyash here for
the main work here, which was to get all the tests and
documentation work moving forward.
Previously, this URL just returned the `raw_content` field. It seems
cleanest to just make it a single-message variant of GET /messages,
deprecating the only format.
Since we've changed the database to contain these new fields, we just
need to stop dropping them in the API code.
This also changes the public API to match the database format again
by removing `prev_subject` from edit history API.
Adds an API changelog feature update for the renamed `prev_subject`
field (to `prev_topic`) and new fields (`topic` and `stream`)
in the message `edit_history`.
Also, documents said `edit_history` in the `MessagesBase` schema
in the api documentation, which is used by the `/get-messages`,
`/get-events` and `/zulip-outgoing-webhooks` endpoints.
Fixes#21076.
Co-authored-by: Lauryn Menard <lauryn.menard@gmail.com>
The database value for expiry_date is None for the invite
that will never expire and the clients send -1 as value
in the API similar to the message retention setting.
Also, when passing invite_expire_in_days as an argument
in various functions, invite_expire_in_days is passed as
-1 for "Never expires" option since invite_expire_in_days
is an optional argument in some functions and thus we cannot
pass "None" value.
This PR changes the library used for panning and zooming in the lightbox
module from timmywil/panzoom to avanka/panzoom.
The original (timmywil) version of the library contains a bug where if
you have a high resolution touchpad and Firefox and you zoom in and out
repeatedly on an image, the image may drift. avanka/panzoom does not
appear to display this behavior.
Restores the behaviour from before 5f83bc5cfe, where clicking
outside the image closes the lightbox, primarily by way of swapping
out the panzoom library.
Fixes: #21163.
After failing to notice a place where we wanted to hide timezone
information, we decided to add timezones to some of the test
users, so that we can better consider the effects of timezones
when manually testing.
Testing:
* ran populate_db and confirmed users had timezones in the UI
* updated test_populate_db.py
Adds `realm_web_public_access_enabled` as a realm-specific server
setting potentially returned by the `/get-server-settings` endpoint
so that clients that support browsing of web-public stream content
without an account can generate a login page that supports that
type of access.
For aliases that will no longer be listed, see the third column of
grep '^L ' zulip-py3-venv/lib/python3.*/site-packages/pytz/zoneinfo/tzdata.zi
Time zones previously set to an alias will be canonicalized on demand.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
The new release adds the commit:
20ac22b96d
Which allows us to get rid of the entire ugly override that was needed
to do this commit's job in our code. What we do here in this commit:
* Use django-scim2 0.17.1
* Revert the relevant parts of f5a65846a8
* Adjust the expected error message in test_exception_details_not_revealed_to_client
since the message thrown by django-scim2 in this release is slightly
different.
We do not have to add anything to set EXPOSE_SCIM_EXCEPTIONS, since
django-scim2 uses False as the default, which is what we want - and we
have the aforementioned test verifying that indeed information doesn't
get revealed to the SCIM client.
This PR changes how the Pan & Zoom feature of images displayed in the
attachment lightbox are handled.
The existing method of using a canvas element is replaced by the Panzoom
library (timmywil/panzoom). This library is lightweight and has 0
transitive dependencies.
This fixes#20759 where the issue is that the viewport of a zoomed image
was not expanding to fill the available space on the page. Switching to
this new library also solves several other UX issues:
* Images are no longer blurred when in Pan & Zoom mode.
* The zoom behavior itself uses focal point zooming: zooming occurs
where the cursor is on the image instead of at the center of the
image, reducing the need for extra panning.
* CSS transitions are used for a more visually pleasing experience
when switching images, toggling zoom off, etc.
* The library has the potential to open other file types which
leaves that option open for us in the future.
Makes `edit_timestamp` and `user_id` required fields for all
`update_message` events.
Adds `rendering_only` as another required field to signal if
events are only updating the rendered content of the message,
which is currently the case for adding inline url previews.
Updates `test_event.py` so that `do_update_message` and
`do_update_embedded_data` refer to the same testing schema
for `update_message` events, and therefore reflect the same
required fields for the `update_message` event.
The OpenAPI definition for `update_message` events is also
updated to reflect the required field and descriptions of
various properties are updated for the addition of the
`rendering_only` property.
As a consequence:
• Bump minimum supported Python version to 3.7.
• Move Vagrant environment to Debian 10, which has Python 3.7.
• Move CI frontend tests to Debian 10.
• Move production build test to Debian 10.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
Updating from pygments 2.10.x to 2.11.x brings new lexers,
including the new Savi lexer which is needed by the Savi community
in our Zulip chat at https://savi.zulipchat.com/.
We now serialize still_url as None for non-animated emojis,
instead of omitting the field. The webapp does proper checks
for falsiness here. The mobile app does not yet use the field
(to my knowledge).
We bump the API version here. More discussion here:
https://chat.zulip.org/#narrow/stream/378-api-design/topic/still_url/near/1302573
Updates the `update_message` event type to always include a `stream_id`
field when the message being edited is a stream message. This change
aligns with the current definition of the `\get-events` endpoint
in the OpenAPI documentation.
In https://github.com/jorisroovers/gitlint/pull/246 I split the
gitlint package into gitlint and gitlint-core, where the latter avoids
pinning exact versions of its requirements so we can use it again.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
Migrates the `/update-subscription-settings` api endpoint to the
`ignored_parameters_unsupported` model, which is also currently used
by `/update-settings` and `update-realm-user-settings-defaults`.
This change is a step towards preparing for an eventual migration to
have all endpoints return an `ignored_parameters_unsupported` block.
Previously the `/update-subscription-settings` endpoint returned a
copy of the data object sent in the request.
Fixes#15307.
We restrict access of messages from web public streams if
anonymous login is disabled via `enable_spectator_access`.
Display of `Anonymous login` button is now controlled by
the value of `enable_spectator_access`.
Admins can toggle `enable_spectator_access` via org settings in UI.
The user id is a very useful piece of information that the mobile
client should have access to - instead of only getting the email. This
makes it much simpler to impleent clients that might be robust to
changes in email address.
TOR users are legitimate users of the system; however, that system can
also be used for abuse -- specifically, by evading IP-based
rate-limiting.
For the purposes of IP-based rate-limiting, add a
RATE_LIMIT_TOR_TOGETHER flag, defaulting to false, which lumps all
requests from TOR exit nodes into the same bucket. This may allow a
TOR user to deny other TOR users access to the find-my-account and
new-realm endpoints, but this is a low cost for cutting off a
significant potential abuse vector.
If enabled, the list of TOR exit nodes is fetched from their public
endpoint once per hour, via a cron job, and cached on disk. Django
processes load this data from disk, and cache it in memcached.
Requests are spared from the burden of checking disk on failure via a
circuitbreaker, which trips of there are two failures in a row, and
only begins trying again after 10 minutes.
In #20012, it was discovered that since our `zulip_bots` package
requires `importlib-metadata >= 3.6; python_version < "3.10"`
whereas the server requires
`importlib-metadata==4.8.1 ; python_version < "3.8". This results
in `importlib-metadata` not being installed on Python 3.8 and
Python 3.9. This commit resolves that discrepancy.
Thanks to Anders Kaseorg (@andersk) for reporting this bug!
Add `escape_navigates_to_default_view` as a bool setting in
UserBaseSettings model and implement it as a checkbox that toggles
the hotkey implementation of escape to the default view in the
advanced user display settings.
With /help/ documentation edits from Alya Abbott.
Fixes#20043.
This tool helps catch common typos in code and documentation, which is
particularly useful for our many contributors who are not native
English speakers.
The config is based on the codespell that I ran in
https://github.com/zulip/zulip/pull/18535.
* Remove unnecessary json_validator for full_name parameter.
* Update frontend to pass the right parameter.
* Update documentation and note the change.
Fixes#18409.
This commit adds django-cte as dependency
which will be used for querying recursive
group membership.
Extracted this commit from #19866.
Co-authored-by: Anders Kaseorg <anders@zulip.com>
Note: These are not functional in enabling/disabling sending of
typing notifications with this commit.
Refactored the privacy settings update to keep the code less
duplicated along with making the addition of new settings easier.
re2[1] compiles (strictly) regular expressions to deterministic finite
automata, which guarantees linear-time behavior; `google-re2` is a
drop-in replacement for the `re` module which uses re2 under the hood.
[1]: https://github.com/google/re2/
Users wanted a feature where they could specify
which users can create public streams and which users can
create private streams.
This splits stream creation code into two parts,
public and private stream creation.
Fixes#17009.
This commit replaces 'allow_message_deleting' boolean setting
with an integer setting 'delete_own_message_policy'. We have a
separate dropdown now for deciding which user-roles can delete
messages sent by themselves and the time-limit setting droddown
is different.
This new setting has two options - everyone and admins only. Other
options including moderators will be added further.
We also remove the "Never" option from the original time-limit
dropdown, as admins are always allowed to delete message. This
never option resembled the case of only admins being allowed to
delete but this state is now resembled by setting the dropdown
to "admins only" and we also disable the time-limit dropdown in
this case as admins are allowed to delete irrespective of limit.
Note, this setting is only for deleting messages sent by the
deleting user themselves, and only admins are allowed to delete
messages sent by others as before.
We make zero invalid value for message_content_delete_limit_seconds and
for handling the case of "Allow to delete message any time", the API-level
value of message_content_delete_limit_seconds is "anytime" and "None"
as the DB-level value. We also use these values for message retention
setting, so it helps maintain consistency.
This commit removes the existing default_twenty_four_hour_time field in
Realm table which was used to set the twenty_four_hour_time setting of
new user on joining and instead we now use the twenty_four_hour_time
field of RealmUserDefault table for the same.
With some tweaks by tabbott to clarify the documentation.
User can now create web public stream via the /subscribe API.
So, when a web public stream present in the API request does not
exist, it will be created now by specifying the is_web_public
parameter. The parameter would have been ignored without this
commit.
gitlint has a bunch of pinned requirements that hold back important
upgrades and conflict with other packages’ requirements. The gitlint
author has rejected proposals to unpin them because it might increase
the amount of maintenance he needs to do
(https://github.com/jorisroovers/gitlint/pull/133). That decision is
his to make, but _somebody_ needs to do the maintenance, so we
delegate it to Debian and Ubuntu. If that means using a significantly
older version of gitlint, that’s a tradeoff we need to make to keep
the rest of our requirements current.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
We add postcss-import for night_mode.css only. This plugin inlines
the imports of external files, instead of letting the file go via
our usual webpack toolchain.
We do this so that we can use the postcss-prefixwrap plugin to scope
the third-party CSS properly and use it inside our night-mode class.
Fixes#10607.
[anders@zulip.com: Replace postcss-wrap with postcss-prefixwrap.]
Co-authored-by: Anders Kaseorg <anders@zulip.com>
Signed-off-by: Anders Kaseorg <anders@zulip.com>
Now, when we add a custom animated emoji to the realm
we also save a still image of it (1st frame of the gif). So
we can avoid showing an animated emoji every time.
This commit adds do_set_realm_user_default_setting which
will be used to change the realm-level defaults of settings
for new users.
We also add a new event type "realm_user_settings_defaults"
for these settings and a "realm_user_settings_default" object
in '/register' response containing all the realm-level default
settings.
This commit adds `demo_organization_scheduled_deletion_date` to
the `realm` section of the `/register` response so that it is
available to clients when enabled.
This is a part of #19523.
Along with the extraction, we do some simplifications of inserting
text in compose too. This same function can now be used in
compose formatting popover too.
We use "text-field-edit", which has good cross-browser `undo` /
`redo` support, to do the text replace for us instead of writing
that logic ourselves.
Till now, we've been forking django-auth-ldap at
https://github.com/zulip/django-auth-ldap to put the
LDAPReverseEmailSearch feature in it, hoping to get it merged
upstream in https://github.com/django-auth-ldap/django-auth-ldap/pull/150
The efforts to get it merged have stalled for now however and we don't
want to be on the fork forever, so this commit puts the email search
feature as a clumsy workaround inside our codebase and switches to using
the latest upstream release instead of the fork.
Recommonmark is no longer maintained, and MyST-Parser is much more
complete.
https://myst-parser.readthedocs.io/
Signed-off-by: Anders Kaseorg <anders@zulip.com>
This commit updates both the stream-level and realm-level message
retention setting to use 'unlimited' instead of 'forever' to set
message retention setting to "retain messages forever".
We send a event with type 'user_settings' on updating user's display
and notification settings.
The old event types - 'update_global_notifications' and
'update_display_settings', are still supported for backwards
compatibility.
Return zulip_merge_base alongside zulip_version
in `/register`, `/event` and `/server_settings`
endpoint so that the value can be used by other
clients.
In this commit:
* We update the `UserStatus` model to accept
`AbstractReaction` as a base class so, we can get all the
fields related to store status emoji.
* We update the user status endpoint
(`users/me/status`) to accept status emoji fields.
* We update the user status event to add status emoji
fields.
Co-authored-by: Yash Rathore <33805964+YashRE42@users.noreply.github.com>
This commit replaces boolean field add_emoji_by_admins_only with an
integer field add_custom_emoji_policy as we would also add full members
and moderators option for this setting in further commits.
Cross realm bots will soon stop being a thing. This param is responsible
for displaying "System Bot" in the user info popover - so this rename is the
right way to handle the situation.
We will likely want to rename the `cross_realm_bots` section as well,
but that is a more involved API migration.
This API change removes unnecessary complexity from a client that
wants to change a user's personal settings, and also saves developers
from needing to make decisions about what sort of setting something is
at the API level.
We preserve the old settings endpoints as mapping to the same function
as the new one for backwards-compatibility. We delete the
documentation for the old endpoints, though the documentation for the
merged /settings endpoint mentions how to use the old endpoints when
needed.
We migrate all backend tests to the new endpoints, except for
individual tests for each legacy endpoint to verify they still work.
Co-authored-by: sahil839 <sahilbatra839@gmail.com>
Sometime in the deep past, Zulip the GET /users/me/subscriptions
endpoint started returning subscribers. We noticed this and made it
optional via the include_subscribers parameter in
1af72a2745, however, we didn't notice
that they were being returned as emails rather than user IDs.
We migrated the core /register code paths to use subscriber IDs years
ago; this change completes that for the endpoints we forgot about.
The documentation allowed this error because we apparently had no
tests for this code path that used the actual API.
We remove the "full_name" and "account_email" fields from the response
of 'PATCH /settings' endpoint. These fields were part of the response
to make sure that we tell that the parameters not present in response
were ignored.
We can remove these fields as 'ignored_parameters_unsupported' now
specifies which parameters were ignored and not supported by the
endpoint.
An organization with at most 5 users that is behind on payments isn't
worth spending time on investigating the situation.
For larger organizations, we likely want somewhat different logic that
at least does not void invoices.
This commit replaces the allow_community_topic_editing boolean with
integer field edit_topic_policy and includes both frontend and
backend changes.
We also update settings_ui.disable_sub_settings_onchange to not
change the color of label as we did previously when the setting
was a checkbox. But now as the setting is dropdown we keep the
label as it is and we don't do anything with label when disabling
dropdowns. Also, this function was used only here so we can safely
change this.
This guards against various mistakes, such as setting defaultMessage
to a computed expression that can’t be extracted.
https://formatjs.io/docs/tooling/linter/
Signed-off-by: Anders Kaseorg <anders@zulip.com>
Rename poll_timeout to event_queue_longpoll_timeout_seconds
and change its value from 90000 ms to 90 sec. Expose its
value in register api response when realm data is fetched.
Bump API_FEATURE_LEVEL to 74.
We now organize the pygment language codes into meaningful categories
- default, custom and aliases.
Further the `lang.json` list now contains a dataset extracted from the
"language" section of https://insights.stackoverflow.com/survey/2020
and is prioritized based on current language trends.
Since this is currently only useful to interpret presence data, we
send this only if presence is requested.
I'm not sure that server_timestamp is the right name for this field,
but ultimately it should match the main presence API format.
We record Git details about the merge-base with upstream branches in
the zulip-git-version file, if the upstream repository is available.
Note that the first Git upgrade after merging the parent commit will
not include the merge-base details, since the upstream repository will
not have been available.
Co-authored-by: Tim Abbott <tabbott@zulip.com>
Signed-off-by: Anders Kaseorg <anders@zulip.com>
* Remove unnecessary json_validator for string parameters.
* Update frontend to pass right parameter.
Bump api feature level and highlight the fix for `emojiset`
parameter of `settings/display` endpoint in zulip.yaml file.
Fixes part of #18035.
This removes unnecessary json_validator for string parameters in the
BigBlueButton video calls endpoints. Note that this breaks links to
video meetings sent before the upgrade; there's not much we can do
about that.
Since this is the last commit in this series, we update the
ZULIP_FEATURE_LEVEL for this batch of changes.
Fixes part of #18035.
Thumbor and tc-aws have been dragging their feet on Python 3 support
for years, and even the alphas and unofficial forks we’ve been running
don’t seem to be maintained anymore. Depending on these projects is
no longer viable for us.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
This is a straightforward upgrade in terms of changes needed.
Necessary changes were:
- Set `DEFAULT_AUTO_FIELD`
https://docs.djangoproject.com/en/3.2/releases/3.2/#customizing-type-of-auto-created-primary-keys
- `The default_app_config application configuration variable is deprecated, due
to the now automatic AppConfig discovery.`
https://docs.djangoproject.com/en/3.2/releases/3.2/#automatic-appconfig-discovery
To handle this one, we can remove default_app_config from
zerver/__init__.py because it satisfies what release notes describe in
https://docs.djangoproject.com/en/3.2/releases/3.2/#automatic-appconfig-discovery:
"Most pluggable applications define an AppConfig subclass in an apps.py
submodule. Many define a default_app_config variable pointing to this
class in their __init__.py. When the apps.py submodule exists and
defines a single AppConfig subclass, Django now uses that configuration
automatically, so you can remove default_app_config."
An important note is that rebuild-test-database needs to be run after
this upgrade in dev environment - if tests are run with test db that was
built on the previous version, they will fail due to a mysterious bug
(?), where changing attributes of a user and .save()ing after logging in
in the test via self.login_user, causes getting logged out - the next
requests via self.client_get etc. are unauthed for some reason,
unless self.login_user is called again. This behavior is no longer
exhibited upon rebuilding the test db - and I can't reproduce it in
production or dev db. So this can likely be reasonably dismissed as some
quirk of the test client system that won't be relevant in the future and
doesn't impact production.
This commit adds both frontend and backend code to invite a user as
moderator. We allow only existing owners and admins to invite a user
as a moderator.
This commit modifies the user objects returned by 'GET /users',
'GET /users/me', 'GET /users/{user_id}' and 'GET /users/{email}'
endpoints to include role field.
We also include role field in the page_params['realm_users'] dict
and in the person object sent in (type="realm_user", op="add")
event.
This extends the /json/typing endpoint to also accept
stream_id and topic. With this change, the requests
sent to /json/typing should have these:
* `to`: a list set to
- recipients for a PM
- stream_id for a stream message
* `topic`, in case of stream message
along with `op`(start or stop).
On receiving a request with stream_id and topic, we send
typing events to clients with stream_typing_notifications set
to True for all users subscribed to that stream.
Previously the outgoing emails were sent over several SMTP
connections through the EmailSendingWorker; establishing a new
connection each time adds notable overhead.
Redefine EmailSendingWorker worker to be a LoopQueueProcessingWorker,
which allows it to handle batches of events. At the same time, persist
the connection across email sending, if possible.
The connection is initialized in the constructor of the worker
in order to keep the same connection throughout the whole process.
The concrete implementation of the consume_batch function is simply
processing each email one at a time until they have all been sent.
In order to reuse the previously implemented decorator to retry
sending failures a new method that meets the decorator's required
arguments is declared inside the EmailSendingWorker class. This
allows to retry the sending process of a particular email inside
the batch if the caught exception leaves this process retriable.
A second retry mechanism is used inside the initialize_connection
function to redo the opening of the connection until it works or
until three attempts failed. For this purpose the backoff module
has been added to the dependencies and a test has been added to
ensure that this retry mechanism works well.
The connection is closed when the stop method is called.
Fixes: #17672.
This reverses the policy that was set, but incompletely enforced, by
commit 951514dd7d. The self-closing tag
syntax is clearer, more consistent, simpler to parse, compatible with
XML, preferred by Prettier, and (most importantly now) required by
FormatJS.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
This commit adds an API to `zproject/urls.py` to edit/update
the realm linkifier. Its helper function to update the
database is added in `zerver/lib/actions.py`.
`zulip.yaml` is documented accordingly as well, clearly
stating that this API updates one linkifier at a time.
The tests are added for the API and helper function which
updates the realm linkifier.
Fixes#10830.
Organization admins can use this setting to restrict the maximum
rating of GIFs that will be retrieved from GIPHY. Also, there
is option to disable GIPHY too.
* This introduces a new event type `realm_linkifiers` and
a new key for the initial data fetch of the same name.
Newer clients will be expected to use these.
* Backwards compatibility is ensured by changing neither
the current event nor the /register key. The data which
these hold is the same as before, but internally, it is
generated by processing the `realm_linkifiers` data.
We send both the old and the new event types to clients
whenever the linkifiers are changed.
Older clients will simply ignore the new event type, and
vice versa.
* The `realm/filters:GET` endpoint (which returns tuples)
is currently used by none of the official Zulip clients.
This commit replaces it with `realm/linkifiers:GET` which
returns data in the new dictionary format.
TODO: Update the `get_realm_filters` method in the API
bindings, to hit this new URL instead of the old one.
* This also updates the webapp frontend to use the newer
events and keys.
Previously, you had to request the `stream` event type in order to get
the stream-level parameters; this was a bad design in part because the
`subscription` event type has similar data and is preferred by most
clients.
So we move these to the `realm` object. We also add the maximum topic
length, as an adjacent parameter.
While changing this, we also fix these to better match the names of
similar API parameters.
* Don't require strings to be unnecessarily JSON-encoded.
* Use check_capped_string rather than custom code for length checks.
* Update frontend to pass the right parameters.
With a much simplified populate_data_for_request design suggested by
Anders; we only support a handful of data types, all of which are
correctly encoded automatically by jQuery.
Fixes part of #18035.
This commit adds backend code for passing can_invite_others_to_realm
field to clients using the fetch_initial_state_data in the page_params
object.
Though this field is not used by webapp as of now, but will be used
to fix a bug of incorreclty showing the invite users option in
settings overlay in the next commit.
This commit replaces invite_by_admins_policy, which was a bool field,
with a new enum field invite_by_realm_policy.
Though the final goal is to add moderators and full members option
using COMMON_POLICY_TYPES, but this will be done in a separate
commit to make this easy for review.
Our aim is to use this library to remove use of bootstrap-tooltip
for showing popovers and tooltips. This will remove our
dependency on bootstrap for showing tooltips. Thus, bootstrap
can be upgrade more independently.
We send the whole data set as a part of the event rather than
doing an add/remove operation for couple of reasons:
* This would make the client logic simpler.
* The playground data is small enough for us to not worry
about performance.
Tweaked both `fetch_initial_state_data` and `apply_events` to
handle the new playground event.
Tests added to validate the event matches the expected schema.
Documented realm_playgrounds sections inside /events and
/register to support our openapi validation system in test_events.
Tweaked other tests like test_event_system.py and test_home.py
to account for the new event being generated.
Lastly, documented the changes to the API endpoints in
api/changelog.md and bumped API_FEATURE_LEVEL.
Tweaked by tabbott to add an `id` field in RealmPlayground objects
sent to clients, which is essential to sending the API request to
remove one.
Adds backend code for the mute users feature.
This is just infrastructure work (database
interactions, helpers, tests, events, API docs
etc) and does not involve any behavioral/semantic
aspects of muted users.
Adds POST and DELETE endpoints, to keep the
URL scheme mostly consistent in terms of `users/me`.
TODOs:
1. Add tests for exporting `zulip_muteduser` database table.
2. Add dedicated methods to python-zulip-api to be used
in place of the current `client.call_endpoint` implementation.
We use GIPHY web SDK to create popover containing GIFs in a
grid format. Simply clicking on the GIFs will insert the GIF in the compose
box.
We add GIPHY logo to compose box action icons which opens the GIPHY
picker popover containing GIFs with "Powered by GIPHY"
attribution.
Lauryn Menard
Alex Vandiver
Aman Agrawal
Anders Kaseorg
Sahil Batra
Prakhar Pratyush
Daniil Fadeev
Zixuan James Li
Tim Abbott
Mateusz Mandera
m-e-l-u-h-a-n
Julia Bichler
PIG208
yogesh sirsat
Raghav Luthra
evykassirer
Dinesh
Kartik Srivastava
Matt Keller
Somesh Ranjan
Suyash Vardhan Mathur
Steve Howell
Lauryn Menard
Austin Riba
Austin Riba
Joe Eli McIlvain
Eeshan Garg
rht
Ganesh Pawar
Eeshan Garg
Gaurav Pandey
sahil839
Rohitt Vashishtha
Riken Shah
Hemanth V. Alluri
Priyansh Garg
Priyank Patel
Vishnu KS
Sumanth V Rao
Cyril Pletinckx
akshatdalton
Abhijeet Prasad Bodas