Commit Graph

755 Commits

Author SHA1 Message Date
Anders Kaseorg 6bc3b267e1 webpack: Switch to css-minimizer-webpack-plugin.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-04-07 18:43:36 -07:00
Anders Kaseorg 2ce24bcd2b dependencies: Upgrade webpack-cli from 3.3.12 to 4.6.0.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-04-07 18:43:36 -07:00
sahil839 86f7b6af60 events: Pass can_invite_others_to_realm to client using page_params.
This commit adds backend code for passing can_invite_others_to_realm
field to clients using the fetch_initial_state_data in the page_params
object.

Though this field is not used by webapp as of now, but will be used
to fix a bug of incorreclty showing the invite users option in
settings overlay in the next commit.
2021-04-07 09:05:16 -07:00
sahil839 4c8339fa8c settings: Replace invite_by_admins_policy with invite_to_realm_policy.
This commit replaces invite_by_admins_policy, which was a bool field,
with a new enum field invite_by_realm_policy.

Though the final goal is to add moderators and full members option
using COMMON_POLICY_TYPES, but this will be done in a separate
commit to make this easy for review.
2021-04-07 09:02:33 -07:00
Aman Agrawal 9c6f5d5946 tippyjs: Add library to be used for showing tooltips.
Our aim is to use this library to remove use of bootstrap-tooltip
for showing popovers and tooltips. This will remove our
dependency on bootstrap for showing tooltips. Thus, bootstrap
can be upgrade more independently.
2021-04-07 01:16:28 -07:00
Sumanth V Rao 1ac8fe7538 events/tests/api: Send realm_playground events to clients.
We send the whole data set as a part of the event rather than
doing an add/remove operation for couple of reasons:
    * This would make the client logic simpler.
    * The playground data is small enough for us to not worry
      about performance.

Tweaked both `fetch_initial_state_data` and `apply_events` to
handle the new playground event.

Tests added to validate the event matches the expected schema.

Documented realm_playgrounds sections inside /events and
/register to support our openapi validation system in test_events.

Tweaked other tests like test_event_system.py and test_home.py
to account for the new event being generated.

Lastly, documented the changes to the API endpoints in
api/changelog.md and bumped API_FEATURE_LEVEL.

Tweaked by tabbott to add an `id` field in RealmPlayground objects
sent to clients, which is essential to sending the API request to
remove one.
2021-04-06 20:56:58 -07:00
Abhijeet Prasad Bodas 3bfcaa3968 mute user: Add backend infrastructure code.
Adds backend code for the mute users feature.
This is just infrastructure work (database
interactions, helpers, tests, events, API docs
etc) and does not involve any behavioral/semantic
aspects of muted users.

Adds POST and DELETE endpoints, to keep the
URL scheme mostly consistent in terms of `users/me`.

TODOs:
1. Add tests for exporting `zulip_muteduser` database table.
2. Add dedicated methods to python-zulip-api to be used
   in place of the current `client.call_endpoint` implementation.
2021-04-06 18:44:08 -07:00
Anders Kaseorg 3c41db7f1a dependencies: Upgrade to webpack-bundle-tracker 1.0.0-alpha.1.
This also seems unmaintained, but is, at least, released.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-04-06 09:31:35 -07:00
Aman Agrawal 5e83965e80 giphy: Use GIPHY web SDK to allow inserting GIFs in compose box.
We use GIPHY web SDK to create popover containing GIFs in a
grid format. Simply clicking on the GIFs will insert the GIF in the compose
box.

We add GIPHY logo to compose box action icons which opens the GIPHY
picker popover containing GIFs with "Powered by GIPHY"
attribution.
2021-04-05 15:04:49 -07:00
Sumanth V Rao e12f682e2e markdown: Include text & url in `topic_links` parameter of our API.
The linkifier code now includes both the shortened text and the expanded
URL, sorted by the order of the occurrence in a topic. This list is passed
back in the `topic_links` parameter of the /messages and the /events APIs.

topic_links earlier vs now:

earlier: ['https://www.google.com', 'https://github.com/zulip/zulip/32']

now: [{'url': 'https://www.google.com', 'text': 'https://www.google/com},
      {'url': 'https://github.com/zulip/zulip/32', 'text': '#32'}]

Similarly, the topic_links local echo logic in the frontend now returns
back an object.

Fixes: #17109.
2021-03-30 15:53:07 -07:00
Anders Kaseorg d55dc6f8f1 requirements: Upgrade python-zulip-api from Git.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-03-26 16:31:03 -07:00
shanukun bc2d58ad4a custom_profile_fields: Remove op field for the event.
* `op` (operation) field, added in f6fb88549f, was never intended for
`custom_profile_fields` event. This commit removes the `op` as it doesn't
have any use in the code.

* As a part of cleanup, this also eliminates the schema check warnings
for `custom_profile_fields` event, mentioned in #17568.
2021-03-26 16:28:33 -07:00
Anders Kaseorg 4703256c91 requirements: Upgrade Python requirements.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-03-26 08:51:16 -07:00
Anders Kaseorg 2831f9b60e install-shfmt: Upgrade shfmt from 3.2.2 to 3.2.4.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-03-26 08:51:16 -07:00
Anders Kaseorg e62b0d6b5f puppeteer_tests: Translate logged traces and errors with source maps.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-03-25 02:05:19 -07:00
Tim Abbott f121e40848 message: Record whether unread_msgs data is truncated.
This is preparatory work for investigating reports of missing unread
messages.

It's a little surprising that not test failed after adding the code
without API documentation.

Co-Author-By: Tushar Upadhyay (tushar912).
2021-03-21 19:48:13 -07:00
Anders Kaseorg 6364e1b5f3 requirements: Upgrade talon fork to 1.4.8.
https://github.com/mailgun/talon/pull/200

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-03-18 17:10:18 -07:00
Tushar912 55de66f944 api: Support user_id in get_user_presence_endpoint.
This is part of our general process of replacing emails, which are not
static with time, with user_ids when referring to users in the API.

We still keep the `email` reference option, since it can be useful for
linking third-party applications to Zulip on an intranet that might
have a user's corporate email handy and not want to do the extra round
trip to lookup the user.

The name of the parameter, user_id_or_email, was chosen to to make it
clear that the default/preferred option is user_id.

Fixes #14304.
2021-03-11 20:14:04 -08:00
Aman Agrawal e587c029f6 display_settings: Allow user to set default_view.
TextField is used to allow users to set long stream + topic narrow
names in the urls.

We currently restrict users to only set "all_messages" and
"recent_topics" as narrows.

This commit achieves 3 things:
* Removes recent topics as the default view which loads when
  hash is empty.
* Loads default_view when hash is empty.
* Loads default_view on pressing escape key when it is unhandled by
  other present UI elements.

NOTE: After this commit loading zulip with an empty hash will
automatically set hash to default_view.  Ideally, we'd just display
the default view without a hash, but that involves extra complexity.

One exception is when user is trying to load an overlay directly,
i.e. zulip is loaded with an overlay hash. In this case,
we render recent topics is background irrespective of default_view.

We consider this last detail to be a bug not important enough to block
adding this setting.
2021-03-11 18:09:08 -08:00
sahil839 9a432b0c3b events: Remove name field from update subscription events.
This commit removes name field from update subscription
events, as it is not used by any of the clients, and use
stream_id in the events code instead.
2021-03-07 22:03:24 -08:00
Steve Howell 30c7108955 zjsunit: Remove rewiremock dependency.
We now just use a module._load hook to inject
stubs into our code.

For conversion purposes I temporarily maintain
the API of rewiremock, apart from the enable/disable
pieces, but I will make a better wrapper in an
upcoming commit.

We can detect when rewiremock is called after
zrequire now, and I fix all the violations in
this commit, mostly by using override.

We can also detect when a mock is needlessly
created, and I fix all the violations in this
commit.

The one minor nuisance that this commit introduces
is that you can only stub out modules in the Zulip
source tree, which is now static/js.  This should
not really be a problem--there are usually better
techniques to deal with third party depenencies.
In the prior commit I show a typical workaround,
which is to create a one-line wrapper in your
test code.  It's often the case that you can simply
use override(), as well.

In passing I kill off `reset_modules`, and I
eliminated the second argument to zrequire,
which dates back to pre-es6 days.
2021-03-06 11:10:57 -05:00
Alex Vandiver ea2e5de54f requirements: Downgrade jedi dependency for ipython compatibility.
ipython < 7.20.0 is incompatible with jedi >= 0.18.0; it fails to
tab-complete in `./manage.py shell`, as described in
ipython/ipython#12740.

We cannot bump the ipython dependency because ipython 7.20.0 requires
Python 3.7, and we must support Python 3.6 due to Ubuntu 18.04
support.  Our only solution is thus to cap the version of `jedi` to
the last one before its API changed.
2021-03-04 18:08:45 -08:00
Anders Kaseorg 4bbfac8aa9 requirements: Upgrade Python requirements.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-03-03 21:46:42 -08:00
Anders Kaseorg b8e5e34c01 install-shfmt: Upgrade shfmt from 3.2.1 to 3.2.2.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-03-03 21:46:42 -08:00
Anders Kaseorg 25bb98dcf5 install-node: Upgrade Node.js from 14.15.1 to 14.16.0.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-03-03 21:46:42 -08:00
Anders Kaseorg 9613c66560 dependencies: Upgrade JavaScript dependencies.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-03-03 21:46:42 -08:00
Anders Kaseorg cce3257f2c zjsunit: Remove babel-plugin-lazy-require.
There are no more require() calls in tested modules.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-03-02 17:06:35 -08:00
sahil839 b53c773987 events: Remove email field from update subscription events.
This commit removes email field from update subscription
events, as email field is of no use in this case.
2021-03-01 14:52:06 -08:00
Anders Kaseorg 4df3fef88f zjsunit: Add babel-plugin-lazy-require.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-02-28 14:23:00 -08:00
Anders Kaseorg 3af6724438 dependencies: Upgrade clean-css 5.0.1 to 5.1.0.
This fixes a minification issue affecting our orange user circles.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-02-25 20:08:36 -08:00
Anders Kaseorg ba0ee6ddfa dependencies: Upgrade Source Sans Pro font to Source Sans 3.
https://blog.adobe.com/en/2020/11/30/whats-new-in-source-sans-3.html

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-02-23 14:55:33 -08:00
Anders Kaseorg cdff4cfe53 dependencies: Add ts-node.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-02-22 16:03:10 -08:00
Anders Kaseorg 3869727655 dependencies: Upgrade puppeteer from 7.0.4 to 7.1.0.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-02-22 16:03:10 -08:00
Tushar912 dfafdda9b3 api: Add REST API endpoint for looking up a user by email address.
Add new rest api endpoint GET users/{email} for looking up a user by
email, which is useful especially for corporate API applications that
might already have a user's email address.

Fixes #14302.
2021-02-15 17:38:33 -08:00
Anders Kaseorg f21fbea7c7 lint: Replace (most of) pycodestyle with Black.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-02-12 13:11:19 -08:00
Anders Kaseorg 6e4c3e41dc python: Normalize quotes with Black.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-02-12 13:11:19 -08:00
Anders Kaseorg 11741543da python: Reformat with Black, except quotes.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-02-12 13:11:19 -08:00
Anders Kaseorg 2024859d74 dependencies: Upgrade JavaScript dependencies.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-02-10 15:55:49 -08:00
Anders Kaseorg f38d68e110 requirements: Upgrade Python requirements.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-02-10 10:15:50 -08:00
Anders Kaseorg 9896782fd1 dependencies: Remove XDate.
It’s even more unmaintained than Moment and doesn’t add any
functionality we don’t already have.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-02-07 06:41:33 -05:00
aryanshridhar f92f99d92d dependencies: Replace moment.js with date-fns.
Replaced methods/functions of moment.js with date-fns library.
The motive was to replace it with a smaller frontend timezone library.

Date-fns ~ 11.51 kb
moment.js ~ 217.87 kb

Some of the format strings change because date-fns encodes them
differently from how moment did.

Fixes #16373.
2021-02-05 11:04:32 -08:00
Anders Kaseorg aa650a4c88 js: Escape strings interpolated into CSS selectors with CSS.escape.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-02-04 11:00:06 -08:00
Aman Agrawal 58a4469896 version: Correct API feature level.
API feature level should have been increased along with #17010
which increased the community topic editing time.
2021-02-03 10:20:59 -05:00
Anders Kaseorg 049d140557 requirements: Upgrade moto to 1.3.17.dev230.
This unblocks the upgrade to responses 0.12.1.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-01-29 12:02:43 -08:00
Anders Kaseorg d33217f2ef requirements: Remove unused httplib2 requirement.
It’s unused since commit 8dd95bd057
(#13718).

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-01-26 16:33:32 -08:00
Anders Kaseorg a7bd1f8049 requirements: Upgrade Python requirements. 2021-01-26 13:27:50 -08:00
Mateusz Mandera 1432067959 dependencies: Upgrade to Django 3.1.
https://docs.djangoproject.com/en/3.1/releases/3.1/

- django.contrib.postgres.fields.JSONField is deprecated and should be
  replaced with models.JSONField
-  The internals of the implementation in the postgresql backend have
   changed a bit in
   f48f671223
   and thus we need to make an ugly tweak in test_runner.
- app_directories.Loader.get_dirs() now returns a list of PosixPath so
  we need to make a small tweak in TwoFactorLoader for that (PosixPath
  is not iterable)

Fixes #16010.
2021-01-26 10:20:00 -08:00
Anders Kaseorg 2db2f2d6cd dependencies: Upgrade JavaScript dependencies.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-01-25 14:53:19 -08:00
Steve Howell d9740045a5 refactor: Eliminate checks in build_stream_dict_for_sub.
We eliminate some redundant checks.

We also consistently provide a `subscribers` field
in our stream data with `[]`, even if our users
can't access subscribers.  We therefore bump
the API version and tweak the docs.  (See further
down for a detailed justification of the change.)

Even though it is sometimes fine to have redundant code
that is defensive in nature, some upcoming changes are gonna
move subscriber-related logic out of build_stream_dict_for_sub
for certain codepaths as part of our effort to streamline
the payload for subscribers within page_params.

So we can't rely on the code that I removed here
inside of build_stream_dict_for_sub.

Anyway, it makes more sense to do these checks explicitly
in the validate function.

The code in build_stream_dict_for_sub was almost effectively
a noop, since the validation function was already preventing
us from getting subscriber info.  The only difference it
made was sometimes converting `[]` to `None`, and then
subsequently omitting the subscribers field.

Neither ZT nor the webapp make any distinction between
`[]` or <missing key> for the `subscribers` data in
`page_params`.

The webapp has had this code for a long time (and now
equivalent code elsewhere in this PR):

    if (!Object.prototype.hasOwnProperty.call(sub, "subscribers")) {
        sub.subscribers = new LazySet([]);
    }

The webapp calculates access based on booleans, anyway:

    sub.can_access_subscribers =
        page_params.is_admin || sub.subscribed ||
        (!page_params.is_guest && !sub.invite_only);

And ZT would choke if `subscribers` were missing, except that
it never gets to the relevant code due to other checks:

    def get_other_subscribers_in_stream(<snip>):
        assert stream_id is not None or stream_name is not None

        if stream_id:
            assert self.is_user_subscribed_to_stream(stream_id)

            return [sub
                    for sub in self.stream_dict[stream_id]['subscribers']
                    if sub != self.user_id]
        else:
            return [sub
                    for _, stream in self.stream_dict.items()
                    for sub in stream['subscribers']
                    if stream['name'] == stream_name
                    if sub != self.user_id]

You could make a semantic argument that we should prefer
<missing key> to `[]` when subscribers aren't even available, but
we have precedent from the way that `bulk_get_subscriber_user_ids`
has traditionally populated its result:

    result: Dict[int, List[int]] =
        {stream["id"]: [] for stream in stream_dicts}

If we changed `stream_dicts` to `target_stream_dicts` we
would faciliate a move toward `None`, but it would just cause
headaches for other server code as well as the frontends
(which, to reiterate, already prefer the empty array
for convenience).
2021-01-21 15:04:07 -08:00
Aman Agrawal c685d36821 hipchat_import: Remove tool from codebase.
Remove functions and scripts used by HipChat import tool and
those which will no longer be required in future.
2020-12-23 08:28:49 -08:00
Anders Kaseorg 6c989550ba dependencies: Upgrade JavaScript dependencies.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-12-22 16:14:28 -08:00
Mateusz Mandera 160cc5120a api: Require can_create_users permission to create users via API.
Allowing any admins to create arbitrary users is not ideal because it
can lead to abuse issues.  We should require something stronger that
requires the server operator's approval and thus we add a new
can_create_users permission.
2020-12-21 13:20:21 -08:00
Sutou Kouhei 0d3f9fc855 install: Use PGroonga packages built for PostgreSQL packages by PGDG
Because we always use PostgreSQL packages by PGDG since Zulip 3.0.

Fixes #16058.
2020-12-18 15:38:21 -08:00
ryanreh99 10559c05ee build_pygments_data: Include zulip specific data in langs.
This fixes a bug where the typeahead did not include the
zulip specific langs (such a `quote`, `spoiler` and `math`)
as these weren't passed to the typeahead's source.

Introduced in af64c52166.

Fixes #16862.
2020-12-15 16:41:19 -08:00
Anders Kaseorg 2ab0b3d4fc validator: Reject ISO 8601 dates missing leading zeros.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-12-15 16:36:50 -08:00
Anders Kaseorg b3aa44c914 install-shfmt: Upgrade shfmt from 3.1.2 to 3.2.1.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-12-09 23:07:40 -08:00
Anders Kaseorg 77fdac3579 install-node: Upgrade Node.js to 14.15.1 and nvm to 0.37.2.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-12-09 23:07:40 -08:00
Anders Kaseorg 984a5bb737 dependencies: Upgrade JavaScript dependencies.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-12-09 23:07:38 -08:00
Anders Kaseorg 362d4a6a8d requirements: Upgrade Python requirements.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-12-09 14:24:54 -08:00
Anders Kaseorg af64c52166 build_pygments_data: Build langs without incorrect mutation.
The previous loop generated broken entries for duplicated language
aliases:

"juttle": {"priority": {"priority": 0, "pretty_name": "Juttle"}, "pretty_name": "Juttle"}

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-11-16 18:51:37 -08:00
Anders Kaseorg 13e35bfa94 mypy: Use sqlalchemy-stubs.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-11-16 18:17:41 -08:00
Anders Kaseorg 1275613812 requirements: Upgrade mypy to 0.790.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-11-12 15:44:30 -08:00
Aman Agrawal b2841c7d02 dev_requirements: Upgrade zulint.
This tells users how autofix errors for linters which support it.
This is important since only way to fix prettier errors is
running lint with `--fix` which now the linter will gladly print
with the error.
2020-11-03 14:49:02 -08:00
Vishnu KS 95cd031274 version: Bump provision version for updating email templates. 2020-10-30 11:50:30 -07:00
Anders Kaseorg 7fbd75630f requirements: Upgrade Python requirements.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-10-28 23:39:15 -07:00
Vishnu KS fdea49742c apps: Use GitHub API for generating the web app download link. 2020-10-28 23:04:14 -07:00
Anders Kaseorg 23a289ecd5 install-node: Upgrade Node.js to 12.19.0 and Yarn to 1.22.10.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-10-28 11:45:02 -07:00
Steve Howell 0e7e4f1ad3 api feature level: Bump level for peer_add/peer_remove. 2020-10-26 07:18:08 -04:00
Anders Kaseorg 0f11e2a361 dependencies: Upgrade JavaScript dependencies.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-10-24 02:09:14 -07:00
Anders Kaseorg efa8dd3a47 compilemessages: Sort language list with Unicode Collation Algorithm.
Right now the list of languages in Display settings → Default language
is sorted in an unintuitive order due to the varying case conventions:

British English
Chinese (Taiwan)
Deutsch
English
Hindi
Indonesian (Indonesia)
Lietuviškai
Magyar
Malayalam
Nederlands
Português
Română
Tiếng Việt
Türkçe
català
español
français
galego
italiano
polski
suomi
svenska
česky
Русский
Українська
български
српски
فارسی
தமிழ்
日本語
简体中文
繁體中文
한국어

Fix the sort to use the locale-independent Unicode Collation
Algorithm:

British English
català
česky
Chinese (Taiwan)
Deutsch
English
español
français
galego
Hindi
Indonesian (Indonesia)
italiano
Lietuviškai
Magyar
Malayalam
Nederlands
polski
Português
Română
suomi
svenska
Tiếng Việt
Türkçe
български
Русский
српски
Українська
فارسی
தமிழ்
한국어
日本語
简体中文
繁體中文

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-10-21 15:22:18 -07:00
Anders Kaseorg 5c6202ca97 notifications: Use a dynamic SVG favicon for unread counts.
Closes #2304.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-10-21 07:43:00 -07:00
Anders Kaseorg 8152cd8566 requirements: Upgrade Python requirements.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-10-19 11:54:14 -07:00
Anders Kaseorg d81a93cdf3 requirements: Upgrade markdown to 3.3.1.
Upstream has slightly changed the whitespace around stashes.  Take
this opportunity to clean up the extra blank lines we were outputting.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-10-19 11:54:14 -07:00
Anders Kaseorg 449b16be59 dependencies: Use patched source-map with a performance fix.
https://github.com/mozilla/source-map/issues/370

This makes Webpack about 9% faster.  Before:

$ multitime -n9 -s0 node_modules/.bin/webpack --config-name=frontend --display=errors-only -p
===> multitime results
1: node_modules/.bin/webpack --config-name=frontend --display=errors-only -p
            Mean        Std.Dev.    Min         Median      Max
real        18.243      0.107       18.090      18.236      18.443
user        27.913      0.188       27.714      27.843      28.251
sys         2.028       0.043       1.933       2.039       2.074

After:

$ multitime -n9 -s0 node_modules/.bin/webpack --config-name=frontend --display=errors-only -p
===> multitime results
1: node_modules/.bin/webpack --config-name=frontend --display=errors-only -p
            Mean        Std.Dev.    Min         Median      Max
real        16.686      0.085       16.542      16.684      16.885
user        25.965      0.167       25.559      26.022      26.163
sys         1.965       0.064       1.807       1.998       2.010

Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2020-10-15 14:38:41 -07:00
Anders Kaseorg 2a4a775e2b requirements: Install PycURL for Thumbor.
Fixes “thumbor:WARNING pycurl usage is advised. It could not be loaded
properly. Verify install...”.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-10-14 10:52:32 -07:00
Anders Kaseorg 4fe06a141e eslint: Enable eslint-plugin-unicorn.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-10-07 16:00:33 -07:00
sahil839 78b98d8067 realm: Add wildcard_mention_policy setting.
We add a new wildcard_mention_policy setting to handle wildcard
mentions in large streams, with a wide range of policies available to
organizations.

We set the default to the safe option for preventing accidental spam:
only stream administrators being able to use wildcard mentions in
large streams.
2020-10-01 12:18:03 -07:00
Anders Kaseorg cfd93096b5 openapi: Remove yamole.
As explained in the previous commit, yamole preprocessed allOf with an
algorithm that is not standards compliant.  We replicate that
algorithm, but importantly, we only use it for our own code and not
for building the openapi_core RequestValidator.

This improves the time taken by OpenAPISpec().check_reload() from
1.69s to 0.53s, nearly all of which is inside
openapi_core.create_spec.

Closes #10484.  Significantly improves #16068.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-09-29 16:47:10 -07:00
Anders Kaseorg e4931c5bc4 people: Remove unorm polyfill.
We no longer support IE11.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-09-25 16:54:25 -07:00
Tim Abbott 089225ea36 version: Update latest desktop release version. 2020-09-25 13:00:46 -07:00
Anders Kaseorg 58808c2362 dependencies: Upgrade JavaScript dependencies.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-09-24 16:33:45 -07:00
Anders Kaseorg 3a39c6f67f dependencies: Downgrade simplebar to 5.2.1.
SimpleBar 6.0.0-beta.2 through -beta.6 are built with ES6 syntax (I
assume inadvertently: https://github.com/Grsmto/simplebar/issues/523),
and its latest tag has moved back to 5.2.1 anyway.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-09-24 16:13:47 -07:00
Alex Vandiver 4361ce1246 markdown: Use tlds package to keep updated list of TLDs.
Also remove a useage of "blacklist."
2020-09-21 21:03:29 -07:00
Sumanth V Rao 2b9f7916c5 api: Document markdown changes for code blocks and bump API_FEATURE_LEVEL. 2020-09-18 17:15:47 -07:00
Sumanth V Rao 4a2791e2a3 tools/build_pygments_data: Map pygments language aliases to lexer name.
We need this information in the frontend to:
    * Display the 'view in playground' option for locally echoed messages.
    * When we add a UI settings for realm admins to configure their
      playground choices, we'll need to use these canonicalized aliases
      for displaying the option.

Hence, this tweaks the tool which generates pygments_data.json to contain
the data we need.

Bumping major PROVISION_VERSION since folks need to provision in both
directions.

Tests amended.
2020-09-18 17:12:26 -07:00
Anders Kaseorg b7874ac82e install-node: Upgrade Node.js to 12.18.4 and Yarn to 1.22.5.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-09-15 16:33:28 -07:00
Anders Kaseorg 7c755f1394 dependencies: Upgrade JavaScript dependencies.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-09-15 16:33:28 -07:00
Anders Kaseorg e84c7fb09f requirements: Remove django-cookies-samesite.
Its functionality was added to Django upstream in 2.1.  Also remove
the SESSION_COOKIE_SAMESITE = 'Lax' setting since it’s the default.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-09-14 17:23:56 -07:00
Anders Kaseorg fd59db292f requirements: Upgrade Python requirements.
Fixes #16335.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-09-13 20:59:09 -07:00
Anders Kaseorg dfab09b17d markdown: Replace hyperlink requirement with urllib.parse.
The previous code only worked by accident and hyperlink 20.0.0 breaks
it.

>>> hyperlink.parse("example.com").replace(scheme="https")
DecodedURL(url=URL.from_text('https:example.com'))

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-09-13 15:37:28 -07:00
Anders Kaseorg d9860d40a6 dependencies: Upgrade JavaScript dependencies.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-09-02 19:34:37 -07:00
Priyank Patel 5c53c44ee5 dependencies: Upgrade babel-plugin-rewire-ts to v1.4.0.
This version keeps the internal and external state of the ES6 module
in sync so its state is similar to a CommonJS module.  Since we don't
need to ask rewire for the internal state of the module, the tests
will be same as they were before the TS/ES6/remove from window
migrations.
2020-09-01 19:55:58 -07:00
Anders Kaseorg b9184eee65 eslint: Use eslint-import-resolver-webpack.
This uses our Webpack configuration to more accurately predict how our
imports will actually behave; for example, it automatically resolves
the .ts extension.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-09-01 16:43:02 -07:00
Priyank Patel 05f3d552d3 yarn.lock: Update lockfile after phantomjs-prebuild removal.
I think this needs an update because 2f5f5d7749,
which removes phantomjs-prebuilt didn't update the yarn.lock.
2020-08-31 11:21:15 -07:00
Priyank Patel c1ba648b4f dependencies: Add xvfb dependency.
The node package allow use to control xvfb apt package in puppeteer
tests. This help us create a fake display so we can run puppeteer in
headful (headless: false) mode, which is required to use the chrome
extension desktop capture API.
2020-08-30 23:38:52 -07:00
Tim Abbott 2f5f5d7749 dependencies: Remove PhantomJS from development environment. 2020-08-30 17:16:02 -07:00
Dinesh 2d22d88700 casper: Remove few traces of casper.
Now that all casper tests have been migrated to
puppeteer, there's no need for having casper
related things.

Removed the casperjs package and removed/replaced
casper in few places with puppeteer.

Only removed few of them which I'm confident
about. Also didn't make any changes in docs
as it would be easier to remove them while
adding puppeteer docs.
2020-08-30 17:16:02 -07:00
Aman 46a86e218e requirements: Upgrade zulint.
This upgrade allows us to exclude a line multiple times for a
pattern.
2020-08-24 16:10:10 -07:00
orientor 372e010dbb events: Add `op` field to `update_message_flags` events.
`update_message_flags` events used `operation` instead of `op`, the
latter being the standard field used in other events. So add `op`
field to `update_message_flags` and mark `operation` as deprecated,
so that it can be removed later.
2020-08-24 12:42:03 -07:00
sahil839 f046c9c58a streams: Add role field to Subscription objects passed to clients.
This commit adds "role" field to the Subscription objects passed to
clients.  This is important preparation for being able to work on the
frontend for this feature.
2020-08-14 16:33:11 -07:00
Anders Kaseorg 4093514b93 dependencies: Upgrade babel-eslint to @babel/eslint-parser.
https://babeljs.io/blog/2020/07/13/the-state-of-babel-eslint

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-08-12 18:37:25 -07:00
Anders Kaseorg bc1dfb4b10 dependencies: Upgrade JavaScript dependencies.
css-loader@^4 is also available, but we can’t use it yet because of
https://github.com/webpack-contrib/css-loader/issues/1164.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-08-12 18:37:25 -07:00
Anders Kaseorg 0f608176ad install-node: Upgrade Node.js from 12.18.2 to 12.18.3.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-08-12 18:37:25 -07:00
Kartik Srivastava 63173d5554 api: Return 'user_id' in 'POST /users' response.
This adds 'user_id' to the simple success response for 'POST /users'
api endpoint, to make it convenient for API clients to get details
about users they just created.  Appropriate changes have been made in
the docs and test_users.py.

Fixes #16072.
2020-08-11 16:40:12 -07:00
Anders Kaseorg 61d0417e75 python: Replace ujson with orjson.
Fixes #6507.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-08-11 10:55:12 -07:00
Anders Kaseorg dbdf67301b memcached: Switch from pylibmc to python-binary-memcached.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-08-06 12:51:14 -07:00
Anders Kaseorg ca2dc639c4 styles: Enable most stylelint-config-standard rules.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-08-04 16:34:19 -07:00
Anders Kaseorg 7c7f030d54 styles: Enable most stylelint-config-recommended rules.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-08-04 16:34:19 -07:00
Anders Kaseorg c6ed5c81de styles: Format CSS with Prettier.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-08-04 16:34:19 -07:00
Manav Mehta 69e0625a09 version: Update version for Zulip Desktop v5.4.0 release.
Update the zulip-desktop download link for the latest v5.4.0 release
This commit only updates zulipchat.com/apps/ links.
2020-07-31 10:36:12 -07:00
Anders Kaseorg b06794e9f7 requirements: Downgrade pip to 20.1.1 due to pip-tools incompatibility.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-07-30 19:16:04 -07:00
Anders Kaseorg d35b5027bb semgrep: Upgrade semgrep to 0.17.0.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-07-30 15:12:22 -07:00
Anders Kaseorg d22ec342e2 requirements: Upgrade Python requirements.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-07-30 15:12:22 -07:00
Gittenburg 6081e0228b emoji: Make 😛 have open eyes.
😛 should be the most general version, which is the one
with open eyes. Other apps do the same and it also means that :P, which
is converted to 😛 is rendered like the emoticon.

Fixes #15970.
2020-07-29 11:18:43 -07:00
Dinesh c15d7e3202 requirements: Update social-auth-core to latest version.
Uses git release as this version 3.4.0 is not released to pypi.
This is required for removing some overriden functions of
apple auth backend class AppleAuthBackend.

With the update we also make following changes:

* Fix full name being populated as "None None".
c5c74f27dd that's included in update assigns first_name and last_name
to None when no name is provided by apple. Due to this our
code is filling return_data['full_name'] to 'None None'.
This commit fixes it by making first and last name strings empty.

* Remove decode_id_token override.
Python social auth merged the PR we sent including the changes
we made to decode_id_token function. So, now there is no
necessity for the override.

* Add _AUDIENCE setting in computed_settings.py.
`decode_id_token` is dependent on this setting.
2020-07-28 17:12:49 -07:00
Anders Kaseorg d2520cd7e0 js: Replace underscore with lodash and remove it from globals.
Tweaked by tabbott to bump PROVISION_VERSION.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-07-26 16:12:06 -07:00
Alex Vandiver 473be34d90 sentry: Add sentry_sdk dependency. 2020-07-24 12:52:15 -07:00
Anders Kaseorg 0771e7fc59 eslint: Add import plugin.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-07-24 09:42:56 -07:00
Anders Kaseorg c9ce77c28b eslint: Use babel-eslint parser.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-07-24 09:42:56 -07:00
Anders Kaseorg 79d20c7c21 version: Bump PROVISION_VERSION for :slight_smile: rename.
Commit 45e19dd6b9 (#15627) required an
emoji rebuild.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-07-21 18:25:18 -07:00
Anders Kaseorg b3da022bdf install-node: Upgrade Node.js to 12.18.2.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-07-20 10:56:31 -07:00
Anders Kaseorg aa16208fd8 dependencies: Upgrade JavaScript dependencies.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-07-20 10:56:31 -07:00
Anders Kaseorg bd45216c21 eslint: Disable syntax rules handled by Prettier.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-07-17 14:31:25 -07:00
Steve Howell c60f4236a9 api: Do not require short_name to create user.
When you post to /json/users, we no longer
require or look at the short_name parameter,
since we don't use it in any meaningful way.

An upcoming commit will eliminate it from the
database.
2020-07-17 11:15:15 -07:00
Steve Howell b375581f58 api: Remove (sender_)short_name from message APIs. 2020-07-17 11:15:14 -07:00
Tim Abbott 23e0ea5e32 version: Update version following 3.0 release. 2020-07-16 02:25:45 -07:00
Tim Abbott d74664a603 Release Zulip Server 3.0. 2020-07-16 02:13:43 -07:00
Anders Kaseorg 56e8cca025 lint: Add Prettier as a linter for YAML files.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-07-14 16:25:31 -07:00
Anders Kaseorg f364414cb9 install-semgrep: Upgrade semgrep to 0.14.0.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-07-14 00:42:07 -07:00
Anders Kaseorg 79b1bf56ab requirements: Upgrade zulint.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-07-14 00:41:20 -07:00
Tim Abbott 6b1f1e74a3 version: Update version string following 3.0-rc2. 2020-07-14 00:29:59 -07:00
Tim Abbott ac0a305f72 Release Zulip Server 3.0-rc2. 2020-07-14 00:24:35 -07:00
Anders Kaseorg 7e8280ef50 requirements: Upgrade Python requirements.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-07-10 00:14:43 -07:00
Rohitt Vashishtha 912e372c4e markdown: Remove !avatar() and !gravatar() syntax.
This particular commit has been a long time coming. For reference,
!avatar(email) was an undocumented syntax that simply rendered an
inline 50px avatar for a user in a message, essentially allowing
you to create a user pill like:

`!avatar(alice@example.com) Alice: hey!`

---

Reimplementation

If we decide to reimplement this or a similar feature in the future,
we could use something like `<avatar:userid>` syntax which is more
in line with creating links in markdown. Even then, it would not be
a good idea to add this instead of supporting inline images directly.

Since any usecases of such a syntax are in automation, we do not need
to make it userfriendly and something like the following is a better
implementation that doesn't need a custom syntax:

`![avatar for Alice](/avatar/1234?s=50) Alice: hey!`

---

History

We initially added this syntax back in 2012 and it was 'deprecated'
from the get go. Here's what the original commit had to say about
the new syntax:

> We'll use this internally for the commit bot.  We might eventually
> disable it for external users.

We eventually did start using this for our github integrations in 2013
but since then, those integrations have been neglected in favor of
our GitHub webhooks which do not use this syntax.

When we copied `!gravatar` to add the `!avatar` syntax, we also noted
that we want to deprecate the `!gravatar` syntax entirely - in 2013!

Since then, we haven't advertised either of these syntaxes anywhere
in our docs, and the only two places where this syntax remains is
our game bots that could easily do without these, and the git commit
integration that we have deprecated anyway.

We do not have any evidence of someone asking about this syntax on
chat.zulip.org when developing an integration and rightfully so- only
the people who work on Zulip (and specifically, markdown) are likely
to stumble upon it and try it out.

This is also the only peice of code due to which we had to look up
emails -> userid mapping in our backend markdown. By removing this,
we entirely remove the backend markdown's dependency on user emails
to render messages.

---

Relevant commits:

- Oct 2012, Initial commit        c31462c278
- Nov 2013, Update commit bot     968c393826
- Nov 2013, Add avatar syntax     761c0a0266
- Sep 2017, Avoid email use       c3032a7fe8
- Apr 2019, Remove from webhook   674fcfcce1
2020-07-07 10:39:44 -07:00
Aman Agrawal a486872a8e requirements: Upgrade Thumbor to 7.0.0a5 on Python 3.
Co-authored-by: Anders Kaseorg <anders@zulip.com>
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-07-06 16:09:53 -07:00
Anders Kaseorg 82f04810d5 requirements: Upgrade Python requirements.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-07-03 19:47:18 -07:00
Tim Abbott 3177378b7d version: Update version strings following 3.0-rc1. 2020-07-01 17:52:39 -07:00
Tim Abbott 4209d7faf0 Release Zulip Server 3.0-rc1. 2020-07-01 17:47:24 -07:00
Anders Kaseorg 8a54dc43ce lint: Upgrade zulint and remove run-isort wrapper.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-07-01 12:53:21 -07:00
orientor 529da34513 openapi: Use third-party validator for schema validation.
Our previous OpenAPI schema validator that we implemented ourselves
was useful training wheels for our understanding OpenAPI properly, and
was mostly correct.  But given that we've finally reached the point
where our OpenAPI file accurately describes the API, it makes sense to
switch to use an official OpenAPI validator.  We lose some ability to
do exclude rules for particular elements, but those were primarily
important for us when we had a lot of them.

As part of this change, we need to add `additionalProperties: false`
for all of our dictonaries/objects where we've documented every
parameter; otherwise the OpenAPI schema checker won't know that we
expect every parameter to be documented.
2020-07-01 11:21:41 -07:00
Manav Mehta 28166c14e8 version: Update for Zulip Desktop v5.1.0 release.
Update the zulip-desktop download link for the latest v5.3.0 release
This only updates zulipchat.com/apps/ downloads link.
2020-06-28 11:50:03 -07:00
Steve Howell 69be97e365 pointer: Remove pointer from API and page_params.
There is still some miscellaneous cleanup that
has to happen for things like analytics queries
and dead code in node tests, but this should
remove the main use of pointers in the backend.

(We will also still need to drop the DB field.)
2020-06-27 16:44:38 -07:00
Anders Kaseorg b554afaeaa requirements: Upgrade isort.
The isort author accidentally fixed the performance regression in his
latest commit.  Quickly upgrade isort before he notices his mistake.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-06-26 22:50:01 -07:00
sahil839 6eb8442a59 invites: Send user_id of the referrer instead of email in invites dict.
We send user_id of the referrer instead of email in the invites dict.
Sending user_ids is more robust, as those are an immutable reference
to a user, rather than something that can change with time.

Updates to the webapp UI to display the inviters for more convenient
inspection will come in a future commit.
2020-06-26 16:57:50 -07:00
Tim Abbott 19d48d0667 docs: Update changelog for 2.1.7 release. 2020-06-25 17:17:42 -07:00
Anders Kaseorg ebb2efa664 requirements: Upgrade Python requirements.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-06-25 13:14:37 -07:00
Eeshan Garg 267cb8999b requirements: Upgrade fakeldap dependency to the latest version. 2020-06-24 20:14:32 -07:00
Tim Abbott e46bbf18eb docs: Change next planned major release to 3.0.
After some discussion, everyone seems to agree that 3.0 is the more
appropriate version number for our next major release.  This updates
our documentation to reflect that we'll be using 3.0 as our next major
release.
2020-06-24 16:27:27 -07:00
MariaGkoulta b10f156250 settings: Add automatic theme detection feature.
With this implementation of the feature of the automatic theme
detection, we make the following changes in the backend, frontend and
documentation.

This replaces the previous night_mode boolean with an enum, with the
default value being to use the prefers-color-scheme feature of the
operating system to determine which theme to use.

Fixes: #14451.

Co-authored-by: @kPerikou <44238834+kPerikou@users.noreply.github.com>
2020-06-21 01:09:01 -07:00
sahil839 63389b3bd3 invite: Add option to invite user as an organization owner.
We can now invite new users as realm owners. We restrict only
owners to invite new users as owners both for single invite
and multiuse invite link. Also, only owners can revoke or resend
owner invitations.
2020-06-20 23:04:59 -07:00
Hashir Sarwar 5200598a31 events: Don't send avatar URLs of long term idle users.
This adds a new client_capability that clients such as the mobile apps
can use to avoid unreasonable network bandwidth consumed sending
avatar URLs in organizations with 10,000s of users.

Clients don't strictly need this data, as they can always use the
/avatar/{user_id} endpoint to fetch the avatar if desired.

This will be more efficient especially for realms with
10,000+ users because the avatar URLs would increase the
payload size significantly and cost us more bandwidth.

Fixes #15287.
2020-06-18 21:35:16 -07:00
sahil839 c488a35f10 streams: Add API for changing stream-level message_retention_days.
This commit adds backend support for setting message_retention_days
while creating streams and updating it for an existing stream. We only
allow organization owners to set/update it for a stream.

'message_retention_days' field for a stream existed previously also, but
there was no way to set it while creating streams or update it for an
exisiting streams using any endpoint.
2020-06-18 17:00:15 -07:00
Tim Abbott 1094e533bf docs: Document change in time mention markup. 2020-06-18 14:21:21 -07:00
Tim Abbott a0eb26951c docs: Update changelog for Zulip Server 2.1.6 release. 2020-06-17 00:42:28 -07:00
Tim Abbott 7872b197c2 docs: Update changelog for 2.1.5 release. 2020-06-16 23:35:39 -07:00
sahil839 791e5de5de api: Remove is_old_stream property from the stream objects.
This commit removes is_old_stream property from the stream objects
returned by the API. This property was unnecessary and is essentially
equivalent to 'stream_weekly_traffic != null'.

We compute sub.is_old_stream in stream_data.update_calculated_fields
in frontend code and it is used to check whether we have a non-null
stream_weekly_traffic or not.

Fixes #15181.
2020-06-16 10:26:33 -07:00
Aman Agrawal cda7b2f539 deletion: Add support for bulk message deletion events.
This is designed to have no user-facing change unless the client
declares bulk_message_deletion in its client_capabilities.

Clients that do so will receive a single bulk event for bulk deletions
of messages within a single conversation (topic or PM thread).

Backend implementation of #15285.
2020-06-14 22:34:00 -07:00
Anders Kaseorg 59e3c1f5e4 install-semgrep: Upgrade semgrep to 0.10.1.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-06-11 18:23:17 -07:00
Anders Kaseorg b666aef2d3 requirements: Upgrade isort to 5.0.0 prerelease from Git.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-06-11 16:05:58 -07:00
Anders Kaseorg 03e147d5e1 python: Replace NamedTuple with dataclass.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-06-11 15:19:31 -07:00
Vishnu KS bc57d9b611 i18n: Don't hardcode zulip.com URLs in translation tags. 2020-06-11 11:09:42 -07:00
Kartik Srivastava 8c39ddfd28 api: Add GET /users/{user_id}/subscription/{stream_id} endpoint.
This new endpoint returns a 'user' dictionary which, as of now,
contains a single key 'is_subscribed' with a boolean value that
represents whether the user with the given 'user_id' is subscribed
to the stream with the given 'stream_id'.

Fixes #14966.
2020-06-10 17:59:14 -07:00
Steve Howell d5cc29755e typing: Accept only user_ids in typing endpoint.
The only clients that should use the typing
indicators endpoint are our internal clients,
and they should send a JSON-formatted list
of user_ids.

We now enforce this, which removes some
complexity surrounding legacy ways of sending
users, such as emails and comma-delimited
strings of user_ids.

There may be a very tiny number of mobile
clients that still use the old emails API.

This won't have any user-facing effect on
the mobile users themselves, but if you type
a message to your friend on an old mobile
app, the friend will no longer see typing
indicators.

Also, the mobile team may see some errors
in their Sentry logs from the server rejecting
posts from the old mobile clients.

The error messages we report here are a bit
more generic, since we now just use REQ
to do validation with this code:

    validator=check_list(check_int)

This also allows us to remove a test hack
related to the API documentation.  (We changed
the docs to reflect the modern API in an
earlier commit, but the tests couldn't be
fixed while we still had the more complex
semantics for the "to" parameter.)
2020-06-10 17:50:32 -07:00
sahil839 7de23b8b5c users: Remove short_name and client_id from get_profile_backend.
This commit removes short_name and client_id fields from the user
objects returned by get_profile_backend because neither of them
had a purpose.

* short_name hasn't been present anywhere else in the Zulip API for
  several years, and isn't set through any coherent algorithm.
* client_id was a forgotten 2013-era predecessor to the queue_id field
  returned by the register_event_queue process.

The combination of these changes gets us close to having `get_profile`
have the exact same format as other endpoints fetching a user object.
2020-06-08 17:01:08 -07:00
Anders Kaseorg cf60493f8e install-shellcheck: Upgrade ShellCheck to 0.7.1.
ShellCheck binary releases have moved to GitHub:

https://github.com/koalaman/shellcheck/issues/1871

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-06-07 11:06:57 -07:00
Anders Kaseorg 0f63753926 install-node: Upgrade Node.js to 12.18.0.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-06-07 11:06:57 -07:00
Anders Kaseorg be8686474b dependencies: Upgrade JavaScript dependencies.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-06-07 11:06:57 -07:00
Anders Kaseorg 02c670f5a3 requirements: Bump zulint.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-06-06 18:05:25 -07:00
Anders Kaseorg 4a4692f817 requirements: Upgrade Python requirements.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-06-05 20:05:18 -07:00
Aman Agrawal a096f34cab move_topic_to_stream: Add option to disable breadcrumb messages.
Option to disable breadcrumb messages were given in both message edit
form and topic edit stream popover.

User now has the option to select which stream to send the notification
of stream edit of a topic via checkboxes in the UI.
2020-06-05 12:28:51 -07:00
Anders Kaseorg 5ad33c9d62 install-semgrep: Upgrade semgrep to 0.9.0.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-06-05 09:35:11 -07:00
Anders Kaseorg 789e612c70 requirements: Update Python requirements.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-06-03 17:23:20 -07:00
Tim Abbott 7dbdfe9a97 models: Remove database columns from old Zoom integration. 2020-06-03 16:39:12 -07:00
Anders Kaseorg 4d04fa3118 compose: Rewrite Zoom video call integration to use OAuth.
This reimplements our Zoom video call integration to use an OAuth
application.  In addition to providing a cleaner setup experience,
especially on zulipchat.com where the server administrators can have
done the app registration already, it also fixes the limitation of the
previous integration that it could only have one call active at a time
when set up with typical Zoom API keys.

Fixes #11672.

Co-authored-by: Marco Burstein <marco@marco.how>
Co-authored-by: Tim Abbott <tabbott@zulipchat.com>
Signed-off-by: Anders Kaseorg <andersk@mit.edu>
2020-06-03 16:39:12 -07:00
Puneeth Chaganti d0f8744d74 provision: Switch to fonts-freefont-ttf from fonts-liberation.
Some UTF-8 characters (★ for example) are not displayed correctly, with
fonts-liberation. Puppeteer recommends[1] installing fonts-freefont-ttf in
their docs on running Puppeteer in docker.

Provisioning forward is sufficient. There's no need to remove the
new font and replace it with the old font, I think.

[1]: https://github.com/puppeteer/puppeteer/blob/master/docs/troubleshooting.md#running-puppeteer-in-docker
2020-06-01 18:03:25 -07:00
sahil839 9ef1c5b1a6 users: Add is_owner field to user objects returned by get endpoints.
This commit adds 'is_owner' field to the user object returned by
'/users', 'users/{user_id}', and '/users/me' endpoints.
2020-06-01 15:33:51 -07:00
sahil839 2fe8971204 populate_db: Change role of 'desdemona' to realm owner.
Modified by tabbott to update PROVISION_VERSION.
2020-06-01 15:22:27 -07:00
sahil9001 328caf1ad5 api: Simplify format of realm_emoji author data.
There's no reason to send data beyond the user `id` of the uploader,
and reason not to, as the previous model was both awkward when
`author=None` and resulted in unecessary parsing complexity for
clients.

Modified by tabbott to add the frontend changes and API documentation.

Fixes #15115.
2020-05-31 17:44:50 -07:00
Kartik Srivastava 7f9207c6b7 dependencies: Install zulip-js npm package. 2020-05-13 10:25:10 -07:00
clarammdantas 7e9024a39c popovers.js: Add version to user avatar request.
When a user changes its avatar image, the user's avatar in popovers
wasn't being correctly updated, because of browser caching of the
avatar image.  We added a version on the request to get the image in
the same format we use elsewhere, so the browser knows when to use the
cached image or to make a new request to the server.

Edited by Tim to preserve/fix sort orders in some tests, and update
zulip_feature_level.

Fixes: #14290
2020-05-12 11:09:01 -07:00
Steve Howell 9137726e38 bots: Remove `owner` (email) from bot-related payloads.
We remove the `owner` field from `page_params/realm_bots`
and bot-related events.

In the recent commit 155f6da8ba
we added `owner_id`, which we now use everywhere we need
bot owners for.

We also bump the `API_FEATURE_LEVEL` to 5 here.  We
had already documented this in the prior commit to
add `owner_id`.

Note that we don't have to worry about mobile/ZT clients
here--we only deal with bot data in the webapp.
2020-05-12 09:48:37 -07:00
Anders Kaseorg 4362cceffb portico: Add setting to put Google Analytics on selected portico pages.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-05-11 23:22:50 -07:00
Aman Agrawal 898e717b9d register:api: Provide additional fields that can be useful to clients.
Fixes #14166

* Bumped the api version to feature level 4.
See https://github.com/zulip/zulip/issues/14166#issuecomment-598961462
for details about implementation and future steps.
2020-05-08 14:54:54 -07:00
Tim Abbott a920544bc3
invite emails: Ensure user-controlled input is always in links.
Popular email clients like Gmail will automatically linkify link-like
content present in an HTML email they receive, even if it doesn't have
links in it.  This made it possible to include what in Gmail will be a
user-controlled link in invitation emails that Zulip sends, which a
spammer/phisher could try to take advantage of to send really bad spam
(the limitation of having the rest of the invitation email HTML there
makes it hard to do something compelling here).

We close this opportunity by structuring our emails to always show the
user's name inside an existing link, so that Gmail won't do new
linkification, and add a test to help ensure we don't remove this
structure in a future design change.

Co-authored-by: Anders Kaseorg <andersk@mit.edu>
2020-05-08 14:02:41 -07:00
Anders Kaseorg 8411f7f884 version: Update for Zulip Desktop v5.2.0.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-05-07 11:25:15 -07:00
Hashir Sarwar 55a8e7dff2 settings: Offer hiding presence info from other users.
For privacy-minded folks who don't want to leak the
information of whether they're online, this adds an
option to disable sending presence updates to other
users.

The new settings lies in the "Other notification
settings" section of the "Notification settings"
page, under a "Presence" subheading.

Closes #14798.
2020-05-02 15:09:05 -07:00
Anders Kaseorg baab041acd lint: Upgrade semgrep from 0.5.0-dev.2 to 0.5.0-dev.4.
The major PROVISION_VERSION bump would not be needed, but it was
missing in commit 5ab62a3514 (#14834),
so I’m doing it here.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-05-02 10:19:40 -07:00
Anders Kaseorg 82f629091a lint: Check docs/THIRDPARTY for format errors.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-05-02 10:10:22 -07:00
Aman Agrawal 5ab62a3514 sgrep/semgrep: Updgrade and rename sgrep to semgrep.
With its new upgrade, sgrep has been renamed to semgrep.

Rename sgrep.yml to semgrep.yml
2020-05-01 11:01:14 -07:00
Manav Mehta 7d33bcc02d version: Update for Zulip Desktop v5.1.0 release.
This only updates zulipchat.com/apps/ downloads page links.
2020-04-30 10:53:50 -07:00
Tim Abbott cbe0723199 api docs: Add an API changelog page.
I imagine this can be improved in various ways, but I've initialized
this with all the **Changes** entries recorded in either zulip.yaml or
the rest of the API documentation, and I expect we'll be able to
iterate on this effectively.

It'll also be useful as a record of changes that we should remember to
document the API documentation as we document more endpoints that
currently don't discuss these issues.

While working on this, I fixed various issues where feature levels
could be mentioned or endpoints didn't properly document changes.
2020-04-28 22:23:25 -07:00
Puneeth Chaganti e41f4b01c0 requirements: Upgrade IPython to the latest version. 2020-04-24 17:25:33 -07:00
Puneeth Chaganti 46a4252139 requirements: Add jsx-lexer for syntax highlighting React code. 2020-04-24 12:39:10 -07:00
Eeshan Garg 75b2264a3f pypi: Upgrade zulip/zulip-bots dependencies to version 0.7.0.
Includes this change:
* openapi/python_examples: Update get_single_user.

This updates get_single_user to pass keyword arguments to
get_user_by_id instead of passing a dictionary.

Which is required for CI to pass, as we indeed fixed the API of that
function (which had only been present with the wrong API for one release).
2020-04-23 17:41:47 -07:00
Steve Howell 02252c255a db tools: Use common scheme for digests.
We have two different digest schemes to make
sure we keep the database up to date.  There
is the migration digest, which is NOT in the
scope of this commit, and which already
used the mechanism we use for other tools.

Here we are talking about the digest for
important files like `populate_db.py`.

Now our scheme is more consistent with how we
check file changes for other tools (as
well as the aformentioned migration files).

And we only write one hash file, instead of
seven.

And we only write the file when things have
actually changed.

And we are explicit about side effects.

Finally, we include a couple new bot settings
in the digest:

    INTERNAL_BOTS
    DISABLED_REALM_INTERNAL_BOTS

NOTE: This will require a one-time transition,
where we rebuild both databases (dev/test).
It takes a little over two minutes for me,
so it's not super painful.

I bump the provision version here, even
though you don't technically need it (since
the relevant tools are actually using the
digest files to determine if they need to
rebuild the database).  I figure it's just
good to explicitly make this commit trigger
a provision, and the user will then see
the one-time migration of the hash files
with a little bit less of a surprise.

And I do a major bump, not a minor bump,
because when we go in the reverse direction,
the old code will have to rebuild the
database due to the legacy hash files not
being around, so, again, I just prefer it
to be explicit.
2020-04-22 14:41:42 -07:00
Aman Agrawal 109e22506a
requirements: Upgrade libthumbor to latest release.
Upgrade libthumbor in main zulip venv. This version drops support
for python 2 and runs on py>=3.6.

As such, it is our first commit taking advantage of our having dropped support
for Debian Stretch and Ubuntu Xenial, our previous Python 3.5-based platforms.
2020-04-21 17:06:01 -07:00
Hashir Sarwar e3b90a5ec8 api: Add a monotonic integer "feature level" for non-webapp clients.
The purpose is to provide a way for (non-webapp) clients,
like the mobile and terminal apps, to tell whether the
server it's talking to is new enough to support a given
API feature -- in particular a way that

* is finer-grained than release numbers, so that for
features developed after e.g. 2.1.0 we can use them
immediately on servers deployed from master (like
chat.zulip.org and zulipchat.com) without waiting the
months until a 2.2 release;

* is reliable, unlike e.g. looking at the number of
commits since a release;

* doesn't lead to a growing bag of named feature flags
which the server has to go on sending forever.

Tweaked by tabbott to extend the documentation.

Closes #14618.
2020-04-21 13:37:57 -07:00
Puneeth Chaganti 7a5033852d provision: Add dependencies for Chromium used by latest puppeteer.
a51902a870 upgraded the puppeteer version to
3.0.0, which upgraded the Chromium version being used[1]. This version of
Chromium requires a couple of extra libraries to be installed.

[1]: https://github.com/puppeteer/puppeteer/releases/tag/v3.0.0
2020-04-20 23:25:45 -07:00
Anders Kaseorg 01a9b11e17 dependencies: Patch spectrum-colorpicker for jQuery 3.5 compatibility.
https://github.com/bgrins/spectrum/pull/556

Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2020-04-19 11:49:31 -07:00
Anders Kaseorg a51902a870 dependencies: Upgrade JavaScript dependencies.
Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2020-04-18 13:17:01 -07:00