Commit Graph

35942 Commits

Author SHA1 Message Date
Mateusz Mandera 5f15af2382 tests: Clean out unnecessary ifs from GitHubAuthBackendTest helper.
After the refactor moving this logic into a helper inside of
GitHubAuthBackendTest, these checks became unnecessary and always True.
2020-05-02 13:40:29 -07:00
Dinesh 9f3872d2b4 tests: Refactor `social_auth_test`.
As "choose email" screen is only used for GitHub auth, the part
that deals with it is separated from `social_auth_test` and
dealt in a new function `social_auth_finish`. This new
`social_auth_finish` contains only the code that deals with
authentication backends that do not have "choose email" screen.
But it is overidden in GitHub test class to handle the
"choose email" screen.
It was refactored because `expect_choose_email_screen` blocks
were confusing while figuring out how tests work on non GitHub
auths.
2020-05-02 13:40:29 -07:00
Hemanth V. Alluri 04811e724d webhooks/sentry: Rewrite the sentry webhook for the latest SDKs.
Sentry has client SDKs for many programming languages and frameworks.
Sentry has deprecated their old "Raven" series of client SDKs in favor
of a new series of client SDKs following their unified API format.

As it stood, our Sentry integration was already outdated being written
for the version 5 payloads (the Raven SDKs stopped at version 6 which
is already vastly different from version 5) when the current and
prominently used version is version 7.

This commit completely rewrites the existing Sentry integration.

Tested and supported events:
- Issue created, resolved, assigned, and ignored events.
- "Sentry events" for "capture exception" and "capture message" with
the Golang, Node.js, and Python SDKs (other SDKs should also work but
only these were used for testing).

For reference:
- Old (Raven) SDK for python:
    https://github.com/getsentry/raven-python
- New (Unified API format) SDK for python:
    https://github.com/getsentry/sentry-python

Signed-off-by: Hemanth V. Alluri <hdrive1999@gmail.com>
2020-05-02 13:39:57 -07:00
Hashir Sarwar 735785f985 stream_topic_history: Track the most recent message in streams.
This adds a way to keep track of max_message_id of a
stream and fetch it using the method get_max_message_id().

This will be useful for sorting streams by most recent
activity which will be implemented in the upcoming commit.

Essentially rewritten by tabbott to have a coherent tracking system,
and provide documentation.

Part of #10794.
2020-05-02 11:16:51 -07:00
Cyril Cohen 5598f8f6b0 gitter: Support importing data from multiple Gitter rooms.
**Features:**
Improving `./manage.py convert_gitter_data`
- If messages have been post-processed to add a 'room' field, we
  create as many streams as existing rooms.
- Messages with a 'room' field go to the corresponding stream.
- This modification is backward compatible. I.e.
  + messages that have no 'room' field go to the default stream/topic
  + messages that do, go to a specific stream

**Implementation:**
- adding a map `stream_map` to map room names to stream ids
- create as many streams as room field messages + 1 default streamFeatures:
- If messages have been post-processed to add a 'room' field to messages,
  we create as many streams as existing rooms.
- Up to renaming of the default stream/topic, this modification is
  backwards compatible.
  I.e. messages that have no 'room' field go to the default stream/topic
       messages that do, go to a specific stream

Implementation:
- adding a map stream_map to map room names to stream ids
- create as many streams as room field messages + 1 default stream

Takes advantage of https://github.com/minrk/archive-gitter/pull/5.
2020-05-02 10:30:18 -07:00
Anders Kaseorg baab041acd lint: Upgrade semgrep from 0.5.0-dev.2 to 0.5.0-dev.4.
The major PROVISION_VERSION bump would not be needed, but it was
missing in commit 5ab62a3514 (#14834),
so I’m doing it here.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-05-02 10:19:40 -07:00
Anders Kaseorg bdc365d0fe logging: Pass format arguments to logging.
https://docs.python.org/3/howto/logging.html#optimization

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-05-02 10:18:02 -07:00
Anders Kaseorg 82f629091a lint: Check docs/THIRDPARTY for format errors.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-05-02 10:10:22 -07:00
Anders Kaseorg b094534319 THIRDPARTY: Add missing license texts for CC-BY-3.0, CC-BY-SA-4.0.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-05-02 10:07:55 -07:00
Anders Kaseorg 587ab554f7 THIRDPARTY: Use standard CC0 short name.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-05-02 10:07:55 -07:00
Anders Kaseorg dbee6b5622 THIRDPARTY: Fix short name references for Apache-2.0, BSD-2-Clause.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-05-02 10:07:55 -07:00
Anders Kaseorg e11f709054 THIRDPARTY: Update Format URL to https.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-05-02 10:07:55 -07:00
Aman Agrawal beaa88a97b casper: Increase default timeout to avoid indeterministic CI failure.
Test fails at default timeout value indeterministically
because we have to wait for the server to start and the
above tests to pass, which takes more than the Default Timeout
of this test.
Hence, 4 x Default timeout value is kept.
2020-05-02 10:33:32 -04:00
sahil839 7cfbf660e6 actions: Remove deactivated stream from default stream groups.
Stream after deactivation should be removed from all the
default stream groups that it belongs to.

Fixes #14760.
2020-05-01 17:45:06 -07:00
jagansivam28 bbf5a5efed invitation: Make Member to see invitations sent by him/her.
Member of the org can able see list of invitations sent by him/her.
given permission for the member to revoke and resend the invitations
sent by him/her and added tests for test member can revoke and resend
the invitations only sent by him/her.

Fixes #14007.
2020-05-01 17:42:49 -07:00
jagansivam28 fc107d2c24 icon: Change icon for 'Invitations' from 'fa-user' to 'fa-user-plus'. 2020-05-01 17:29:45 -07:00
Tim Abbott 4f3976b917 process_fts_updates: Clean up logging output.
This saves a couple lines of spammy output in the run-dev.py startup
experience, and will be better output in production as well.
2020-05-01 11:51:20 -07:00
Steve Howell ce55ef6e4f tests: Use streamlined server for api/casper tests.
We now use the `--streamlined` options for `run-dev.py`
when we use `test_server_running` for `test-api` and
`test-js-with-casper` (and its experimental
replacement, `test-js-with-puppeteer`).

This means we don't slow anything down with
processes like thumbor, process_fts_updates, etc.,
which aren't meaningfully exercised by these tests.

We may eventually want some tests to meaningfully
exercise those processes, and when that day comes,
we will need to add an extra argument to
`test_server_running`, probably, but until then,
we just always set `--streamlined` in that codepath.

There is actually a tool called `./tools/test-run-dev`
that we run in CI, and it will use the full mode.
It just doesn't verify much stuff--it mostly polls
the server without testing specific features.

This seems to save about 1s of the startup time on a system I use
(~10.6s -> ~9.7s).
2020-05-01 11:37:09 -07:00
Steve Howell d9f8ec1fe7 run-dev: Add streamlined option.
For basic testing (either manual or automated), we
generally only need the server and tornado running.

Obviously, it's nice to test the complete system,
but if you're on a slow PC, the overhead can be
annoying.

Note that we don't launch any of these processes
in `--streamlined` mode:

    process_queue
    process_fts_updates
    deliver_scheduled_messages
    thumbor

And then by not launching process_queue, we avoid
several child processes.

Basic functionality like sending messages will
still work here.

The streamlined mode may be helpful in debugging
our generally slow server startup time.  Obviously,
some of the problem with startup is the auxiliary
processes here, but removing them as a variable
could help us focus on getting the core stuff fast.

Note that we still have the webpack watcher running
in streamlined mode.

For the particular case of thumbor, note that we
modify the proxy server to explicitly print and
return an error if we get a `/thumbor/*` request.
2020-05-01 11:36:43 -07:00
Steve Howell 28a2b90b04 run-dev: Extract server_processes().
We clean up the code related to launching
processes here.

We extract:

    server_processes

We also extract these helper for webpack
stuff:

    do_one_time_webpack_compile
    start_webpack_watcher

And then we move the code to actually launch
them lexically within the file (so as not to
be obscured by various function definitions).
2020-05-01 11:36:43 -07:00
Steve Howell 9cdc9cbca6 run-dev: Display ports more nicely.
Here is the new output for displaying ports:

    Zulip services will listen on ports:
       9991: web proxy
       9992: Django
       9993: Tornado
       9994: webpack
       9995: Thumbor

    Note to Vagrant users: Only the proxy port (9991) is exposed.

I tone down the yellow for the Vagrant warning, and I show
the web proxy in cyan to emphasize it.

I also extracted the code into a function, and I don't call
that function until after `app.listen()`.  (The users probably
won't notice much difference in the timing of this message, but
the message won't show if the `listen` step fails for some
reason, which I think is what we want here.)
2020-05-01 11:36:43 -07:00
Aman Agrawal 9089fd5b08 semgrep: Remove unrequired file exclude.
This file doen't contain any matching pattern which violates the
rule.
2020-05-01 11:08:47 -07:00
Aman Agrawal af0c267cd4 semgrep: Move migrations import check lint rule to semgrep.
We change how a few imports in migrations are done to be easier to
lint and more consitsent with our typical import style.
2020-05-01 11:07:53 -07:00
Aman Agrawal 8e29c88beb semgrep: Add rule to enforce no use of stream.objects.filter. 2020-05-01 11:01:14 -07:00
Aman Agrawal 06c831f146 semgrep: Reformat and correct doc link. 2020-05-01 11:01:14 -07:00
Aman Agrawal 5ab62a3514 sgrep/semgrep: Updgrade and rename sgrep to semgrep.
With its new upgrade, sgrep has been renamed to semgrep.

Rename sgrep.yml to semgrep.yml
2020-05-01 11:01:14 -07:00
vaibhavrajsingh2001 a3d51ae9af regression: Add minimum height to custom user field textbox.
Also, if the width of the texbox increased too much, the avatar icon
would move below. Fixed by setting max-width that matches the width of
the other textboxes.

Fixes: #14826
2020-05-01 11:00:56 -07:00
Steve Howell ea52bc987d run-dev: Clean up argument parsing code.
We remove the import-tools code that was plunked
right into the middle of our command line
arguments.

Then we add a local var called `DESCRIPTION` to
fix some ugly code formatting, and we stop with the
unnecessary `r` prefix to the multi-line string.
2020-05-01 10:40:34 -07:00
Rohitt Vashishtha 7d3a31cd8b bugdown: Support hanging_lists preprocessor for indented lists.
Previously, hanging_lists preprocessor didn't consider anything
indented at 4 or above spaces to be a list. This meant that when
we had a list like:

1. 1
  2. 2
    3. 3
  2. 2a
1. 1a

We would insert a newline between 3. 3 and 2. 2a. This resulted
in the block processor breaeking down 1 list into 2 blocks, which
messed up the nesting and indentation for the second block.
2020-04-30 17:54:40 -07:00
shubhamgupta2956 e1b8e9d7da css: Merge css properties of "#streams_header". 2020-04-30 16:56:34 -07:00
shubhamgupta2956 ff17279edb css: Use SCSS nesting for '.pm-box'. 2020-04-30 16:56:34 -07:00
shubhamgupta2956 fc8e6d6370 css: Use SCSS nesting in left-sidebar.scss.
Use SCSS nesting for "#streams_inline_cog, streams_filter_icon in
left-sidebar.scss.
2020-04-30 16:56:34 -07:00
shubhamgupta2956 628d9d1f25 css: Use nesting for "li.expanded_private_message". 2020-04-30 16:56:34 -07:00
shubhamgupta2956 85e151cc38 css: Use SCSS nesting for '.show-all-streams'. 2020-04-30 16:56:33 -07:00
shubhamgupta2956 18d7036eb6 css: Reorder ".show-all-streams". 2020-04-30 16:56:33 -07:00
shubhamgupta2956 974fd26139 css: Merge common CSS property in chevron styling. 2020-04-30 16:56:31 -07:00
shubhamgupta2956 662f202fa8 css: Use SCSS nesting for '#global_filters'. 2020-04-30 16:56:23 -07:00
shubhamgupta2956 e0071af1c7 css: Reorder "#global_filters". 2020-04-30 16:56:23 -07:00
shubhamgupta2956 3f76c963b2 css: Use SCSS nesting for "#left-sidebar". 2020-04-30 16:56:23 -07:00
shubhamgupta2956 ad3e7eebe3 css: Reorder "#left-sidebar" in left-sidebar.scss. 2020-04-30 16:56:23 -07:00
shubhamgupta2956 b9b73695cc css: Use SCSS nesting for "#stream_filters". 2020-04-30 16:56:23 -07:00
shubhamgupta2956 fb69457989 css: Reorder "#stream_filters". 2020-04-30 16:56:23 -07:00
Anders Kaseorg a552c2e5f9 auth: Use the clipboard instead of zulip:// for desktop auth flow.
This does not rely on the desktop app being able to register for the
zulip:// scheme (which is problematic with, for example, the AppImage
format).

It also is a better interface for managing changes to the system,
since the implementation exists almost entirely in the server/webapp
project.

This provides a smoother user experience, where the user doesn't need
to do the paste step, when combined with
https://github.com/zulip/zulip-desktop/pull/943.

Fixes #13613.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-04-30 16:45:00 -07:00
sahil839 1f809f338b node tests: Add test_realm_boolean test for default_twenty_four_hour_time.
This adds missing test in dispatch.js.
2020-04-30 16:15:52 -07:00
Pranav 43b0acdae8 settings: Use & selector reference to correctly style alerts.
The alert-error class styling was not being applied to alert-notifications
because of incorrect styling. The alert-notifications and the
alert-error classes were applied to the same element, whereas the CSS
expected the alert-error class to be applied to children of the class
with alert-notifications. Fixed this by using the parent-selector
reference '&' to correctly select the styling.
2020-04-30 16:13:49 -07:00
Pranav 41cdf11e16 settings_account: Remove redundant call to loading.make_indicator.
Previously, a spinner was created and this spinner element passed to
do_settings_change function, which also created a spinner, making the
first spinner creation useless. This commit removes the spinner creation
logic and just passes the element where the spinner is to be rendered.
2020-04-30 16:13:49 -07:00
vaibhavrajsingh2001 6292e96e4b i18n: Add translation tags to missed-message emails.
This fixes missing translation tags in our missed-message emails,
which is surely the most user-facing part of the production that
wasn't internationalized.

Fixes: #14398
2020-04-30 16:11:47 -07:00
Abhishek-Balaji 43e39718c4 user_name: Prevent users from setting name ending with |number.
We've had bugs in the past where users with a name in the format
"Alice|999" would confuse our markdown rendering or typeahead.  While
that's a fully solvable problem, there's no real use case for that, so
it's probably simpler to just prevent users from setting their name
that way.

Fixes #13923.
2020-04-30 15:59:12 -07:00
Wyatt Hoodes 5a58b7c549 data exports: Keep deleted export in UI table.
It makes sense to keep a deleted export in the table,
along with the time of deletion, for auditing reasons.
2020-04-30 13:00:59 -07:00
Wyatt Hoodes 2204f33936 data exports: Switch to solely using a status column. 2020-04-30 13:00:59 -07:00