Commit Graph

4231 Commits

Author SHA1 Message Date
Steve Howell 6f027d84a9 presence: Move get_status_dict_by_realm. 2020-02-06 17:16:22 -08:00
Tim Abbott 7c0a98754a home: Refactor logic for show_invites and show_add_streams. 2020-02-05 16:05:02 -08:00
Anders Kaseorg 4d49a20430 requirements: Upgrade django-sendfile2 from 0.4.3 to 0.5.1.
The module was renamed from sendfile to django_sendfile.

Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2020-02-05 12:38:10 -08:00
Ryan Rehman 174b2abcfd settings: Migrate to stream_post_policy structure.
This commit includes a new `stream_post_policy` setting,
by replacing the `is_announcement_only` field from the Stream model,
which is done by mirroring the structure of the existing
`create_stream_policy`.

It includes the necessary schema and database migrations to migrate
the is_announcement_only boolean field to stream_post_policy,
a smallPositiveInteger field similar to many other settings.

This change is done to allow organization administrators to restrict
new members from creating and posting to a stream. However, this does
not affect admins who are new members.

With many tweaks by tabbott to documentation under /help, etc.

Fixes #13616.
2020-02-04 17:08:08 -08:00
Mateusz Mandera 5de832283b test_auth_backends: Remove logger mocking that will fail on Django 2.2.
On Django 2.2 there is no longer a logger object in that module, so it's
best to remove this mocking as it's not essential to the tests.
2020-02-04 12:46:53 -08:00
Steve Howell e3ad9baf1d presence: Add process_presence_event.
This lets us conditionally remove the email
field from a presence event if the client
has registered with the slim_presence flag.
2020-02-04 12:30:36 -08:00
Steve Howell a672a00677 presence: Add user_id to presence event.
In a later commit, we will eliminate email for
clients who have set slim_presence as their
preference.
2020-02-04 12:30:36 -08:00
Steve Howell bf9144ff69 presence: Add slim_presence flag.
This flag affects page_params and the
payload you get back from POSTs to this
url:

    users/me/presence

The flag does not yet affect the
presence events that get sent to a
client.
2020-02-04 12:30:34 -08:00
Vishnu KS df5345705c import: Support importing team icon from slack. 2020-02-03 14:09:05 -08:00
Vishnu Ks 2ea53a347a import: Support importing realm icon and logo.
Fixes #11216
2020-02-03 14:09:05 -08:00
Ryan Rehman 3dc7d60ffe muting: Record DateTime when a Topic is muted.
This includes the necessary migration to add
the date_muted field to the MutedTopic class
and populates it with a hard coded value.
2020-02-02 20:49:53 -08:00
Mateusz Mandera 6aacc4195e login: Make authentication_methods data available to JavaScript.
This is intended to simplify overriding these buttons' controls in the
desktop app to do the authentication in the user's default browser.
2020-02-02 20:22:49 -08:00
Tim Abbott 4fba725803 test_auth_backends: Fix errors after rebasing.
Apparently, the rate-limiting PR had some import conflicts with our
recent authentication backend testing changes.
2020-02-02 20:22:32 -08:00
Mateusz Mandera 7b34853328 rate_limiter: Rename authenticate domain to authenticate_by_username.
This prepares for adding authenticate_by_ip_address.
2020-02-02 19:15:13 -08:00
Mateusz Mandera 7c78d8a966 rate_limiter: Limit the amount of password reset emails to one address.
This limits the possibility to use the password reset form to make us
spam an email address with password reset emails.
2020-02-02 19:15:13 -08:00
Mateusz Mandera 06198af5b9 auth: Handle rate limiting in OurAuthenticationForm and user_settings.
These parts of the code should catch the RateLimited exception and
generate their own, apprioprate user-facing error message.
2020-02-02 19:15:13 -08:00
Mateusz Mandera 5f94ea3d54 auth: Rate limit username+password authenticate() calls.
This applies rate limiting (through a decorator) of authenticate()
functions in the Email and LDAP backends - because those are the ones
where we check user's password.
The limiting is based on the username that the authentication is
attempted for - more than X attempts in Y minutes to a username is not
permitted.

If the limit is exceeded, RateLimited exception will be raised - this
can be either handled in a custom way by the code that calls
authenticate(), or it will be handled by RateLimitMiddleware and return
a json_error as the response.
2020-02-02 19:15:13 -08:00
Mateusz Mandera eea68ce92d auth: Support desktop_flow_otp with remote_user_sso. 2020-02-02 19:14:40 -08:00
Mateusz Mandera 05e08891b2 auth: Replace deprecated password_reset_confirm.
Tests require adjusting, because the class-based view has an additional
redirect - through /uid/set-password/ and the token is read from the
session. See Django code of PasswordResetConfirmView.
2020-02-02 17:03:10 -08:00
Steve Howell 3090cc0cd0 Add slim_presence param for get_status_dict_by_realm.
We want to move away from email keys, but we need to
keep backward compatibility for mobile.
2020-02-02 16:57:55 -08:00
Tim Abbott 5825a155cc users: Use format_user_row in events system as well.
This completes the deduplication of our logic for turning users into
dictionaries in the Zulip API.
2020-01-31 14:47:16 -08:00
akashaviator 20b8b29d11 users: Rewrite get_cross_realm_dicts to call format_user_row.
This modifies get_cross_realm_dicts in zerver.lib.users to call
format_user_row.  This is done to remove current and prevent future
inconsistencies between in the dictionary formats for get_raw_user_data
and get_cross_realm_dicts.

Implementation substantially rewritten by tabbott.

Fixes #13638.
2020-01-31 14:28:46 -08:00
Tim Abbott cfe3ba31f5 test_openapi: Add prioritization for list of undocumented endpoints. 2020-01-31 13:52:36 -08:00
Vishnu KS db97e0e329 emails: Use the word email instead of message in do not reply sentence.
Fixes #13693
2020-01-31 12:29:58 -08:00
Vishnu KS 47e442e4a4 emails: Show proper message when email content is not shown. 2020-01-31 12:29:58 -08:00
Vishnu KS 6dca46f206 tests: Check whether body include multiple strings in _test_cases. 2020-01-31 12:29:58 -08:00
Mateusz Mandera c618f0770e social_auth: Clear session fields leftover from previous auth attempts.
Fixes #13560.
2020-01-30 14:45:12 -08:00
Mateusz Mandera 8bd3752d13 email_mirror: Handle encoded attachment filenames. 2020-01-30 13:03:47 -08:00
Mateusz Mandera 39b012a276 server_initialization: Set internal bots owners to themselves. 2020-01-29 17:26:45 -08:00
Tim Abbott 6e75c27f74 test_events: Fix nested use of check_events_dict.
In addition to making our schema check stricter, it also makes it
possible for us to extend check_events_dict to do additional
validation that's only expected for the full event object.
2020-01-29 13:02:05 -08:00
Mateusz Mandera 9dcf677bf9 email_mirror: Parse encoded From headers with show_sender=True. 2020-01-29 12:27:35 -08:00
Tim Abbott b25fea24e7 messages: Simplify API for use_first_unread_anchor.
Now that we have the type situation of having anchor support passing a
string, this is a much more natural way to implement
use_first_unread_anchor.

We still support the old interface to avoid breaking compatibility
with legacy versions of the mobile apps.
2020-01-29 12:17:03 -08:00
Tim Abbott 05108760f6 narrow: Add support for passing oldest/newest for anchor.
A wart that has long been present inin Zulip's get_messages API is how
to request "the latest messages" in the API.  Previously, the
recommendation was basically to pass anchor=10000000000000000 (for an
appropriately huge number). An accident of the server's implementation
meant that specific number of 0s was actually important to avoid a
buggy (or at least wasteful) value of found_newest=False if the query
had specified num_after=0 (since we didn't check).

This was the cause of the mobile issue
https://github.com/zulip/zulip-mobile/issues/3654.

The solution is to allow passing a special value of anchor='newest',
basically a special string-type value that the server can interpret as
meaning the user precisely just wants the most recent messages.  We
also add an analogous anchor='oldest' or similar to avoid folks
needing to write a somewhat ugly anchor=0 for fetching the very first
messages.

We may want to also replace the use_first_unread_anchor argument to be
a "first_unread" value for the anchor parameter.

While it's not always ideal to make a value have a variable type like
this, in this case it seems like a really clean way to express the
idea of what the user is asking for in the API.
2020-01-29 12:14:06 -08:00
Greg Price a5aa541999 portico: Provide isolated single-page versions of /terms and /privacy .
The `isolated_page` context flag we rely on was added in the
parent commit.
2020-01-29 11:54:20 -08:00
Tim Abbott bcbc8f2bd5 portico: Move portico view code to its own file.
This improves the readability of the codebase.
2020-01-29 11:54:20 -08:00
Tim Abbott e2810d7549 narrow: Fix server handling of other larger anchor values.
This fixes a bug where that clients using the legacy approach of a
"very large anchor" value with the intent to only get the most recent
messages would only get found_newest=True if they used the specific
value LARGER_THAN_MAX_MESSAGE_ID.  Now any value at least that large
will work.

In upcoming commits, we plan to replace this with passing the string
"last", but it seems worth removing the buggy "special value" behavior
while we're touching this code.
2020-01-29 11:24:58 -08:00
Tim Abbott c0712431df openapi: Add hacky support for oneOf parameter types.
This is required for the upcoming type behavior of the "anchor"
parameter.

This change is the minimal work required to have our OpenAPI code not
fail when checking a union-type value of this form.  We'll likely want
to, in the future, do something nicer, but it'd require more extensive
infrastructure for parsing of OpenAPI data that it's worth with our
current approach (we may want to switch to using a library).
2020-01-29 11:24:58 -08:00
Tim Abbott 62500bcfae test_narrow: Add LARGER_THAN_MAX_MESSAGE_ID corner case tests.
These tests reveal the buggy behavior of very larger anchors other
than LARGER_THAN_MAX_MESSAGE_ID.
2020-01-29 11:24:58 -08:00
Rohitt Vashishtha 630c564fc7 bugdown: Rewrite List Preprocessor logic to properly parse fences.
Previously, we didn't track opening and closing fences separately,
with led to bugs like not parsing a list that was immediately after
a quoted fence; we treated each ``` as a new fence.

This commit rewrites the function to maintain a stack of currently
open fences. If any of the parent fences is a code fence, we do not
insert a new line before a list.

We also add some test cases specifically to test this behavior with
complexly nested lists.

Fixes #13745.
2020-01-27 17:14:27 -08:00
Hashir Sarwar 13b3eb24b0 invitations: Avoid adding to notifications stream unconditionally.
Adding invited users to the notifications stream unconditionally isn't
a correct behaviour for guest users, where the previous behavior of
including the notifications stream no longer makes sense. Therefore,
while inviting a new user, the notifications stream is listed along
with other streams with a message "recieves notifications for new
streams" in order to distinguish it from other streams.

Fixes #13645.
2020-01-27 15:36:59 -08:00
Mateusz Mandera 92c16996fc redis_utils: Require key_format argument in get_dict_from_redis. 2020-01-26 21:40:15 -08:00
Mateusz Mandera ad460e6ccb redis_utils: Validate requested key length in helper functions. 2020-01-26 21:40:15 -08:00
Mateusz Mandera 859bde482d auth: Implement server side of desktop_flow_otp. 2020-01-26 21:40:15 -08:00
Mateusz Mandera 8d987ba5ae auth: Use tokens, with data stored in redis, for log_into_subdomain.
The desktop otp flow (to be added in next commits) will want to generate
one-time tokens for the app that will allow it to obtain an
authenticated session. log_into_subdomain will be the endpoint to pass
the one-time token to. Currently it uses signed data as its input
"tokens", which is not compatible with the otp flow, which requires
simpler (and fixed-length) token. Thus the correct scheme to use is to
store the authenticated data in redis and return a token tied to the
data, which should be passed to the log_into_subdomain endpoint.

In this commit, we replace the "pass signed data around" scheme with the
redis scheme, because there's no point having both.
2020-01-26 21:32:44 -08:00
Abhishek-Balaji 434e8d3104 home: Extract compute_show_invites_and_add_streams.
This extracts a function for computing show_invites and
show_add_streams, for better readability and testability.

This commit was substantially cleaned up by tabbott.
2020-01-25 23:41:08 -08:00
Vishnu KS 97a25657a5 tests: Set class name of video call test to TestVideoCall.
The previous class name TestFeedbackBot was probably
leftover from copy paste.
2020-01-25 22:54:59 -08:00
Vishnu KS 05b4610381 bots: Remove feedback cross realm bot.
This completes the remaining pieces of removing this missed in
d70e799466 (mostly in tests).
2020-01-25 22:54:44 -08:00
Tim Abbott d70e799466 bots: Remove FEEDBACK_BOT implementation.
This legacy cross-realm bot hasn't been used in several years, as far
as I know.  If we wanted to re-introduce it, I'd want to implement it
as an embedded bot using those common APIs, rather than the totally
custom hacky code used for it that involves unnecessary queue workers
and similar details.

Fixes #13533.
2020-01-25 22:41:39 -08:00
Mateusz Mandera bce50ee652 auth: Use authenticate_remote_user in remote_user_jwt.
authenticate_remote_user already takes care of calling the authenticate
with the dummy backend. Also, return_data is not used and catching
DoesNotExist exception is not needed, as the dummy backend just returns
None if user isn't found.
2020-01-23 16:24:07 -08:00
Tim Abbott e052ec58db slack import: Improve error messages around invalid tokens.
This updates our error handling of invalid Slack API tokens (and other
networking error handling) to mostly make sense:
* A token that doesn't start with `xoxp-` gives an extended error early.
* An AssertionError for the codebase is correctly declared as such.
* We check for token shape errors before querying the Slack API.

We could still do useful work to raise custom exception classes here.

Thanks to @stavrospat for raising this issue.
2020-01-22 14:48:32 -08:00