Commit Graph

37704 Commits

Author SHA1 Message Date
Mateusz Mandera 890cafac11 retention: Use batch size of 100 for stream messages.
Streams can have lots of subscribers, meaning that the archiving process
will be moving tons of UserMessages per message. For that reason, using
a smaller batch size for stream messages is justified.

Some personal messages need to be added in test_scrub_realm to have
coverage of do_delete_messages_by_sender after these changes.
2020-06-24 10:41:00 -07:00
Mateusz Mandera 0c6497d43a retention: Add restore_retention_policy_deletions_for_stream function. 2020-06-24 10:40:38 -07:00
Mateusz Mandera 468f8cf488 retention: Improve logging of transactions. 2020-06-24 10:40:38 -07:00
Pragati Agrawal 33fa307c95 org settings: Display empty input when message retention days is forever.
Currently we display -1 in input box of id_realm_message_retention_days
when realm_message_retention_days is -1, which isn't user friendly.
Displaying the input box as empty is more intuitive.

And if the user tries to submit an empty input box we throw invalid JSON
error that isn't user friendly either, so fixed that too. In the ideal
case, we shouldn't send the request at first place to the backend when we
don't have any input.
2020-06-24 10:38:58 -07:00
Pragati Agrawal 1562ec758e org settings: Use 'forever' value instead of -1 for message_retention_days.
Currently, we use -1 as the Realm.message_retention_days value to retain
message forever unless specified at stream level for a particular stream,
that is, no policy set at the realm level. But this is incoherent with what
we use for Stream.message_retention_days where -1 means

> disable retention policy for this stream unconditionally

that can be confusing from an API standpoint.

So instead of trying some hack to reset the value to NULL or using some
other value like -2 for RETAIN_MESSAGE_FOREVER and use that for API. It is
much more intuitive to use a string like 'forever' that can be mapped to
RETAIN_MESSAGE_FOREVER at the backend. And this is similar to what we use
for streams settings as well.
2020-06-24 10:38:58 -07:00
Pragati Agrawal 7b3f4755ff streams: Fix bug that allows 0 value for message_retention_days. 2020-06-24 10:38:58 -07:00
Pragati Agrawal 72dd5bfa77 org settings: Use get_input_element_value style function to set value.
`get_input_element_value()` function is more reliable to detect the input
element type and extract it's value. But the current way of setting the
value of input elements relies on first checking the `property_value` type.
Which is fine, but for the cases when the property value is null, and we
want to set element value as empty, this method will throw an error as it's
unable to detect the appropriate element type. This new function
`set_input_element_value` first rely on property value and then use
`setting-widget-type` as a fallback.
2020-06-24 10:38:58 -07:00
Pragati Agrawal 1737cfdc7d org settings: Fix change detection of setting input elements.
This fixes the change detection of org settings input elements. Luckily,
this regression didn't break our populate_data_for_request and that's also
why this didn't come to notice.
2020-06-24 10:38:58 -07:00
Mateusz Mandera 7a03e2a7fe retention: Replace Realm.message_retention_days None value with -1.
To be more consistent with the meaning in the Stream model, and to make
it easier to have a reasonable settings API, we get rid of the None
value for Realm.message_retention_days in favor of the value -1 to
represent the "don't delete messages" default policy.
2020-06-24 10:33:21 -07:00
Hashir Sarwar 9b8521faee people: Enable `user_avatar_url_field_optional` in webapp.
In 5200598a31, we introduced a new
client capability that can be used to avoid unreasonable network
bandwidth consumed sending avatar URLs of long term idle users in
organizations with 10,000s members.

This commit enables this feature and adds support for it to the web
client.
2020-06-24 10:06:35 -07:00
Vishnu KS 138a0cb073 docs: Use modern django URL patterns in examples. 2020-06-24 09:57:43 -07:00
Steve Howell bc53fe1606 mypy: Use equals([]) to check services.
We are trying to deprecate sub_validator=None
for check_list.
2020-06-24 10:35:36 -04:00
Greg Price baa09f0518 docs: Clarify instructions on setting up email for Apple auth.
This comes after I read through all the linked docs, and went through
the UI to do this registration.
2020-06-23 19:04:26 -07:00
Greg Price 71c995d50d docs: Explain a bit more explicitly how to configure Apple auth.
In particular the Services ID and Bundle ID each have one of Apple's
random-looking 10-character identifiers, in addition to the Java-style
names the admin chooses.  Best to be clear about what names are
supposed to be the chosen names and which are supposed to be the
random-looking assigned names.

(I don't know of any docs elsewhere making this clear -- but I guessed
it'd be this way, and empirically it works.)

Also mention you need to enable the backend. :-)
2020-06-23 19:04:26 -07:00
Greg Price 0021dfe8e7 auth settings: Put Apple "Team ID" before the things it namespaces.
I believe the Bundle ID (aka App ID) and Services ID have meaning only
relative to a specific Team ID.  In particular, in some places in the
developer.apple.com UI, they're displayed in a fully-qualified form
like "ABCDE12345.com.example.app", where "com.example.app" is the
App ID or Services ID and ABCDE12345 is the Team ID.
2020-06-23 19:04:26 -07:00
Greg Price 9b620dfe6c docs: Break up a long sentence in Apple auth instructions.
This also helps keep the actionable part short, by separating
the actionable instruction from the longer parenthetical
explaining what it's for.
2020-06-23 19:04:26 -07:00
Ryan Rehman b57ebe717e input pill: Backspace deletes pills if caret is at start of input.
This allows deleting the last pill by the backspace key,
even if there is text input present.
2020-06-23 17:26:46 -07:00
Ryan Rehman 1e0340e0a3 css: Allow cross icon to stay at end of searchbox.
Fixes #10026.
2020-06-23 17:25:45 -07:00
Tim Abbott 4f7848d9b2 css: Fix starred messages not being displayed.
This change was missed in a4f5b0c635.

Ideally, we'd figure out a refactoring that made these definitions
only appear in one place.
2020-06-23 17:22:16 -07:00
Mateusz Mandera 7fe52bbb9e tests: Clean up the subdomain argument to social_auth_test.
subdomain=None didn't make much sense as a value, and wasn't actually in
use anywhere, except one test where it was accidental. All tests specify
the subdomain explicitly, so we should change the type to str, and make
it an obligatory kwarg.
2020-06-23 17:14:31 -07:00
Brainrecursion 30eaed0378 saml: Add option to restrict subdomain access based on SAML attributes.
Adds the ability to set a SAML attribute which contains a
list of subdomains the user is allowed to access. This allows a Zulip
server with multiple organizations to filter using SAML attributes
which organization each user can access.

Cleaned up and adapted by Mateusz Mandera to fit our conventions and
needs more.

Co-authored-by: Mateusz Mandera <mateusz.mandera@zulip.com>
2020-06-23 17:14:31 -07:00
Alex Vandiver 2283a16476
ci: Only attempt to post the failure if we have a key.
Only some branches define the `ZULIP_BOT_KEY`; others, including PRs,
thus currently fail to POST:

    {"result":"error","msg":"Malformed API key","code":"INVALID_API_KEY"}

Simply abort early, without attempting to report the result, if the
key is not present.
2020-06-23 16:43:18 -07:00
Anders Kaseorg d6655689f5 validator: Remove unused type_structure system.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-06-23 16:30:17 -07:00
Anders Kaseorg 07fa63e0c8 validator: Fix type errors hidden by bad Any use in set_type_structure.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-06-23 16:30:17 -07:00
Ryan Rehman fae365f8c7 message fetch: Fix anchor parameter before fetching messages.
If the `load_messages_for_narrow` function tried to fetch messages with
a floating point anchor (locally echoed message id) we get a 400 error.

The logic to remove the rounding of the anchor parameter was done in
commit bf2f36e6b4, as it would always be
an integer in all incoming code paths.

However, since then we have updated the GET /messages API, to also allow
strings as the anchor parameter and this value's parsing is done in the
backend.

Fixes #15497.
2020-06-23 15:46:04 -07:00
Ryan Rehman 59f7e66b60 message view: Fix erroneous "message outside current narrow" notice.
We update the logic for displaying the "message is outside current
narrow" notice, by comparing whether the message the client sent can
be appended to the current narrow's filter or not (this results in us
correctly falling through to reporting the user needs to scroll down).
2020-06-23 15:46:04 -07:00
wowol fec77381ee urls: Migrate zproject urls to use modern django patterns.
This migrations use of url() to path() or re_path().  In this commit,
we only migration regular expressions to path where the translation is trivial:
* URLs with no parameters in them
* URLs with only integer parameters in them
* Strings where there regular expression just checked for `/`s
path; strings, which can have variable validation in the URLs that
need by-hand auditing, we leave for future commits that are easier to
review and think about the individual changes.

Modified by tabbott to convert back to `re_path` various URLs with
strings that had been converted to use `path()` with string
validation to simplify review.

Fixes #14770.
2020-06-23 15:23:09 -07:00
wowol a3c475f137 urls: Migrate dev urls to use modern django patterns. 2020-06-23 15:02:44 -07:00
wowol 035d047dff urls: Migrate analytics urls to use modern django patterns. 2020-06-23 15:02:42 -07:00
Kartik Srivastava 3b8ee2a30f js_examples: Migrate and test update_message example. 2020-06-23 14:42:03 -07:00
Kartik Srivastava 8a00b1f1aa js_examples: Migrate and test update_message_flags example. 2020-06-23 14:42:03 -07:00
Kartik Srivastava cc4086e98a js_examples: Migrate and test remove_subscriptions example. 2020-06-23 14:35:08 -07:00
Kartik Srivastava db303ccaef js_examples: Migrate and test add_subscriptions example. 2020-06-23 14:35:08 -07:00
Kartik Srivastava cf6e1de0de js_examples: Migrate and test set_typing_status example. 2020-06-23 14:35:08 -07:00
Kartik Srivastava a1f6540cec js_examples: Migrate and test render_message example. 2020-06-23 14:35:07 -07:00
Kartik Srivastava 5a4d31825c js_examples: Migrate and test register_queue example. 2020-06-23 14:35:07 -07:00
Kartik Srivastava 4aec4cbfae js_examples: Migrate and test get_users example. 2020-06-23 14:35:07 -07:00
Kartik Srivastava 69a7d7fd30 js_examples: Migrate and test get_subscriptions example.
Also, `send_message` example is altered to send a message to the
stream 'social' to avoid getting a "first_message_id: null"
in the response for `get_subscriptions` example, that caused
`validate_against_openapi_schema` to throw an error.
2020-06-23 14:35:07 -07:00
Kartik Srivastava cc7671e602 js_examples: Migrate and test get_stream_topics example. 2020-06-23 14:35:07 -07:00
Kartik Srivastava 2c5815edbb js_examples: Migrate and test get_stream_id example. 2020-06-23 14:35:07 -07:00
Kartik Srivastava 63127d4990 js_examples: Migrate and test get_own_user example. 2020-06-23 14:35:07 -07:00
Kartik Srivastava 4615519d77 js_examples: Migrate and test get_messages example. 2020-06-23 14:35:07 -07:00
Kartik Srivastava 57e27ff7db js_examples: Migrate and test delete_queue example. 2020-06-23 14:35:07 -07:00
Kartik Srivastava f96c86ad63 js_examples: Migrate and test get_custom_emoji example. 2020-06-23 14:35:07 -07:00
jagansivam28 eaa085a6f8 realm logo: Add title for realm day/night logo.
Having title for realm day/night logo elements is
pretty good so that we can reduce confusion for new users.
2020-06-23 14:24:56 -07:00
jagansivam28 762f6c6cbc realm logo: Move day/night logo elements to allow them being alongside.
Previously in desktop view, the realm day/night logo element is arranged
one by one which is not looking good since we have a lot of space on the
the right side of the logo elements so we can move day/night logo elements
to allow them being alongside.

In mobile view, we don't have any space on the right side of the logo
elements so we don't have to change anything.
2020-06-23 14:24:56 -07:00
Alex Vandiver b165b4144d installer: Prevent flags which conflict with `--no-overwrite-settings`.
Since `--postgres-missing-dictionaries` edits `/etc/zulip/zulip.conf`,
it interferes with the intent of `--no-overwrite-settings`.

Make the two settings conflict, to prevent this unclear state.
2020-06-23 13:40:28 -07:00
Alex Vandiver 7f4a2527c0 installer: Make `--no-overwrite-settings` also preserve `zulip.conf`.
This allows a path through the installer for places that have already
configured `zulip.conf`, by extending the existing flag and behavior.
2020-06-23 13:40:28 -07:00
Alex Vandiver 27100b4507 installer: Fix mis-indentation. 2020-06-23 13:36:26 -07:00
Alex Vandiver 5b7be7ba5d installer: Do not initialize db with --no-init-db.
The `--no-init-db` option previously only controlled if
`initialize-database` was run, which sets up the tables inside the
database.  If PostgreSQL was installed locally, it still attempted to
create the user and empty database.

This fails on hosts which are remote PostgreSQL hosts, and not
application hosts, as:

 - They may already have a local database, and while
 `initialize-datbase` will detect and offer to abort if one is
 found,`--no-init-db` seems like it should be the option to not
 overwrite it

 - `flush-memcached` requires that a local venv be installed, which it
 often is not on non-frontend machines.

Skip the database configuration when run with `--no-init-db`.
2020-06-23 13:36:26 -07:00