Commit Graph

31571 Commits

Author SHA1 Message Date
Puneeth Chaganti d56b16b275 url preview: Ignore open graph tags without a content attribute. 2019-05-06 12:37:32 -07:00
Puneeth Chaganti d02eb99831 url preview: Return generic parser <p> text as str (not bs4 string). 2019-05-06 12:37:32 -07:00
Wyatt Hoodes 2b3de816a8 test_classes.py: Add activation to default language.
Running the backend tests with a high number of processes can cause
unexpected errors with language changes.  When certain tests that change
the default language, (without explicitly overriding the teardown method
to reset the default language), interleave with other tests that are
expecting the language to be in English, discrepancies arise.

This fixes a common nondeterministic test failure with high levels of
parallelization.
2019-05-06 11:16:08 -07:00
Tim Abbott 1af4f8fe20 soft_deactivation: Add some explanatory comments.
This function still doesn't make sense in the way I'd like it to, but
this at least records what algorithm we're trying to implement.
2019-05-05 18:33:15 -07:00
Tim Abbott eb97e9fae0 soft_deactivation: Fix buggy handling of race condition.
If a soft deactivated user had a subscription double-toggled without
any new messages being sent in between, add_missing_messages might
incorrectly process those two subscription changes in the wrong order.

Fortunately, the failure mode was usually to throw this exception:

django.db.utils.IntegrityError: duplicate key value violates unique
constraint "zerver_usermessage_user_profile_id_message_id_4936d0df_uniq"
DETAIL:  Key (user_profile_id, message_id)=(4, 57) already exists.

Our unit tests actually had this precise setup some fraction of the
time, because a bit of the test setup code subscribed+unsubscribed the
target user without sending any messages in between, resulting in a
test failure something like 50% of the time.

The original exception was hard to reproduce reliably originally
(resulting in an extremely annoying nondetermnistic test failure), but
is easily reproducible by changing the "id" to "-id" in this change to
always mis-order the processing of those RealmAuditLog events.
2019-05-05 18:29:20 -07:00
Tim Abbott 8218bf101c soft_deactivation: Fix buggy handling of other streams.
Previously, our soft-deactivation logic incorrectly did not filter the
set of stream subscription changes to look at to only include the
target stream.

This could result in unspecified buggy behavior.
2019-05-05 18:29:20 -07:00
Tim Abbott 8e995deab5 soft_deactivation: Make the stream_messages mutation logic clearer.
This changes our code from something that's pretty nasty bad behavior
mutating objects during a loop to only somewhat bad behavior.
2019-05-05 18:29:20 -07:00
Tim Abbott a1d2b73790 soft_deactivation: Clarify loop logic around stream_messages.
Break will do the same thing as continue here, as each iteration will
have the same result, and it's also worth explaining why this isn't
one layer up in the loop setup.
2019-05-05 18:29:20 -07:00
Mayank Madan 99414e2d96 provision: Add support for Debian 10 Buster. 2019-05-05 17:57:19 -07:00
Mayank Madan aae01f5795 dependencies: Install line_profiler from our fork.
This is required for Python 3.7 support; line_profiler needs a release
with Cython rerun following the Python 3.7 release, and this achieves
that goal.
2019-05-05 17:53:00 -07:00
Mayank Madan a86eccb2b6 pgroonga: Upgrade from 2.1.6 to 2.1.8.
This is necessary to support postgres 11 in Debian Buster.
2019-05-05 17:52:32 -07:00
Puneeth Chaganti dc1571426e open graph: Use the complete URL for open graph URLs.
Closes #12199
2019-05-05 17:51:53 -07:00
Raymond Akornor e01d3be1ba tools: Remove check-urls linter.
This commit removes `tools/check-urls`. It was added as
a useful tool in preparation for the Django 1.10 migration.
Since we completed that migration, it is no longer needed.

Fixes #12180.
2019-05-05 17:47:41 -07:00
Anders Kaseorg bc9d7141b2 mypy: Enable strict_optional for test_outgoing_webhook_interfaces.
Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-05-04 23:15:58 -07:00
Vishnu Ks 176d37a513 tests: Make tests work in reverse order in mattermost_importer. 2019-05-04 08:49:34 -07:00
vinitS101 81b5a72252 admin_settings: Change maxfilesize to max_file_upload_size.
Renamed maxfilesize to max_file_upload_size for consistency.
Related to #12152.
2019-05-03 17:36:09 -07:00
vinitS101 3cb502974e compose: Change file upload error message to show correct file size.
The error message, that is displayed when a user tries to upload a file
larger than the allowed size, currently displays a hardcoded max file
size of 25MB.
Changed this to show the correct max_file_upload_size value that has
been set by the server admin.
2019-05-03 17:36:09 -07:00
Kouhei Sutou a046a0957b docs: Fix upgrade process for Ubuntu 16.04 -> 18.04.
We need to disable "CREATE EXTENSION pgroonga" in zulip-puppet-apply
by creating /usr/share/postgresql/10/pgroonga_setup.sql.applied.
Because PostgreSQL 10 isn't running in this case. If PostgreSQL 10
isn't running, we can't run "CREATE EXTENSION pgroonga".

We can't use pg_upgrade with PGroonga. PGroonga's install SQL
https://github.com/pgroonga/pgroonga/blob/master/data/pgroonga.sql has
conditions to support multiple PostgreSQL versions. So it's not safe
to use pg_upgrade. pg_upgrade copies metadata for PostgreSQL 9.5 to
PostgreSQL 10. We need to use pg_dump and pg_restore to upgrade
PGroonga correctly for PostgreSQL 10.
2019-05-03 17:35:32 -07:00
Kouhei Sutou 407ed45ec2 docs: Use systemctl instead of service.
Because memcached package on Ubuntu 18.04 supports systemd.
2019-05-03 17:33:29 -07:00
David Wood 16e9892ab2 webhooks/jira: Include issue title in messages. 2019-05-03 20:03:31 -02:30
Vishnu Ks 123bcea518 management: Don't use sys.exit(1).
Using sys.exit in a management command makes it impossible
to unit test the code in question.  The correct approach to do the same
thing in Django management commands is to raise CommandError.

Followup of b570c0dafa
2019-05-03 14:20:39 -07:00
Tim Abbott 687f1bcdaf test_classes: Fix missing flushing of per-request caches.
We should definitely be starting each test case with an empty copy of
the per-request caches, since their intended duration is even shorter
than a request.

This was masked by the fact that these caches are automatically
flushed when one makes an actual request to the Zulip API; so the
problems were only manifesting in tests like test_events, where we
call lower-level functions that access a per-request cache without
using the Zulip API.
2019-05-03 13:52:56 -07:00
Tim Abbott 5cddf07361 test_mattermost_importer: Fix buggy use of wrong paths.
This worked for me when testing locally because it was using a
constant path, and so was accessing the product under `var/` of the
previous test run.
2019-05-03 13:30:54 -07:00
Tim Abbott 0adb93db32 test_classes: Fix data import path reuse parallelism errors.
The make_import_output_dir helper function used a path determined
primarily by the filename of the fixture being used, and expected to
have complete control over that path for the duration of the test.

This resulted in nondeterministic errors if our two test classes that
ran Mattermost import code ran at the same time.
2019-05-03 13:23:09 -07:00
Shubham Padia 884c19c3bc settings: NAME_CHANGES_DISABLED disables the name change button.
Fixes #12251.

Previously when disabling name changes in server settings, instead
of realm settings, the name edit button did not get disabled.
Changing name resulted in a message stating `no changes made`.
2019-05-03 12:58:51 -07:00
Shubham Padia 8c01f396db settings: Add AVATAR_CHANGES_DISABLED server setting.
Fixes #12132.
Realm setting to disable avatar changes is already present.
The `AVATAR_CHANGES_DISABLED` setting now follows the same
2-setting model as `NAME_CHANGES_DISABLED`.
2019-05-03 12:58:51 -07:00
Shubham Padia 4994a6c2c9 realm: Add setting to disable avatar changes.
This is useful when syncing avatars from an integrated LDAP/active
directory.

The upload avatar and delete avatar buttons are hidden if avatar
changes are disabled and the user is a non-admin.
If the user has a gravatar set, then the user will not be able to
upload an image as their avatar if avatar changes are disabled.

Part of #12132.
2019-05-03 12:52:43 -07:00
Mateusz Mandera 63c170c390 circleci: Fix redis on Bionic.
Fixes #11079.
Because of "service redis-server start" not working properly, in "do
Bionic hack" another instance of the server was being started. However,
once in a while this would fail for unknown reasons causing flakes in
tests involving redis. The cause of the service not starting was the
error:
::1:6379: bind: Cannot assign requested address
(see https://github.com/antirez/redis/issues/3241)
This is fixed by editing /etc/redis/redis.conf to not attempt binding
on IPv6. We change "do Bionic hack" to a simple sed command,
changing the appropriate line in the file.
By default, the line is "bind 127.0.0.1 ::1".
We change it to "bind 0.0.0.0".
2019-05-03 11:33:39 -07:00
Tim Abbott b570c0dafa convert_mattermost_data: Don't sys.exit(1).
This behavior ends up being quite problematic for running unit tests;
the thing one is supposed to do in management commands is raise
CommandError.
2019-05-02 11:13:41 -07:00
Wyatt Hoodes 21c67ea1c2 test-backend: Change --parallel default to use `cpu_count`.
The number of processes to run the backend tests is currently a
hardcoded value, this commit transistions the default to be based on the
number of logical CPUs available.
2019-05-02 10:48:20 -07:00
neiljp (Neil Pilgrim) 4ea5fea8a1 mypy: Fully check actions.py by removing mypy.ini special case. 2019-05-01 17:52:37 -07:00
neiljp (Neil Pilgrim) 7bbc620b28 mypy: Require topics to be non-None in Addressee. 2019-05-01 17:52:37 -07:00
Puneeth Chaganti bfc3e3c0c7 html_to_text: Add delimiters between text from different elements.
This module is used to render the HTML of pages like our user documentation 
into text for use in open graph previews of those articles.  It provided somewhat
confusing output in the case that there were paragraph breaks in the original message,
because text with multiple paragraphs and list items does't read very well. This commit
adds `|` as a delimiter between paragraphs, and prefixes list items with a `*`.

Closes #12228
2019-05-01 17:35:20 -07:00
Yashashvi Dave 22f172114d subscription: Update existing sub-row in settings on sub events.
Update stream-subscription-row in stream settings, when
subsctiption add/removal event is received. This is only
to show dynamic effects to user on sub events.
2019-05-01 17:23:44 -07:00
Yashashvi Dave a593f73e9a static/js/subs.js: Extract `update_stream_row_in_settings_tab` function. 2019-05-01 17:23:44 -07:00
Yashashvi Dave ed2dd612fb static/js/subs.js: Extract `is_subscribed_stream_tab_active` function. 2019-05-01 17:23:44 -07:00
okay 1694831029 bugdown: Fix double processed emoji tags inside inline tags.
When an emoji is nested inside another inline tag - like em or strong -
it was getting double processed because of the way the inlinePattern
TreeProcessor runs (it runs recursively). With this fix, we set the
inner text of the emoji span as an AtomicString, preventing us from
double processing the emoji's text.

Fixes #11621

Test Plan:

* Add test case for **😄**, verify it passes.
* Go into local dev server and send "**😄**" to self and verify the DOM
does not have double <span> tags for the emoji.
* Run zerver.tests.test_push_notifications and verify the markdown test case matches
the text_content field properly
2019-05-01 17:03:15 -07:00
Alexandra Ciobica 8aa982f7ba user status: Add "clear message (x)" button for status message input.
This adds the same "x" button as we have in "stream search" or "people
search" to the user status modal.

The button is shown if someone types something, or if the status
message was already set (meaning there was already a value in the
input field). If the input field is empty, the button is not visible.

This fixes the follow-up comments from #12179.
2019-05-01 16:36:45 -07:00
Alexandra Ciobica 61758735f3 user status: Clean up code for user status options.
- Changed the <p>s to <button>s and modified the css accordingly.
- Changed the css to use scss nesting.
- Changed the line-height from 1.0em to 1.1em, because on Safari the "g" was not fully displayed.
2019-05-01 16:34:54 -07:00
Eeshan Garg 28109e0f9e webhooks/jira: Ignore comment_created message.
comment_created payloads may not contain the required issue data
to format a useful notification, therefore, it is better to handle
issue comments through issue_updated events (which we already do).

Fixes: #11995.
2019-05-01 16:32:58 -07:00
Mateusz Mandera c1ceba9037 rate_limiter: Move email_mirror limiter to use rate_limit_entity.
We change the rate limiting code in the email mirror to use the new,
general rate_limit_entity function.
2019-05-01 12:54:32 -07:00
Mateusz Mandera 40763070b7 rate_limiter: Generalise some rate_limit_user code.
We create rate_limit_entity as a general rate-limiting function for
RateLimitedObjects, from code that was possible to abstract away from
rate_limit_user and that will be used for other kinds of rate limiting.
We make rate_limit_user use this new general framework from now.
2019-05-01 12:54:32 -07:00
neiljp (Neil Pilgrim) 827c016b98 mypy: Enable strict-optional for messages.py.
This also fixes an issue where we were setting num_after to None,
which should have been 0, to disable fetching messages after the
anchor.
2019-05-01 11:24:47 -07:00
neiljp (Neil Pilgrim) 47de637c3d mirrors: Refactor create_mirrored_message_users to use new Exception.
This enables the function to either return a valid UserProfile or raise
InvalidMirrorInput, which is clearer and more pythonic than the previous
approach of a tuple of a bool and Optional[UserProfile].

In making the type clearer, this improves checking with mypy.

Tests updated.
2019-05-01 11:18:22 -07:00
neiljp (Neil Pilgrim) 3d43682f1b mypy: Enable strict-optional for auth.py. 2019-05-01 10:49:25 -07:00
neiljp (Neil Pilgrim) f3ef2a186c mypy: Enable strict-optional for calculate_first_visible_message_id. 2019-05-01 10:49:24 -07:00
Tim Abbott edb956091f export: Add a blank line in S3 upload output.
This should be more readable.
2019-04-30 16:37:23 -07:00
Shubham Padia 27ff71c96f settings: Don't check mobile notifications if they are disabled.
Fixes #12198.
2019-04-30 14:56:26 -07:00
Vishnu Ks ed81906fc9 docs: Direct GSoD students to #documentation instead of #GSoC. 2019-04-30 14:52:53 -07:00
Yashashvi Dave 7a168216ff stream_ui_updates: Extract `update_change_stream_privacy_settings`. 2019-04-30 14:37:30 -07:00