Commit Graph

4264 Commits

Author SHA1 Message Date
Yashashvi Dave 02a1b2e26b server events: Add realm-time-sync event to update stream privacy.
Add event to update stream settings whenever stream privacy is changed
accordingly.

Fixes #9470
2019-05-07 17:33:59 -07:00
Eeshan Garg c4e4ceadb5 webhooks/git: Use proper punctuation for branch creation templates. 2019-05-07 16:45:01 -07:00
Eeshan Garg ab8aae6d0c git_webhooks: Use proper punctuation for PR/issue messages. 2019-05-07 16:45:01 -07:00
Anders Kaseorg 9efda71a4b get_realm: raise DoesNotExist instead of returning None.
This makes the implementation of `get_realm` consistent with its
declared return type of `Realm` rather than `Optional[Realm]`.

Fixes #12263.

Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-05-06 21:58:16 -07:00
Puneeth Chaganti 735b6cb761 digest: Remove code to gather new users and unread pms. 2019-05-06 17:43:53 -07:00
David Wood 34d810aac3 settings: Migrate to create_stream_policy structure.
This commit replaces the `create_stream_by_admins_only` setting with a
new `create_stream_policy` setting, which mirroring the structure of
the existing `invite_to_stream_policy`.

This is important preparation for migrating the waiting period feature
to be its own independent setting.

Fixes #12236.
2019-05-06 16:27:55 -07:00
Tim Abbott 3ecdabdc77 soft_deactivation: Add temporary nocoverage to fix CI. 2019-05-06 16:14:31 -07:00
Puneeth Chaganti da33b72848 url preview: Use in-memory caching in dev environment. 2019-05-06 12:37:32 -07:00
Puneeth Chaganti 1f6306a5a7 url preview: Cleanup import ordering. 2019-05-06 12:37:32 -07:00
Puneeth Chaganti d56b16b275 url preview: Ignore open graph tags without a content attribute. 2019-05-06 12:37:32 -07:00
Puneeth Chaganti d02eb99831 url preview: Return generic parser <p> text as str (not bs4 string). 2019-05-06 12:37:32 -07:00
Wyatt Hoodes 2b3de816a8 test_classes.py: Add activation to default language.
Running the backend tests with a high number of processes can cause
unexpected errors with language changes.  When certain tests that change
the default language, (without explicitly overriding the teardown method
to reset the default language), interleave with other tests that are
expecting the language to be in English, discrepancies arise.

This fixes a common nondeterministic test failure with high levels of
parallelization.
2019-05-06 11:16:08 -07:00
Tim Abbott 1af4f8fe20 soft_deactivation: Add some explanatory comments.
This function still doesn't make sense in the way I'd like it to, but
this at least records what algorithm we're trying to implement.
2019-05-05 18:33:15 -07:00
Tim Abbott eb97e9fae0 soft_deactivation: Fix buggy handling of race condition.
If a soft deactivated user had a subscription double-toggled without
any new messages being sent in between, add_missing_messages might
incorrectly process those two subscription changes in the wrong order.

Fortunately, the failure mode was usually to throw this exception:

django.db.utils.IntegrityError: duplicate key value violates unique
constraint "zerver_usermessage_user_profile_id_message_id_4936d0df_uniq"
DETAIL:  Key (user_profile_id, message_id)=(4, 57) already exists.

Our unit tests actually had this precise setup some fraction of the
time, because a bit of the test setup code subscribed+unsubscribed the
target user without sending any messages in between, resulting in a
test failure something like 50% of the time.

The original exception was hard to reproduce reliably originally
(resulting in an extremely annoying nondetermnistic test failure), but
is easily reproducible by changing the "id" to "-id" in this change to
always mis-order the processing of those RealmAuditLog events.
2019-05-05 18:29:20 -07:00
Tim Abbott 8218bf101c soft_deactivation: Fix buggy handling of other streams.
Previously, our soft-deactivation logic incorrectly did not filter the
set of stream subscription changes to look at to only include the
target stream.

This could result in unspecified buggy behavior.
2019-05-05 18:29:20 -07:00
Tim Abbott 8e995deab5 soft_deactivation: Make the stream_messages mutation logic clearer.
This changes our code from something that's pretty nasty bad behavior
mutating objects during a loop to only somewhat bad behavior.
2019-05-05 18:29:20 -07:00
Tim Abbott a1d2b73790 soft_deactivation: Clarify loop logic around stream_messages.
Break will do the same thing as continue here, as each iteration will
have the same result, and it's also worth explaining why this isn't
one layer up in the loop setup.
2019-05-05 18:29:20 -07:00
Tim Abbott 687f1bcdaf test_classes: Fix missing flushing of per-request caches.
We should definitely be starting each test case with an empty copy of
the per-request caches, since their intended duration is even shorter
than a request.

This was masked by the fact that these caches are automatically
flushed when one makes an actual request to the Zulip API; so the
problems were only manifesting in tests like test_events, where we
call lower-level functions that access a per-request cache without
using the Zulip API.
2019-05-03 13:52:56 -07:00
Tim Abbott 0adb93db32 test_classes: Fix data import path reuse parallelism errors.
The make_import_output_dir helper function used a path determined
primarily by the filename of the fixture being used, and expected to
have complete control over that path for the duration of the test.

This resulted in nondeterministic errors if our two test classes that
ran Mattermost import code ran at the same time.
2019-05-03 13:23:09 -07:00
neiljp (Neil Pilgrim) 7bbc620b28 mypy: Require topics to be non-None in Addressee. 2019-05-01 17:52:37 -07:00
Puneeth Chaganti bfc3e3c0c7 html_to_text: Add delimiters between text from different elements.
This module is used to render the HTML of pages like our user documentation 
into text for use in open graph previews of those articles.  It provided somewhat
confusing output in the case that there were paragraph breaks in the original message,
because text with multiple paragraphs and list items does't read very well. This commit
adds `|` as a delimiter between paragraphs, and prefixes list items with a `*`.

Closes #12228
2019-05-01 17:35:20 -07:00
okay 1694831029 bugdown: Fix double processed emoji tags inside inline tags.
When an emoji is nested inside another inline tag - like em or strong -
it was getting double processed because of the way the inlinePattern
TreeProcessor runs (it runs recursively). With this fix, we set the
inner text of the emoji span as an AtomicString, preventing us from
double processing the emoji's text.

Fixes #11621

Test Plan:

* Add test case for **😄**, verify it passes.
* Go into local dev server and send "**😄**" to self and verify the DOM
does not have double <span> tags for the emoji.
* Run zerver.tests.test_push_notifications and verify the markdown test case matches
the text_content field properly
2019-05-01 17:03:15 -07:00
Mateusz Mandera c1ceba9037 rate_limiter: Move email_mirror limiter to use rate_limit_entity.
We change the rate limiting code in the email mirror to use the new,
general rate_limit_entity function.
2019-05-01 12:54:32 -07:00
Mateusz Mandera 40763070b7 rate_limiter: Generalise some rate_limit_user code.
We create rate_limit_entity as a general rate-limiting function for
RateLimitedObjects, from code that was possible to abstract away from
rate_limit_user and that will be used for other kinds of rate limiting.
We make rate_limit_user use this new general framework from now.
2019-05-01 12:54:32 -07:00
Tim Abbott edb956091f export: Add a blank line in S3 upload output.
This should be more readable.
2019-04-30 16:37:23 -07:00
David Wood 272ed90685 settings: Create an explicit invite_to_stream_policy setting.
This commit creates a new organization setting that determines whether
a user can invite other users to streams. Previously this was linked
to the waiting period threshold, but this was both not documented and
overly limiting.

With significant tweaks by tabbott to change the database model to not
involve two threshhold fields, edit the tests, etc.

This requires follow-up work to make the create stream policy setting
work how this code implies it should.

Fixes #12042.
2019-04-29 17:11:28 -07:00
Puneeth Chaganti f3d0ccb9f6 models: Add a digest_weekday field to the Realm model.
Allow realms to specify the day of the week when the digest should be sent out.
When enqueue-ing digests, pick only the realms that chose the current weekday as
the day to send out digests.
2019-04-27 15:20:41 -07:00
Puneeth Chaganti b0f485e95f org settings: Allow admins to enable or disable digest emails. 2019-04-27 15:20:41 -07:00
Mayank Madan 4c81474a4f webhooks: Add Buildbot integration.
With some minor changes by Eeshan Garg.

Fixes: #5639.
2019-04-27 15:19:16 -07:00
Eeshan Garg a73e8109b7 webhooks: Remove the legacy GitHub integration.
The github-services model for how GitHub would send requests to this
legacy integration is no longer available since earlier in 2019.
Removing this integration also allows us to finally remove
authenticated_api_view, the legacy authentication model from 2013 that
had been used for this integration (and other features long since
upgraded).

A few functions that were used by the Beanstalk webhook are moved into
that webhook's implementation directly.
2019-04-27 15:13:44 -07:00
Wyatt Hoodes d4715f23d7 public_export: Add backend API endpoint for triggering export.
An endpoint was created in zerver/views.  Basic rate-limiting was
implemented using RealmAuditLog.  The idea here is to simply log each
export event as a realm_exported event.  The number of events
occurring in the time delta is checked to ensure that the weekly
limit is not exceeded.

The event is published to the 'deferred_work' queue processor to
prevent the export process from being killed after 60s.

Upon completion of the export the realm admin(s) are notified.
2019-04-26 17:24:29 -07:00
Tim Abbott 8b5d2e9631 export: Return the S3 URL we uploaded data to.
This will make it possible to access that URL from the caller for the
data export tool.
2019-04-26 17:22:02 -07:00
Puneeth Chaganti dabe267d00 html_to_text: Add arg to specify html tags for generating text.
Closes #11497
2019-04-25 15:29:16 -07:00
Puneeth Chaganti 2d9c5b3575 open graph: Display realm description as open graph description. 2019-04-25 15:29:16 -07:00
Puneeth Chaganti a653fcca93 html_to_text: Escape text when using as description. 2019-04-25 15:29:16 -07:00
Puneeth Chaganti 7d7134d45d html_to_text: Extract code for html to plain text conversion. 2019-04-25 15:29:16 -07:00
sameerchoubey 048ec39824 tools/test-api: Test Client.update_stream. 2019-04-25 16:24:59 -02:30
sameerchoubey 6790c22d8d tools/test-api: Test Client.update_presence. 2019-04-25 16:24:59 -02:30
sameerchoubey 598727f696 tools/test-api: Test Client.update_notification_settings. 2019-04-25 16:24:59 -02:30
sameerchoubey cedce30d99 tools/test-api: Test Client.update_user_group_members. 2019-04-25 16:24:59 -02:30
sameerchoubey 3140aabbd7 tools/test-api: Test Client.remove_user_group. 2019-04-25 16:24:59 -02:30
sameerchoubey 49e843cd75 tools/test-api: Test Client.update_user_group. 2019-04-25 16:24:59 -02:30
sameerchoubey 2c675efbba tools/test-api: Test Client.create_user_group. 2019-04-25 16:24:59 -02:30
sameerchoubey 53b58d0c7f tools/test-api: Test Client.remove_alert_words. 2019-04-25 16:24:59 -02:30
sameerchoubey c5c2c13880 tools/test-api: Test Client.add_alert_words. 2019-04-25 16:24:59 -02:30
sameerchoubey b26b4ce60c tools/test-api: Test Client.get_alert_words. 2019-04-25 16:24:59 -02:30
sameerchoubey 2d9b54f0d5 tools/test-api: Test Client.delete_stream. 2019-04-25 16:24:59 -02:30
sameerchoubey 56ad6758f6 tools/test-api: Test Client.remove_reaction. 2019-04-25 16:24:59 -02:30
Anders Kaseorg 9a9de156c3 lint: Fix calls to _() on computed strings.
Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-04-23 15:23:03 -07:00
Anders Kaseorg 643bd18b9f lint: Fix code that evaded our lint checks for string % non-tuple.
Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-04-23 15:21:37 -07:00