Commit Graph

3658 Commits

Author SHA1 Message Date
Mateusz Mandera 40f5755546 email_mirror: Handle case of unspecified charset in Content-Type header.
If the text part of an email message didn't specify the charset in the
Content-Type header, the text content wouldn't be found. We fix this, by
assuming us-ascii charset in those cases, as specified by RFC6657:
https://tools.ietf.org/html/rfc6657
2019-05-09 09:57:40 -07:00
Harshit Bansal b553507412 subscriptions: Migrate notification setting defaults model.
This commit migrates the Subscription's notification fields from a
BooleanField to a NullBooleanField where a value of None means to
inherit the value from user's profile.

Also includes a migrations to set the corresponding settings to None
if they match the user profile's values. This migration helps us in
getting rid of the weird "Apply to all" widget that we offered on
subscription settings page.

The mobile apps can't handle None appearing as the stream-level
notification settings, so for backwards-compatibility we arrange to
only send True/False to the mobile apps by applying those defaults
server-side.  We introduce a notification_settings_null value within a
client_capabilities structure that newer versions of the mobile apps
can use to request the new model.

This mobile compatibility code is pretty effectively tested by the
existing test_events tests for the subscriptions subsystem.
2019-05-08 17:45:10 -07:00
Tim Abbott 8a2057e3cf test_events: Print out the state when it hasn't changed.
This is often a part of what one will want when debugging.
2019-05-08 17:43:21 -07:00
vinitS101 18a424be79 uploads: Remove unusable UI elements if file uploading is disabled.
If MAX_FILE_UPLOAD_SIZE is set to 0, then UI elements like the upload
icon in the compose and message edit UI and "Attachments" menu in
"/#settings" are not displayed.
A different error message is also displayed if a user tries to drag and
drop or paste a file into the compose message box.

Fixes #12152.
2019-05-08 17:10:07 -07:00
Rishi Gupta 98da11c558 support: Rename deactive to deactivated. 2019-05-08 15:09:48 -07:00
Puneeth Chaganti d474a41c03 digest: Turn off digest_emails_enabled flag for realms by default. 2019-05-08 14:39:12 -07:00
Shubham Padia dd28413c4a ldap: Do not modify self._LDAPUser.attrs in test_query_email_attr.
Fixes #12273.
When running the test_query_email_attr test in reverse, the test failed
because self._LDAPUser.attrs was being modified and it was being shared
with other tests.
2019-05-08 09:55:44 -07:00
Vishnu Ks ce88df5069 tests: Patch maybe_update_first_visible_message_id properly.
Otherwise this will result in error when running
tests with --parallel=1 --reverse.
2019-05-08 09:55:03 -07:00
Yashashvi Dave 02a1b2e26b server events: Add realm-time-sync event to update stream privacy.
Add event to update stream settings whenever stream privacy is changed
accordingly.

Fixes #9470
2019-05-07 17:33:59 -07:00
Anders Kaseorg 9efda71a4b get_realm: raise DoesNotExist instead of returning None.
This makes the implementation of `get_realm` consistent with its
declared return type of `Realm` rather than `Optional[Realm]`.

Fixes #12263.

Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-05-06 21:58:16 -07:00
Puneeth Chaganti 735b6cb761 digest: Remove code to gather new users and unread pms. 2019-05-06 17:43:53 -07:00
Puneeth Chaganti 77e0154800 test_users: Add assertion that guests can't see all realm users.
This commit adds coverage for the `UserProfile.can_access_all_realm_members`
method outside of tests in `test_digest`.
2019-05-06 17:43:53 -07:00
Puneeth Chaganti be762f9485 digest: Strip down the digest email removing a lot of fluff. 2019-05-06 17:43:52 -07:00
David Wood 34d810aac3 settings: Migrate to create_stream_policy structure.
This commit replaces the `create_stream_by_admins_only` setting with a
new `create_stream_policy` setting, which mirroring the structure of
the existing `invite_to_stream_policy`.

This is important preparation for migrating the waiting period feature
to be its own independent setting.

Fixes #12236.
2019-05-06 16:27:55 -07:00
Tim Abbott cf0fc7c221 test_link_embed: Fix unused variable.
This should have been in bc2ebd0f09.
2019-05-06 16:04:37 -07:00
Puneeth Chaganti bc2ebd0f09 url preview: Refactor test code to create mock responses. 2019-05-06 12:37:32 -07:00
Puneeth Chaganti da33b72848 url preview: Use in-memory caching in dev environment. 2019-05-06 12:37:32 -07:00
Puneeth Chaganti dc1571426e open graph: Use the complete URL for open graph URLs.
Closes #12199
2019-05-05 17:51:53 -07:00
Vishnu Ks 176d37a513 tests: Make tests work in reverse order in mattermost_importer. 2019-05-04 08:49:34 -07:00
vinitS101 81b5a72252 admin_settings: Change maxfilesize to max_file_upload_size.
Renamed maxfilesize to max_file_upload_size for consistency.
Related to #12152.
2019-05-03 17:36:09 -07:00
Vishnu Ks 123bcea518 management: Don't use sys.exit(1).
Using sys.exit in a management command makes it impossible
to unit test the code in question.  The correct approach to do the same
thing in Django management commands is to raise CommandError.

Followup of b570c0dafa
2019-05-03 14:20:39 -07:00
Tim Abbott 5cddf07361 test_mattermost_importer: Fix buggy use of wrong paths.
This worked for me when testing locally because it was using a
constant path, and so was accessing the product under `var/` of the
previous test run.
2019-05-03 13:30:54 -07:00
Shubham Padia 884c19c3bc settings: NAME_CHANGES_DISABLED disables the name change button.
Fixes #12251.

Previously when disabling name changes in server settings, instead
of realm settings, the name edit button did not get disabled.
Changing name resulted in a message stating `no changes made`.
2019-05-03 12:58:51 -07:00
Shubham Padia 8c01f396db settings: Add AVATAR_CHANGES_DISABLED server setting.
Fixes #12132.
Realm setting to disable avatar changes is already present.
The `AVATAR_CHANGES_DISABLED` setting now follows the same
2-setting model as `NAME_CHANGES_DISABLED`.
2019-05-03 12:58:51 -07:00
Shubham Padia 4994a6c2c9 realm: Add setting to disable avatar changes.
This is useful when syncing avatars from an integrated LDAP/active
directory.

The upload avatar and delete avatar buttons are hidden if avatar
changes are disabled and the user is a non-admin.
If the user has a gravatar set, then the user will not be able to
upload an image as their avatar if avatar changes are disabled.

Part of #12132.
2019-05-03 12:52:43 -07:00
Puneeth Chaganti bfc3e3c0c7 html_to_text: Add delimiters between text from different elements.
This module is used to render the HTML of pages like our user documentation 
into text for use in open graph previews of those articles.  It provided somewhat
confusing output in the case that there were paragraph breaks in the original message,
because text with multiple paragraphs and list items does't read very well. This commit
adds `|` as a delimiter between paragraphs, and prefixes list items with a `*`.

Closes #12228
2019-05-01 17:35:20 -07:00
okay 1694831029 bugdown: Fix double processed emoji tags inside inline tags.
When an emoji is nested inside another inline tag - like em or strong -
it was getting double processed because of the way the inlinePattern
TreeProcessor runs (it runs recursively). With this fix, we set the
inner text of the emoji span as an AtomicString, preventing us from
double processing the emoji's text.

Fixes #11621

Test Plan:

* Add test case for **😄**, verify it passes.
* Go into local dev server and send "**😄**" to self and verify the DOM
does not have double <span> tags for the emoji.
* Run zerver.tests.test_push_notifications and verify the markdown test case matches
the text_content field properly
2019-05-01 17:03:15 -07:00
Mateusz Mandera c1ceba9037 rate_limiter: Move email_mirror limiter to use rate_limit_entity.
We change the rate limiting code in the email mirror to use the new,
general rate_limit_entity function.
2019-05-01 12:54:32 -07:00
Mateusz Mandera 40763070b7 rate_limiter: Generalise some rate_limit_user code.
We create rate_limit_entity as a general rate-limiting function for
RateLimitedObjects, from code that was possible to abstract away from
rate_limit_user and that will be used for other kinds of rate limiting.
We make rate_limit_user use this new general framework from now.
2019-05-01 12:54:32 -07:00
neiljp (Neil Pilgrim) 47de637c3d mirrors: Refactor create_mirrored_message_users to use new Exception.
This enables the function to either return a valid UserProfile or raise
InvalidMirrorInput, which is clearer and more pythonic than the previous
approach of a tuple of a bool and Optional[UserProfile].

In making the type clearer, this improves checking with mypy.

Tests updated.
2019-05-01 11:18:22 -07:00
David Wood 272ed90685 settings: Create an explicit invite_to_stream_policy setting.
This commit creates a new organization setting that determines whether
a user can invite other users to streams. Previously this was linked
to the waiting period threshold, but this was both not documented and
overly limiting.

With significant tweaks by tabbott to change the database model to not
involve two threshhold fields, edit the tests, etc.

This requires follow-up work to make the create stream policy setting
work how this code implies it should.

Fixes #12042.
2019-04-29 17:11:28 -07:00
Vishnu Ks 89ada6c770 tests: Remove unecessary word from convert_data test name. 2019-04-29 11:23:17 -07:00
Vishnu Ks ecdd3bea43 billing: Add cron job to run invoice_plans once a day.
Fixes #11960
2019-04-29 11:23:17 -07:00
Puneeth Chaganti f3f172991b open graph: Use uploaded realm icon for open graph images. 2019-04-28 22:50:27 -07:00
Tim Abbott 9e8cfbea3b test_email_notifications: Fix a nondeterministic test.
This test made the assumption that the two emails would be ordered in
a query without a ordering.
2019-04-28 22:00:03 -07:00
Puneeth Chaganti a98341c79f digest: Add an organisation setting to choose the digest weekday.
Closes #549.
2019-04-28 18:40:07 -07:00
Puneeth Chaganti b0f485e95f org settings: Allow admins to enable or disable digest emails. 2019-04-27 15:20:41 -07:00
Eeshan Garg a73e8109b7 webhooks: Remove the legacy GitHub integration.
The github-services model for how GitHub would send requests to this
legacy integration is no longer available since earlier in 2019.
Removing this integration also allows us to finally remove
authenticated_api_view, the legacy authentication model from 2013 that
had been used for this integration (and other features long since
upgraded).

A few functions that were used by the Beanstalk webhook are moved into
that webhook's implementation directly.
2019-04-27 15:13:44 -07:00
Wyatt Hoodes d4715f23d7 public_export: Add backend API endpoint for triggering export.
An endpoint was created in zerver/views.  Basic rate-limiting was
implemented using RealmAuditLog.  The idea here is to simply log each
export event as a realm_exported event.  The number of events
occurring in the time delta is checked to ensure that the weekly
limit is not exceeded.

The event is published to the 'deferred_work' queue processor to
prevent the export process from being killed after 60s.

Upon completion of the export the realm admin(s) are notified.
2019-04-26 17:24:29 -07:00
Puneeth Chaganti dabe267d00 html_to_text: Add arg to specify html tags for generating text.
Closes #11497
2019-04-25 15:29:16 -07:00
Puneeth Chaganti 2d9c5b3575 open graph: Display realm description as open graph description. 2019-04-25 15:29:16 -07:00
Puneeth Chaganti a653fcca93 html_to_text: Escape text when using as description. 2019-04-25 15:29:16 -07:00
Puneeth Chaganti 009beabe53 test_middleware: Use test helper in nonexistent page open graph test. 2019-04-25 15:29:16 -07:00
Puneeth Chaganti 41ff94bdba test_middleware: Parse html in tests instead of using regex.
This slows down the tests by about 5-10% -- the tests go from 0.6s to 0.630s or
so. But, this seems like a change worth making to prevent open-graph metadata
breaking HTML.
2019-04-25 15:29:16 -07:00
Tim Abbott cc421d4415 tests: Fix bad use of mock local variable name.
This ended up masking the mock module.
2019-04-25 15:28:10 -07:00
Anders Kaseorg 21dc34cc52 open graph: HTML-escape og:description, twitter:description.
The entire idea of doing this operation with unchecked string
replacement in a middleware class is in my opinion extremely
ill-conceived, but this fixes the most pressing problem with it
generating invalid HTML.

Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-04-23 15:53:59 -07:00
Rishi Gupta 29d30ceab7 settings: Decouple enable_push_notifications_offline from PM setting.
Note that this setting has always applied to both streams and PMs; the test
just clarifies that that is the case.
2019-04-23 15:24:39 -07:00
Anders Kaseorg 643bd18b9f lint: Fix code that evaded our lint checks for string % non-tuple.
Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-04-23 15:21:37 -07:00
Anders Kaseorg 61982d9d47 uploads: Revert "Url encoded name of the file should be an ascii."
This reverts commit fd9dd51d16 (#1815).

The issue described does not exist in Python 3, where urllib.parse now
_only_ accepts (Unicode) str and does the right thing with it.  The
workaround was not being triggered and would have failed if it were.

Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-04-22 22:28:39 -07:00
Vishwesh Jainkuniya c007b9ea4a notifcations: Remove `user` from the payload.
This contains email of the user to whom notification is being
send. This has not been used in any past mobile releases, so it is
safe to remove it.

As user_id will be stable for the user, but not email. So it's better to
start consuming `user_id` instead of email on mobile.
2019-04-22 14:50:04 -07:00