Commit Graph

9641 Commits

Author SHA1 Message Date
neiljp (Neil Pilgrim) 47de637c3d mirrors: Refactor create_mirrored_message_users to use new Exception.
This enables the function to either return a valid UserProfile or raise
InvalidMirrorInput, which is clearer and more pythonic than the previous
approach of a tuple of a bool and Optional[UserProfile].

In making the type clearer, this improves checking with mypy.

Tests updated.
2019-05-01 11:18:22 -07:00
neiljp (Neil Pilgrim) 3d43682f1b mypy: Enable strict-optional for auth.py. 2019-05-01 10:49:25 -07:00
Tim Abbott edb956091f export: Add a blank line in S3 upload output.
This should be more readable.
2019-04-30 16:37:23 -07:00
Tim Abbott 98a1e340ad filters: Don't include realm_counts/installation_counts in errors.
This avoids unnecessarily long error emails for these.
2019-04-30 14:10:59 -07:00
David Wood 272ed90685 settings: Create an explicit invite_to_stream_policy setting.
This commit creates a new organization setting that determines whether
a user can invite other users to streams. Previously this was linked
to the waiting period threshold, but this was both not documented and
overly limiting.

With significant tweaks by tabbott to change the database model to not
involve two threshhold fields, edit the tests, etc.

This requires follow-up work to make the create stream policy setting
work how this code implies it should.

Fixes #12042.
2019-04-29 17:11:28 -07:00
Vishnu Ks 89ada6c770 tests: Remove unecessary word from convert_data test name. 2019-04-29 11:23:17 -07:00
Vishnu Ks ecdd3bea43 billing: Add cron job to run invoice_plans once a day.
Fixes #11960
2019-04-29 11:23:17 -07:00
Puneeth Chaganti f3f172991b open graph: Use uploaded realm icon for open graph images. 2019-04-28 22:50:27 -07:00
Tim Abbott 9e8cfbea3b test_email_notifications: Fix a nondeterministic test.
This test made the assumption that the two emails would be ordered in
a query without a ordering.
2019-04-28 22:00:03 -07:00
Puneeth Chaganti a98341c79f digest: Add an organisation setting to choose the digest weekday.
Closes #549.
2019-04-28 18:40:07 -07:00
Puneeth Chaganti f3d0ccb9f6 models: Add a digest_weekday field to the Realm model.
Allow realms to specify the day of the week when the digest should be sent out.
When enqueue-ing digests, pick only the realms that chose the current weekday as
the day to send out digests.
2019-04-27 15:20:41 -07:00
Puneeth Chaganti b0f485e95f org settings: Allow admins to enable or disable digest emails. 2019-04-27 15:20:41 -07:00
Mayank Madan 4c81474a4f webhooks: Add Buildbot integration.
With some minor changes by Eeshan Garg.

Fixes: #5639.
2019-04-27 15:19:16 -07:00
Eeshan Garg a73e8109b7 webhooks: Remove the legacy GitHub integration.
The github-services model for how GitHub would send requests to this
legacy integration is no longer available since earlier in 2019.
Removing this integration also allows us to finally remove
authenticated_api_view, the legacy authentication model from 2013 that
had been used for this integration (and other features long since
upgraded).

A few functions that were used by the Beanstalk webhook are moved into
that webhook's implementation directly.
2019-04-27 15:13:44 -07:00
Wyatt Hoodes d4715f23d7 public_export: Add backend API endpoint for triggering export.
An endpoint was created in zerver/views.  Basic rate-limiting was
implemented using RealmAuditLog.  The idea here is to simply log each
export event as a realm_exported event.  The number of events
occurring in the time delta is checked to ensure that the weekly
limit is not exceeded.

The event is published to the 'deferred_work' queue processor to
prevent the export process from being killed after 60s.

Upon completion of the export the realm admin(s) are notified.
2019-04-26 17:24:29 -07:00
Tim Abbott 8b5d2e9631 export: Return the S3 URL we uploaded data to.
This will make it possible to access that URL from the caller for the
data export tool.
2019-04-26 17:22:02 -07:00
Puneeth Chaganti dabe267d00 html_to_text: Add arg to specify html tags for generating text.
Closes #11497
2019-04-25 15:29:16 -07:00
Puneeth Chaganti 2d9c5b3575 open graph: Display realm description as open graph description. 2019-04-25 15:29:16 -07:00
Puneeth Chaganti a653fcca93 html_to_text: Escape text when using as description. 2019-04-25 15:29:16 -07:00
Puneeth Chaganti 009beabe53 test_middleware: Use test helper in nonexistent page open graph test. 2019-04-25 15:29:16 -07:00
Puneeth Chaganti 41ff94bdba test_middleware: Parse html in tests instead of using regex.
This slows down the tests by about 5-10% -- the tests go from 0.6s to 0.630s or
so. But, this seems like a change worth making to prevent open-graph metadata
breaking HTML.
2019-04-25 15:29:16 -07:00
Puneeth Chaganti 7d7134d45d html_to_text: Extract code for html to plain text conversion. 2019-04-25 15:29:16 -07:00
Tim Abbott cc421d4415 tests: Fix bad use of mock local variable name.
This ended up masking the mock module.
2019-04-25 15:28:10 -07:00
sameerchoubey 048ec39824 tools/test-api: Test Client.update_stream. 2019-04-25 16:24:59 -02:30
sameerchoubey 6790c22d8d tools/test-api: Test Client.update_presence. 2019-04-25 16:24:59 -02:30
sameerchoubey 598727f696 tools/test-api: Test Client.update_notification_settings. 2019-04-25 16:24:59 -02:30
sameerchoubey cedce30d99 tools/test-api: Test Client.update_user_group_members. 2019-04-25 16:24:59 -02:30
sameerchoubey 3140aabbd7 tools/test-api: Test Client.remove_user_group. 2019-04-25 16:24:59 -02:30
sameerchoubey 49e843cd75 tools/test-api: Test Client.update_user_group. 2019-04-25 16:24:59 -02:30
sameerchoubey 2c675efbba tools/test-api: Test Client.create_user_group. 2019-04-25 16:24:59 -02:30
sameerchoubey 53b58d0c7f tools/test-api: Test Client.remove_alert_words. 2019-04-25 16:24:59 -02:30
sameerchoubey c5c2c13880 tools/test-api: Test Client.add_alert_words. 2019-04-25 16:24:59 -02:30
sameerchoubey b26b4ce60c tools/test-api: Test Client.get_alert_words. 2019-04-25 16:24:59 -02:30
sameerchoubey 2d9b54f0d5 tools/test-api: Test Client.delete_stream. 2019-04-25 16:24:59 -02:30
sameerchoubey 56ad6758f6 tools/test-api: Test Client.remove_reaction. 2019-04-25 16:24:59 -02:30
Anders Kaseorg 21dc34cc52 open graph: HTML-escape og:description, twitter:description.
The entire idea of doing this operation with unchecked string
replacement in a middleware class is in my opinion extremely
ill-conceived, but this fixes the most pressing problem with it
generating invalid HTML.

Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-04-23 15:53:59 -07:00
Rishi Gupta 29d30ceab7 settings: Decouple enable_push_notifications_offline from PM setting.
Note that this setting has always applied to both streams and PMs; the test
just clarifies that that is the case.
2019-04-23 15:24:39 -07:00
Anders Kaseorg 9a9de156c3 lint: Fix calls to _() on computed strings.
Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-04-23 15:23:03 -07:00
Anders Kaseorg 643bd18b9f lint: Fix code that evaded our lint checks for string % non-tuple.
Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-04-23 15:21:37 -07:00
Eeshan Garg c012362fde webhooks/librato: Use proper punctuation. 2019-04-23 14:32:21 -07:00
Eeshan Garg d733cc500c webhooks/groove: Rewrite the integration and improve messages.
The initial goal was to improve message formatting and punctuation
but after a closer look, I realized that a larger refactor was
worth it for clarity and redability.
2019-04-23 14:32:21 -07:00
Tim Abbott cb6c1e7a92 export: Fix log line about zerver_scheduledemail_users. 2019-04-23 13:52:02 -07:00
Anders Kaseorg 098e5ad3de templates: Fix more low-hanging HTML validation problems.
"http://localhost:9981/help/enable-emoticon-translations":323.13-324.31: error: The “align” attribute on the “th” element is obsolete. Use CSS instead.
"http://localhost:9981/help/enable-emoticon-translations":324.45-325.31: error: The “align” attribute on the “th” element is obsolete. Use CSS instead.
"http://localhost:9981/help/enable-emoticon-translations":329.13-330.23: error: The “align” attribute on the “td” element is obsolete. Use CSS instead.
"http://localhost:9981/help/enable-emoticon-translations":330.44-331.23: error: The “align” attribute on the “td” element is obsolete. Use CSS instead.
"http://localhost:9981/help/enable-emoticon-translations":337.6-337.9: error: Stray end tag “p”.
"http://localhost:9981/help/enable-emoticon-translations":337.6-337.9: error: Cannot recover after last error. Any further errors will be ignored.
"http://localhost:9981/api/incoming-webhooks-walkthrough":381.4-381.77: error: An “img” element must have an “alt” attribute, except under certain conditions. For details, consult guidance on providing text alternatives for images.
"http://localhost:9981/apps/":192.21-192.34: error: Bad value “” for attribute “src” on element “img”: Must be non-empty.
"http://localhost:9981/apps/":192.21-192.34: error: An “img” element must have an “alt” attribute, except under certain conditions. For details, consult guidance on providing text alternatives for images.
"http://localhost:9981/features/":225.9-225.82: error: An “img” element must have an “alt” attribute, except under certain conditions. For details, consult guidance on providing text alternatives for images.
"http://localhost:9981/features/":230.72-232.8: error: Text not allowed in element “svg” in this context.
"http://localhost:9981/features/":259.9-259.91: error: An “img” element must have an “alt” attribute, except under certain conditions. For details, consult guidance on providing text alternatives for images.

Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-04-23 13:00:59 -07:00
Tim Abbott fa37401e69 api docs: Fix typo in typing notifications docs.
notification_to should be just to here.

This resulted from the person doing the port into these docs not
understanding the full REQ API.
2019-04-23 11:24:07 -07:00
Anders Kaseorg 61982d9d47 uploads: Revert "Url encoded name of the file should be an ascii."
This reverts commit fd9dd51d16 (#1815).

The issue described does not exist in Python 3, where urllib.parse now
_only_ accepts (Unicode) str and does the right thing with it.  The
workaround was not being triggered and would have failed if it were.

Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-04-22 22:28:39 -07:00
Vishwesh Jainkuniya c007b9ea4a notifcations: Remove `user` from the payload.
This contains email of the user to whom notification is being
send. This has not been used in any past mobile releases, so it is
safe to remove it.

As user_id will be stable for the user, but not email. So it's better to
start consuming `user_id` instead of email on mobile.
2019-04-22 14:50:04 -07:00
Vishwesh Jainkuniya 447a517e6f notifications: Add `user_id` in the GCM & APNS payload.
This makes it easy to uniquely identify the user account associated
with a notification by, for example, the (realm_uri, user_id) pair.

This helps improve notifications in the mobile apps.
See https://github.com/zulip/zulip-mobile/pull/3407#discussion_r266196616

Fixes #11961.
2019-04-22 14:49:03 -07:00
Puneeth Chaganti 184c0b9870 render_markdown_path: Convert dicts to dict item tuples for caching.
Calls to `render_markdown_path` weren't getting cached since the context
argument is unhashable, and the `ignore_unhashable_lru_cache` decorator ignores
such calls. This commit adds a couple of more decorators - one which converts
dict arguments to the function to a dict items tuple, and another which converts
dict items tuple arguments back to dicts. These two decorators used along with
the `ignore_unhashable_lru_cache` decorator ensure that the calls to
`render_markdown_path` with the context dict argument are also cached.

The time to run zerver.tests.test_urls.PublicURLTest.test_public_urls drops by
about 50% from 8.4s to 4.1s with this commit. The time to run
zerver.tests.test_docs.DocPageTest.test_doc_endpoints drops by about 20% from
3.2s to 2.5s.
2019-04-22 13:53:18 -07:00
Eeshan Garg d8d9695087 webhooks/newrelic: Improve message formatting and punctuation. 2019-04-17 16:22:43 -07:00
Eeshan Garg f86e445fe6 webhooks/sentry: Improve message formatting and punctuation. 2019-04-17 16:22:43 -07:00