Commit Graph

9627 Commits

Author SHA1 Message Date
Wyatt Hoodes d4715f23d7 public_export: Add backend API endpoint for triggering export.
An endpoint was created in zerver/views.  Basic rate-limiting was
implemented using RealmAuditLog.  The idea here is to simply log each
export event as a realm_exported event.  The number of events
occurring in the time delta is checked to ensure that the weekly
limit is not exceeded.

The event is published to the 'deferred_work' queue processor to
prevent the export process from being killed after 60s.

Upon completion of the export the realm admin(s) are notified.
2019-04-26 17:24:29 -07:00
Tim Abbott 8b5d2e9631 export: Return the S3 URL we uploaded data to.
This will make it possible to access that URL from the caller for the
data export tool.
2019-04-26 17:22:02 -07:00
Puneeth Chaganti dabe267d00 html_to_text: Add arg to specify html tags for generating text.
Closes #11497
2019-04-25 15:29:16 -07:00
Puneeth Chaganti 2d9c5b3575 open graph: Display realm description as open graph description. 2019-04-25 15:29:16 -07:00
Puneeth Chaganti a653fcca93 html_to_text: Escape text when using as description. 2019-04-25 15:29:16 -07:00
Puneeth Chaganti 009beabe53 test_middleware: Use test helper in nonexistent page open graph test. 2019-04-25 15:29:16 -07:00
Puneeth Chaganti 41ff94bdba test_middleware: Parse html in tests instead of using regex.
This slows down the tests by about 5-10% -- the tests go from 0.6s to 0.630s or
so. But, this seems like a change worth making to prevent open-graph metadata
breaking HTML.
2019-04-25 15:29:16 -07:00
Puneeth Chaganti 7d7134d45d html_to_text: Extract code for html to plain text conversion. 2019-04-25 15:29:16 -07:00
Tim Abbott cc421d4415 tests: Fix bad use of mock local variable name.
This ended up masking the mock module.
2019-04-25 15:28:10 -07:00
sameerchoubey 048ec39824 tools/test-api: Test Client.update_stream. 2019-04-25 16:24:59 -02:30
sameerchoubey 6790c22d8d tools/test-api: Test Client.update_presence. 2019-04-25 16:24:59 -02:30
sameerchoubey 598727f696 tools/test-api: Test Client.update_notification_settings. 2019-04-25 16:24:59 -02:30
sameerchoubey cedce30d99 tools/test-api: Test Client.update_user_group_members. 2019-04-25 16:24:59 -02:30
sameerchoubey 3140aabbd7 tools/test-api: Test Client.remove_user_group. 2019-04-25 16:24:59 -02:30
sameerchoubey 49e843cd75 tools/test-api: Test Client.update_user_group. 2019-04-25 16:24:59 -02:30
sameerchoubey 2c675efbba tools/test-api: Test Client.create_user_group. 2019-04-25 16:24:59 -02:30
sameerchoubey 53b58d0c7f tools/test-api: Test Client.remove_alert_words. 2019-04-25 16:24:59 -02:30
sameerchoubey c5c2c13880 tools/test-api: Test Client.add_alert_words. 2019-04-25 16:24:59 -02:30
sameerchoubey b26b4ce60c tools/test-api: Test Client.get_alert_words. 2019-04-25 16:24:59 -02:30
sameerchoubey 2d9b54f0d5 tools/test-api: Test Client.delete_stream. 2019-04-25 16:24:59 -02:30
sameerchoubey 56ad6758f6 tools/test-api: Test Client.remove_reaction. 2019-04-25 16:24:59 -02:30
Anders Kaseorg 21dc34cc52 open graph: HTML-escape og:description, twitter:description.
The entire idea of doing this operation with unchecked string
replacement in a middleware class is in my opinion extremely
ill-conceived, but this fixes the most pressing problem with it
generating invalid HTML.

Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-04-23 15:53:59 -07:00
Rishi Gupta 29d30ceab7 settings: Decouple enable_push_notifications_offline from PM setting.
Note that this setting has always applied to both streams and PMs; the test
just clarifies that that is the case.
2019-04-23 15:24:39 -07:00
Anders Kaseorg 9a9de156c3 lint: Fix calls to _() on computed strings.
Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-04-23 15:23:03 -07:00
Anders Kaseorg 643bd18b9f lint: Fix code that evaded our lint checks for string % non-tuple.
Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-04-23 15:21:37 -07:00
Eeshan Garg c012362fde webhooks/librato: Use proper punctuation. 2019-04-23 14:32:21 -07:00
Eeshan Garg d733cc500c webhooks/groove: Rewrite the integration and improve messages.
The initial goal was to improve message formatting and punctuation
but after a closer look, I realized that a larger refactor was
worth it for clarity and redability.
2019-04-23 14:32:21 -07:00
Tim Abbott cb6c1e7a92 export: Fix log line about zerver_scheduledemail_users. 2019-04-23 13:52:02 -07:00
Anders Kaseorg 098e5ad3de templates: Fix more low-hanging HTML validation problems.
"http://localhost:9981/help/enable-emoticon-translations":323.13-324.31: error: The “align” attribute on the “th” element is obsolete. Use CSS instead.
"http://localhost:9981/help/enable-emoticon-translations":324.45-325.31: error: The “align” attribute on the “th” element is obsolete. Use CSS instead.
"http://localhost:9981/help/enable-emoticon-translations":329.13-330.23: error: The “align” attribute on the “td” element is obsolete. Use CSS instead.
"http://localhost:9981/help/enable-emoticon-translations":330.44-331.23: error: The “align” attribute on the “td” element is obsolete. Use CSS instead.
"http://localhost:9981/help/enable-emoticon-translations":337.6-337.9: error: Stray end tag “p”.
"http://localhost:9981/help/enable-emoticon-translations":337.6-337.9: error: Cannot recover after last error. Any further errors will be ignored.
"http://localhost:9981/api/incoming-webhooks-walkthrough":381.4-381.77: error: An “img” element must have an “alt” attribute, except under certain conditions. For details, consult guidance on providing text alternatives for images.
"http://localhost:9981/apps/":192.21-192.34: error: Bad value “” for attribute “src” on element “img”: Must be non-empty.
"http://localhost:9981/apps/":192.21-192.34: error: An “img” element must have an “alt” attribute, except under certain conditions. For details, consult guidance on providing text alternatives for images.
"http://localhost:9981/features/":225.9-225.82: error: An “img” element must have an “alt” attribute, except under certain conditions. For details, consult guidance on providing text alternatives for images.
"http://localhost:9981/features/":230.72-232.8: error: Text not allowed in element “svg” in this context.
"http://localhost:9981/features/":259.9-259.91: error: An “img” element must have an “alt” attribute, except under certain conditions. For details, consult guidance on providing text alternatives for images.

Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-04-23 13:00:59 -07:00
Tim Abbott fa37401e69 api docs: Fix typo in typing notifications docs.
notification_to should be just to here.

This resulted from the person doing the port into these docs not
understanding the full REQ API.
2019-04-23 11:24:07 -07:00
Anders Kaseorg 61982d9d47 uploads: Revert "Url encoded name of the file should be an ascii."
This reverts commit fd9dd51d16 (#1815).

The issue described does not exist in Python 3, where urllib.parse now
_only_ accepts (Unicode) str and does the right thing with it.  The
workaround was not being triggered and would have failed if it were.

Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-04-22 22:28:39 -07:00
Vishwesh Jainkuniya c007b9ea4a notifcations: Remove `user` from the payload.
This contains email of the user to whom notification is being
send. This has not been used in any past mobile releases, so it is
safe to remove it.

As user_id will be stable for the user, but not email. So it's better to
start consuming `user_id` instead of email on mobile.
2019-04-22 14:50:04 -07:00
Vishwesh Jainkuniya 447a517e6f notifications: Add `user_id` in the GCM & APNS payload.
This makes it easy to uniquely identify the user account associated
with a notification by, for example, the (realm_uri, user_id) pair.

This helps improve notifications in the mobile apps.
See https://github.com/zulip/zulip-mobile/pull/3407#discussion_r266196616

Fixes #11961.
2019-04-22 14:49:03 -07:00
Puneeth Chaganti 184c0b9870 render_markdown_path: Convert dicts to dict item tuples for caching.
Calls to `render_markdown_path` weren't getting cached since the context
argument is unhashable, and the `ignore_unhashable_lru_cache` decorator ignores
such calls. This commit adds a couple of more decorators - one which converts
dict arguments to the function to a dict items tuple, and another which converts
dict items tuple arguments back to dicts. These two decorators used along with
the `ignore_unhashable_lru_cache` decorator ensure that the calls to
`render_markdown_path` with the context dict argument are also cached.

The time to run zerver.tests.test_urls.PublicURLTest.test_public_urls drops by
about 50% from 8.4s to 4.1s with this commit. The time to run
zerver.tests.test_docs.DocPageTest.test_doc_endpoints drops by about 20% from
3.2s to 2.5s.
2019-04-22 13:53:18 -07:00
Eeshan Garg d8d9695087 webhooks/newrelic: Improve message formatting and punctuation. 2019-04-17 16:22:43 -07:00
Eeshan Garg f86e445fe6 webhooks/sentry: Improve message formatting and punctuation. 2019-04-17 16:22:43 -07:00
Eeshan Garg 424061a07a webhooks/papertrail: Improve message formatting and punctuation. 2019-04-17 16:22:43 -07:00
Eeshan Garg 1c1b440adf webhooks/pingdom: Improve message formatter and punctuation. 2019-04-17 16:22:43 -07:00
Eeshan Garg b44d352af6 webhooks/pagerduty: Use colons before quote blocks. 2019-04-17 16:22:43 -07:00
Eeshan Garg e682889437 webhooks/insping: Improve message formatting and punctuation. 2019-04-17 16:22:43 -07:00
Eeshan Garg 5fccf197d1 webhooks/greenhouse: Improve message formatting and punctuation. 2019-04-17 16:22:43 -07:00
Eeshan Garg 674fcfcce1 webhooks/semaphore: Improve message formatting and punctuation. 2019-04-17 16:22:43 -07:00
Eeshan Garg 9d79becc0e webhooks/splunk: Improve message formatting and punctuation. 2019-04-17 16:22:43 -07:00
Eeshan Garg 21cdb6decc webhooks/wordpress: Improve message formatting and punctuation. 2019-04-17 16:22:43 -07:00
Eeshan Garg 0a0df3bc58 webhooks/zabbix: Improve message formatting and punctuation. 2019-04-17 16:22:43 -07:00
Shubham Padia 7743fa5297 auth: Redirect deactivated user to /login when attempting social login. (#12130) 2019-04-17 12:28:57 -07:00
Shubham Padia 31cedbc9b8 backup: Make `--output` an optional argument instead of a positional one.
Fixes #12150.
Previously the `output` argument was positional, while the docs described
the arguments as optional in the form of `--output=...`.
2019-04-17 12:14:11 -07:00
Shubham Padia 32cf192b9b backup: Add option `--skip-uploads` to skip uploads backup. 2019-04-17 12:14:11 -07:00
Shubham Padia 984d3f2f0e backup: Add option `--skip-db` to skip database backup.
This may be relevant for organizations that use a corporate remote
postgres server and just want to backup their configuration.
2019-04-17 12:14:11 -07:00
neiljp (Neil Pilgrim) 1348c8fd05 requirements: Upgrade mypy to 0.670.
Requires minor adjustment for accented 'i' in variable name.
2019-04-16 17:50:12 -07:00