Commit Graph

7630 Commits

Author SHA1 Message Date
Tim Abbott 9f844ff681 tornado: Fix logging of tornado activity level.
This logging was apparently broken when sorting imports; it's a fairly
unique thing in our codebase that this would be a problem.  Prevent
future regressions by adding this exception explicitly to the isort
configuration.
2018-04-17 15:59:01 -07:00
Vishnu Ks cc93ac34a8 coverage: Add coverage to estimate_recent_messages.
With this message.py is fully covered and can be
removed from not_yet_fully_covered in test-backend.
2018-04-17 11:01:20 -07:00
Eeshan Garg 4fbdfef63b webhooks/stripe: Update docs to conform to new style guide. 2018-04-17 09:07:27 -07:00
Eeshan Garg dde9bb448f webhooks/circleci: Add steps instead of linking to CircleCI docs.
We let Markdown increment the list step numbers, which is more
reliable than keeping track of numbered-steps manually.

Also, instead of linking to the CircleCI docs, we now have full
instructions for how to setup a webhook by modifying the circle.yml
file.
2018-04-16 16:39:23 -07:00
Tim Abbott 5cc70675c6 webhooks: Suppress errors from very old GitLab versions.
Ancient GitLab from several years ago doesn't include the
HTTP_X_GITLAB_EVENT header (and seems to have a different format), so
we should ignore its requests.

Might be good to document the version threshhold, but it's very hard
to tell from Googling what it is.
2018-04-16 16:13:20 -07:00
Tim Abbott e2f8bc9eac /api: Fix tests for /api homepage. 2018-04-16 16:13:20 -07:00
Rhea Parekh a2070fb7e5 slack importer: Add comment on size information of avatars.
The size information of an avatar is not required during the import.
Check function 'import_uploads_local' and 'import_uploads_s3'
in 'export.py' for this.
2018-04-16 14:44:57 -07:00
Nikhil Kumar Mishra 91412e5843 test_upload: Add test for `get_realm_for_filename`. 2018-04-16 11:52:44 -07:00
Nikhil Kumar Mishra c96dc1652e test_upload: Add tests for `resize_emoji`. 2018-04-16 11:52:44 -07:00
Tim Abbott 0c30a26d81 bulk_create: Remove some long-dead code.
We used to use these in populate_db, but haven't done so in a long
time, and it doesn't seem likely that will change anytime in the
future.
2018-04-16 11:41:42 -07:00
Ben Reeves fdfbd45208 soft_deactivation: Change `<` to `<=` in add_missing_messages.
We should still short-circuit the iteration in
`add_missing_messages` if the unsubscription was the last
thing to happen to the user before unsubscription and
soft deactivation.
2018-04-16 11:28:08 -07:00
Alyssa Wagenmaker d4e5777296 tests: Test user unsubscribing before soft deactivation.
Brings lib/soft_deactivation.py up to 100% test coverage.

Improves: #7089.
2018-04-16 11:28:08 -07:00
Rhea Parekh f6b6aa1e75 slack import: Implement threading as a management command. 2018-04-15 19:53:02 +05:30
Rhea Parekh 7c0c3930a8 slack importer: Thread avatar downloads. 2018-04-15 19:53:01 +05:30
Rhea Parekh ebc2ee28e9 slack importer: Thread emoji downloads. 2018-04-15 19:52:59 +05:30
Rhea Parekh 8a291d0232 slack importer: Thread attachment downloads.
Use Zulip's run_parallel method to run thread downloads.
2018-04-15 19:51:58 +05:30
Eeshan Garg 6d86c83966 webhooks/solano: Update docs to conform to style guide. 2018-04-14 09:38:22 -07:00
Eeshan Garg eec7e17e70 webhook/raygun: Update docs to conform to style guide. 2018-04-14 09:38:22 -07:00
Eeshan Garg c51a3dce62 webhooks/pivotal: Update docs to conform to style guide. 2018-04-14 09:38:22 -07:00
Eeshan Garg 911b9582bd webhooks/opbeat: Update docs to conform to style guide. 2018-04-14 09:38:22 -07:00
Eeshan Garg 3e0eb9530c webhooks: Remove the Facebook integration.
Rishi and I decided that it makes sense to get rid of the Facebook
integration for a few reasons, some of which are:

* The setup process is too complicated on Facebook's end. The users
  will surely have to browse Facebook's huge API reference before even
  having a vague idea of what they want.
* Slack chooses not to have a Facebook integration, but relies on
  Zapier for it. Zaps that integrate with Facebook are much more
  streamlined and the setup process isn't as much of a pain. Zapier's
  Facebook Zaps are much more fine-tuned and there are different Zaps
  for different parts of the FB API, a luxury that would likely span
  2K+ lines of code on our end if we were to implement it from
  scratch. So, I think we should relegate integration with Facebook to
  Zapier as well!
* After thoroughly testing the setup process, we concluded that the
  person who submitted the FB integration didn't really test it
  thoroughly because there were some gaping holes in the docs (missing
  steps, user permissions, etc.).
2018-04-14 09:38:22 -07:00
Tim Abbott 5ddf2614f0 uploads: Add new way of querying for mobile uploads endpoint.
This extends the /user_uploads API endpoint to support passing the
authentication credentials via the URL, not the HTTP_AUTHORIZATION
headers.  This is an important workaround for the fact that React
Native's Webview system doesn't support setting HTTP_AUTHORIZATION;
the app will be responsible for rewriting URLs for uploaded files
directly to add this parameter.
2018-04-13 17:51:45 -07:00
Tim Abbott db830c4085 bugdown: Replace link to old pre-open-source Zulip trac. 2018-04-13 08:41:44 -07:00
Rohitt Vashishtha 9e7929417d markdown: Increase rendered_content length limit.
This commit increases the rendered_content limit from 2x to 10x of the
original message length.

Earlier, we had placed a limit of MAX_MESSAGE_LENGTH * 2 for the
rendered content (explained in commit
77addc5456).  That limit was based on
the assumption that in most cases, the rendered content wouldn't cause
a large increase in message length. However, quite prominently in
syntax highlighted codeblocks, that wasn't true and this caused the
limit condition to be hit for long messages composed primarily of code
blocks.

Example: The following message would render close to 10x it's original size.

```py
if:
def:
print("x", var)
x = y
```

Because the syntax highlighted logic is extremely compressible, having
rendered_content reach up to 100KB doesn't create a network
performance problem.
2018-04-13 08:39:51 -07:00
Rohitt Vashishtha 3bdc8bbaa5 CVE-2018-9986: Fix XSS issues with frontend markdown processor.
This fixes a set of XSS issues with Zulip's frontend markdown
processor, which is used in a limited set of contexts, such as local
echo of messages and the drafts feature.

The implementation of several syntax elements, including the <em>
syntax, user and stream mentions, and some others failed to properly
escape the content inside the syntax.

Fix this, and add tests for each corrected code path.

Thanks to w2w for reporting this issue.
2018-04-12 09:46:37 -07:00
Tim Abbott e6833b6427 cleanup: Remove the legacy Dropbox file upload integration.
This has been hard-disabled for years, we have no plans to re-enable
it, and it has some hacky code in it.
2018-04-11 11:39:48 -07:00
Aditya Bansal 6c1a50da76 csp_reports: Add endpoint to handle logging of reports sent by clients. 2018-04-11 23:01:13 +05:30
Vishwesh Jainkuniya 383c62fb03 dev_login: Identify each user's realm when listing them.
This is a mobile-specific endpoint used for logging into a dev server.
On mobile without this realm_uri it's impossible to send a login request
to the corresponding realm on the dev server and proceed further; we can
only guess, which doesn't work for using multiple realms.

Also rename the endpoint to reflect the additional data.

Testing Plan:
Sent a request to the endpoint, and inspected the result.

[greg: renamed function to match, squashed renames with data change,
 and adjusted commit message.]
2018-04-10 17:03:36 -07:00
Rhea Parekh f7398cbb09 slack import: Implement custom profile fields.
Add custom profile fields in the slack converted
data 'realm' file.
Added tests for the custom profile fields.

Fixes #8928
2018-04-10 13:28:53 -07:00
Rhea Parekh 852e8516b4 slack import: Add custom profile fields.
Build CustomProfileField and CustomProfileFieldValue
for every user and process the field type after getting an
entire list of the custom fields.
2018-04-10 13:28:53 -07:00
Rhea Parekh ccefaf7b26 scripts: Remove the depreciated script 'postgres-reset-sequences'. 2018-04-10 13:07:14 -07:00
rht a183186672 slack importer: User session.get to recycle previous connections. 2018-04-09 22:02:01 -07:00
Shubham Dhama b650b6b38c markdown: Add @stream as an alias for @all.
Fixes: #8930.
2018-04-09 16:35:14 -07:00
rht 7a8655cc50 Slack importer: Add test for Slack channel mention to Zulip stream mention. 2018-04-09 10:47:39 -07:00
rht 630adb406b Slack importer: Map Slack channel mentions to Zulip stream mentions. 2018-04-09 10:47:39 -07:00
Rhea Parekh 035c440ff3 import script: Support import custom profile fields.
Import of Custom profile fields is only supported for slack
import script for now.
2018-04-09 10:45:35 -07:00
Tim Abbott c41d7ee300 slack import: Write reasonable multi-line JSON.
This is a lot better for debugging.
2018-04-09 10:45:35 -07:00
Rhea Parekh ed7127c8b4 import script: Delete medium sized avatars if it exists.
Deletion of medium sized image is done if it exists before calling the
function 'ensure_medium_avatar_image', to avoid potentially confusing
problems with left-over medium-size avatar images from a previous run
being used when repeatedly importing the same realm in a development
environment..

Fixes #8949.
2018-04-08 07:04:24 -07:00
rht f6feac1316 Slack importer: Map Slack command for mentions to Zulip's all.
Fixes #9003.
2018-04-07 20:02:39 -07:00
Rhea Parekh e037c2f93e import script: Fix upload links.
Rendered content is None for Slack imports, hence it is replaced only
for Zulip->Zulip imports.

Fixes #8959.
2018-04-07 20:01:20 -07:00
Rhea Parekh b3f951d2cf import script: User profile ids should be allocated before allocating bot ids. 2018-04-07 13:28:33 +05:30
Vishnu Ks e92838a31f registration: Catch email validation error and show error message. 2018-04-06 15:18:32 -07:00
Shubham Dhama 0e6757af5c org settings: Change default realm description to empty string.
This fixes a traceback that users would get when editing the realm
description just after creating a new organization.
2018-04-06 15:15:47 -07:00
Rhea Parekh 2baa9bc16e Import: Add subdomain in the import script.
Also remove user input of subdomain in the slack data
conversion script.
2018-04-06 09:12:56 -07:00
Tim Abbott ad861c5fae messages: Improve comment on need_messages. 2018-04-06 08:57:46 -07:00
Eeshan Garg 42bbfea775 webhooks/splunk: Update docs to conform to style guide. 2018-04-05 23:28:27 -07:00
Eeshan Garg 7b1ce446cf webhook/opsgenie: Update docs to conform to style guide. 2018-04-05 22:47:22 -07:00
Eeshan Garg 2e700477e3 webhooks/groove: Update docs to conform to style guide. 2018-04-05 22:47:21 -07:00
Eeshan Garg 902ab01785 webhooks/trello: Update docs to conform to style guide.
This one is one of the most tedious to set up and get working.

We now also rely on the Trello scripts available as part of the
`python-zulip-api/zulip` API package to make the setup process
easier.
2018-04-05 14:48:37 -07:00
Tim Abbott b0b134cb4c help: Clean up settings_html and subscriptions_html.
After some thinking, I don't think there's any actual value to doing
the ../ style relative links here, whereas there is actual harm from
the links being slightly broken in the current model.  We fix this by
just using /#settings as the URL.

Fixes #8978.
2018-04-05 14:48:26 -07:00