Commit Graph

3397 Commits

Author SHA1 Message Date
Tomasz Kolek a79acf854f docs: Automate creation of Hubot documentation lozenges.
This removes a bunch of semi-duplicated code.
2016-11-26 15:27:54 -08:00
Rishi Gupta 4b183cd526 domain migration: Remove several instances of get_realm.
Remove the easy to remove instances of get_realm.
2016-11-26 15:19:56 -08:00
Umair Khan 0536aeba4d Django 1.10: Use same cache prefix for JS tests.
Previously, the key prefix was based on the process id due to which
the JS tests couldn't properly flush user profiles from the cache as
our application spans over multiple processes. This problem becomes
apparent when in json_change_settings view after changing the user_profile
the tornado views continue to get the cached user profile corresponding
to their process id.
2016-11-26 15:10:50 -08:00
Umair Khan 671c45524b Django 1.10: Use new app discovery system.
'django.db.models.loading' is removed due to which both
'django.db.models.get_app' and 'django.db.models.get_models' are removed.

See https://docs.djangoproject.com/en/1.10/releases/1.9/#features-removed-in-1-9
2016-11-26 15:04:20 -08:00
Umair Khan c8fa25ab88 Django 1.10: Update session hash when password is changed.
Ref: https://docs.djangoproject.com/en/1.10/topics/auth/default/#session-invalidation-on-password-change for details.
2016-11-26 15:04:17 -08:00
Calvin Lee 7a3ef2b0eb tests: Add assert_in_success_response in ZulipTestCase.
Clean up the instances of self.assertIn("string", result.content.decode("utf-8")),
and replace them with self.assert_in_response("string").

Fixes: #2313
2016-11-26 14:52:44 -08:00
Steve Howell 6ed1dc9341 Create api docs from url coverage data from tests.
I also retire tools/analyze-url-coverage in this commit,
since the API docs cover most of the functionality.
2016-11-26 14:49:23 -08:00
Anders Kaseorg 207cf6302b Always start python via shebang lines.
This is preparation for supporting using Python 3 in production.

Signed-off-by: Anders Kaseorg <andersk@mit.edu>
2016-11-26 14:46:37 -08:00
Anders Kaseorg 573ec14955 Remove shebang line from non-scripts
Signed-off-by: Anders Kaseorg <andersk@mit.edu>
2016-11-26 13:20:22 -08:00
Steve Howell 78474c8bee Add test users with non-lowercase emails.
We are prone to case-sensitivity bugs, so I added AARON and ZOE.
Also, for good measure, I insert them in non-alphabetical order
to try to drive out bugs from non-consistent sorting of user ids.
2016-11-26 11:48:52 -08:00
Steve Howell 92030204bb Look for untested urls more exhaustively. 2016-11-26 09:52:37 -08:00
Steve Howell 87bcbc9616 Add test_generate_204 for /api/v1/generate_204.
This endpoint is apparently needed by the desktop app.
2016-11-26 09:52:37 -08:00
Steve Howell 1d87d6fd86 Add test_refer_friend(). 2016-11-26 09:52:37 -08:00
Steve Howell 1527823b9f code cleanup: Remove some unused variables in test classes. 2016-11-26 09:52:37 -08:00
Rishi Gupta c5d3c09a4a mypy: Change six.text_type to typing.Text for models.py. 2016-11-25 13:55:38 -08:00
Rishi Gupta 17f71befb4 mypy: Change six.text_type to typing.Text for a few files.
Preparation for GCI.
2016-11-25 13:55:38 -08:00
Steve Howell 5f5e6b6d83 tests: Enforce 100% URL coverage.
We now instrument URL coverage whenever you run the back end tests,
and if you run the full suite and fail to test all endpoints, we
exit with a non-zero exit code and report failures to you.

If you are running just a subset of the test suite, you'll still
be able to see var/url_coverage.txt, which has some useful info.

With some tweaks to the output from tabbott.

Fixes #1441.
2016-11-18 18:10:29 -08:00
Tomasz Kolek d8dee522b6 Fix trello integration by adding handling HEAD confirmation request.
Previously, we rejected the HEAD requests that the trello integration
uses to check if the server accepts the integration.

Add decorator for returning 200 status code if request is HEAD.

Fixes: #2311.
2016-11-18 16:27:16 -08:00
Tim Abbott fe837e58b4 test_realm_filters: Fix missing mypy annotation. 2016-11-17 20:29:06 -08:00
Vladislav Manchev d7e1e4a2c0 Add initial implementation of custom realm filters.
This PR was abandoned by Vladislav and then substantially modified by
Igor Tokarev and Tim Abbott to complete it and fix a number of bugs.

Fixes #544.
2016-11-17 17:11:25 -08:00
paxapy 012ec2beda test-backend: Use a fixed whitelist of shallow-tested templates.
This enforces the requirement that all templates not on this list are
tested.  Part of #1677.
2016-11-17 11:34:50 -08:00
Steve Howell ca43cbc654 logging: Reducing logging in run-dev.py for queue workers.
In dev, we no longer log that individual queue workers were launched.

Instead, in dev (and prod as well), we log a message with the total
count.
2016-11-17 11:12:02 -08:00
hackerkid cdb716aa63 bugdown: Add explanation for unicode-emoji JS regex. 2016-11-17 10:44:07 -08:00
Steve Howell 3a8282ff44 Add test coverage for /notify_tornado.
We now have test coverage for all of our endpoints.
2016-11-17 10:11:46 -08:00
Pweaver (Paul Weaver) 8f05ea877d lint: Fix pep8 linter error regressions.
Since `lint-all --pep8` wasn't running in CI, a few errors have leaked
in.
2016-11-17 00:09:21 -08:00
Tim Abbott e6f4cb2635 attachment: Remove DB limit on length of filenames.
This fixes an issue where we threw on error on files with long names.
2016-11-15 21:12:05 -08:00
Tim Abbott fa8480bb62 subdomains: Add option to make subdomains homepage not a login form. 2016-11-15 19:05:06 -08:00
reyha 9f8630fed8 tests: Access realms by string_id.
Replaces get_realm() function by get_realm_by_string_id() function
in zerver/tests.

Fixes #2226.
2016-11-15 13:32:20 -08:00
Tomasz Kolek 074571dddc Fix English grammar in push/remove tag message. 2016-11-15 10:48:12 -08:00
Tomasz Kolek 1feedbe346 Add github webhook integration. 2016-11-15 10:48:12 -08:00
Umair Khan 2314adc4fc Django 1.10: Compare CSRF tokens after unsalting them.
For reference: https://docs.djangoproject.com/en/1.10/ref/csrf/#how-it-works
2016-11-15 09:34:08 -08:00
Umair Khan 29254142f6 Django 1.10: Run maybe_update_realm_filter when it's needed. 2016-11-15 09:34:08 -08:00
Umair Khan 5af50db430 Django 1.10: Fix client cache while running tests.
Django reverts all the changes after running a test but the
client cache retained the deleted value, this caused the
subsequent tests to fail due to invalid foreign key constraints.
This commit fixes the issue by prefixing the cache name in
client cache with KEY_PREFIX which is bounced after every test.
2016-11-15 09:34:08 -08:00
hackerkid 67120d7a9d Support backend unicode-emoji rendering without whitespace. 2016-11-14 21:02:18 -08:00
Tim Abbott 8a5ae2893e send_pm_if_empty_stream: Clean up unnecessary nesting.
This has no functional changes.
2016-11-14 20:40:05 -08:00
Tim Abbott b09c2369c1 send_pm_if_empty_stream: Use realm object to check cross-realm.
The previous check didn't work correctly in the event that the target
stream did not exist.
2016-11-14 20:36:46 -08:00
Tim Abbott 4cfcc70e78 test_messages: Expand tests for send_pm_if_empty_stream.
This would have caught the last issue we had with this code path.
2016-11-14 20:32:52 -08:00
Tim Abbott af675998d5 Fix send_pm_if_empty_stream with nonexistent streams. 2016-11-14 20:32:29 -08:00
Umair Khan 9b0f9bf326 Django 1.10: apply_response_fixes was removed.
Ref: https://code.djangoproject.com/ticket/26052?cversion=0&cnum_hist=2
2016-11-14 16:09:12 -08:00
Umair Khan 5cfd54994a Django 1.10: validate is replaced with check.
See
https://docs.djangoproject.com/en/dev/releases/1.7/#modeladmin-validators
for explanation.
2016-11-14 16:09:12 -08:00
hackerkid b2e91eae13 Add marked.js support to emojis in miscellaneous and dingbats block.
https://en.wikipedia.org/wiki/Emoji#Unicode_blocks
2016-11-14 15:58:16 -08:00
Steve Howell fad0a86554 Add HelpTest for help pages. 2016-11-14 15:47:35 -08:00
Steve Howell 02cf40ade8 Add test_redirect_endpoints() in test_signup.py. 2016-11-14 15:47:35 -08:00
hackerkid 8c83b34ad5 Support backend emoji rendering without whitespace. 2016-11-11 16:18:57 -08:00
Feorlen 4ed5c21821 Update Desk.com webhook integration.
Updates the HTML docs to match changes to the Desk.com website,
including all new screenshots for the custom action workflow.

Tests four types of messages that could be sent as notifications from
Desk.com. Desk.com allows an account administrator to send any text
in a custom action, so there isn't a standard format.

Custom actions send URL-encoded POST data, the test fixtures contain
URL-encoded text like what could be sent by a custom action configured
as described in the Zulip Integrations documentation to post a new
message to a stream. (See also #2169, errors in this documentation.)

New zerver.tests.webhooks.test_deskdotcom.DeskDotComHookTests:

* Static text: minimal plain text string
* Case updated: activity alert with link to a Desk.com case and message
* Unicode text Italian: activity alert with message in Italian
* Unicode text Japanese: activity alert with message in Japanese

Each posts a new message in the deskdotcom stream.

Tested on Ubuntu 14.04. I created the fixtures with Emacs, I would
appreciate if someone can check that the Italian and Japanese messages
look ok. I used the same text for a live test and it displayed correctly.

Fixes #2031
2016-11-11 16:06:22 -08:00
Tomasz Kolek 4659abae65 test_fixtures: handle a case when given database does not exist.
Sometimes i.e. when you force close (ctrl-c) during ./tools/test-backend
a next execution might raise an error becuase DATBASE_NAME does not exist.
2016-11-11 15:40:10 -08:00
Tomasz Kolek f05b71b9ca Add handling multiple push info in one request to Bitbucket2 integration. 2016-11-11 15:36:45 -08:00
Tomasz Kolek 085cdb7028 Fix Bitbucket2 push message by truncated commits list. 2016-11-11 15:36:45 -08:00
Tomasz Kolek ed1d7f7997 Add is_truncated param to git integrations push commits event.
Add is_truncated param for cases when we know that there were more commits
but we dont know actual number and want just inform about existing of the rest.
2016-11-11 15:36:45 -08:00
Tomasz Kolek 81f18b829d Normalize pushing tags event messages in Bitbucket2 integration. 2016-11-11 15:36:45 -08:00
Tomasz Kolek 5980a75e07 Normalize pushing tags event messages in GitLab integration. 2016-11-11 15:36:45 -08:00
Tomasz Kolek 6be27a1396 Add get_push_tag_event_message function to git integrations common (git.py). 2016-11-11 15:36:45 -08:00
Rishi Gupta 5681ee90fa models.py: Rename split_email_to_domain to email_to_domain.
In order to better match models.email_to_username.
2016-11-11 15:26:51 -08:00
Rishi Gupta c1713c9659 Prevent code from using email domain to determine realm when subdomains.
Also removes the intermediate step of going through Realm.domain in the
non-subdomains case. Part of a larger project to remove Realm.domain
entirely.
2016-11-11 15:26:51 -08:00
Rishi Gupta 411a7b08e0 bulk_create_users: No longer de-alias user email addresses.
bulk_create.bulk_create_users is only called by initialize_voyager_db.py and
populate_db.py, both of which only have realms with single domains.

Part of a larger project to remove the Realm.domain field.
2016-11-11 15:26:51 -08:00
Rishi Gupta 91f2582c2b templates: Remove for_you block from jinja templates.
No change in behavior. The for_you block had already been removed in
portico.html long ago in a6889080ce. The
contents of the block are still present in the non-portico 404.html
and 5xx.html error pages.
2016-11-11 15:26:08 -08:00
Rishi Gupta 6608627bc0 models.py: Move Realm and RealmAlias functions to appropriate locations.
Currently refactoring the Realm and RealmAlias portions of models.py to no
longer use Realm.domain. Move various functions related and not-related to
the two model objects to more conventional places in the file.

No change to behavior.
2016-11-11 15:24:42 -08:00
Tim Abbott 48f671a314 same_realm_jabber_user: Remove old ist.mit.edu hack.
This is no longer in use.
2016-11-10 20:50:40 -08:00
Tim Abbott 3b145b2370 add_new_user_history: Fix ordering of created UserMessage rows.
This has no functional effect because user_message IDs aren't used,
but will make the database cleaner.
2016-11-10 20:39:34 -08:00
Tim Abbott 08ca209aed add_new_user_history: Fix race with new messages arriving during signup.
Previously, if a new message arrived between when a user is subscribed
to the default streams and when the user's initial messages are
queried, we would try to create two UserMessage rows for the same
Message, resulting in an IntegrityError crash.  We fix this and add a
test for that race condition.
2016-11-10 20:38:27 -08:00
Tim Abbott 80c2df616e signup: Extract add_new_user_history. 2016-11-10 20:37:03 -08:00
sinwar 4582a98c09 tests: Split out ZulipTestCase and WebhookTestCase to a separate file.
Fixes #1671.
2016-11-10 19:29:43 -08:00
Igor Tokarev 852bc6b491 bugdown: Re-enable support for italics in bugdown.
Fixes: #1103.
2016-11-10 19:14:40 -08:00
Rishi Gupta 3c75ef2d79 Allow users to submit empty content when editing messages.
The closest we have to deleting a message. Was original behavior, but was
changed in 09754c9.
2016-11-10 16:45:18 -08:00
paxapy 4438a09b30 Add migration to create attachments for old uploads.
This migration ensures that all historically uploaded files from
before we started tracking files in the Zulip database via the
Attachment model have Attachment objects.

This has been tested by tabbott against a production server to ensure
that it results in all old uploaded files having corresponding
attachment objects.

Merging this change is a key prerequisite for making our adding
attachment access controls in an enforcing fashion.
2016-11-10 12:14:08 -08:00
Umair Khan 86b75aade2 Django 1.10: Fix i18n tests.
SimpleCookie does not accept unicode strings and the name of
Simplified Chinese is changed to zh-HANS.
2016-11-10 16:20:03 +05:00
Umair Khan ee3ec96f38 Django 1.10: Suppress logs. 2016-11-10 16:20:03 +05:00
Umair Khan 1e91b946d9 Django 1.10: Sign google oauth requests using csrf token.
In Django 1.10, the get_token function returns a salted version of
csrf token which changes whenever get_token is called. This gives
us wrong result when we compare the state after returning from
Google authentication servers. The solution is to unsalt the token
and use that token to find the HMAC so that we get the same value
as long as t he token is same.
2016-11-10 16:20:03 +05:00
Umair Khan 010ab0637d Django 1.10: Upgrade integration urls. 2016-11-10 16:20:03 +05:00
Tim Abbott 02855498e9 render_markdown_path: Add fenced code and codehilite support.
This also causes us to only initialize the extensions once per process.
2016-11-09 21:39:55 -08:00
Tim Abbott 08043ff9a0 render_markdown_path: Add anchors for table-of-contents. 2016-11-09 21:39:55 -08:00
Tim Abbott 0568ade496 render_markdown_path: Decode UTF-8 after reading from disk.
This is required to support unicode characters in our markdown filesq.
2016-11-09 21:39:55 -08:00
Tim Abbott cf2007d4e0 docs: Add a simple markdown-based help center.
The plan is to use this for adding user documentation to Zulip.
2016-11-09 21:39:55 -08:00
Tim Abbott a715f083bb render_markdown_path: Allow HTML in templates. 2016-11-09 21:12:44 -08:00
Tim Abbott d9b1a5a417 render_markdown_path: Don't cache in development.
That caching made the edit/refresh cycle for docs annoying.
2016-11-09 21:12:44 -08:00
Rishi Gupta 1c869c2287 models.email_allowed_for_realm: Use RealmAlias instead of domain. 2016-11-09 16:31:48 -08:00
Rishi Gupta 70ab79d056 accounts_register: Apply email_allowed_for_realm check to all code paths.
Needed in case the user was allowed to join the realm when they got the
confirmation email, but is no longer allowed to do so. Check was previously
applied to invited users (those with a prereg_user.referred_by), and is now
applied regardless of how they get to accounts_register.
2016-11-09 16:31:48 -08:00
Rishi Gupta d5f99c4438 views.accounts_register: Remove almost all dependence on domain. 2016-11-09 16:31:48 -08:00
Rishi Gupta 823915fa46 UserSignUpTest: Add tests for forms.HomepageForm. 2016-11-09 16:31:48 -08:00
Rishi Gupta 1e6ae537bd Remove special flow for open realm sign-up when realms have subdomains.
Redirects /register/<domain> to /accounts/register when
REALMS_HAVE_SUBDOMAINS.
2016-11-09 16:31:48 -08:00
Rishi Gupta b114690bd5 forms.HomepageForm: Rename subdomain to string_id.
No change to behavior.
2016-11-09 16:31:48 -08:00
Rishi Gupta 38f1ab325c forms.HomepageForm: No longer take a domain argument.
domain was unused. No change to behavior.
2016-11-09 16:31:48 -08:00
Rishi Gupta ff2fe0cf92 forms.HomepageForm: Improve error messages.
No change to behavior.
2016-11-09 16:31:48 -08:00
Rishi Gupta d1acc54b2f forms.HomepageForm: Stop users from signing up for realms they can't join.
We do this check in accounts_register as well, but might as well tell the
user now not to get their hopes up.
2016-11-09 16:31:48 -08:00
Rishi Gupta 82b60ad29e forms.HomepageForm: Disallow mit mailing lists from open mirroring realms.
I'm guessing the old behavior was unintended, rather than policy.
2016-11-09 16:31:48 -08:00
Rishi Gupta 78dee73511 forms.HomepageForm: Consolidate check for open realm.
Does change/fix behavior in various corner cases when the domain passed in
to HomepageForm and subdomain passed in to HomepageForm correspond to
different realms.
2016-11-09 16:31:48 -08:00
Rishi Gupta 01526677c4 forms.HomepageForm: Use string_id to determine intended signup realm.
If the user comes in to HomepageForm with a set subdomain, use that to
determine the signup realm instead of the email address.

In the non-REALMS_HAVE_SUBDOMAINS case, still allow using the email address
if no subdomain is passed.
2016-11-09 16:31:48 -08:00
Rishi Gupta eddd8fb5fe forms.HomepageForm: Reorder email checks for future edits.
No change to behavior.
2016-11-09 16:31:48 -08:00
Rishi Gupta 6e517b2dec models.py: Disable unique_open_realm feature when realms have subdomains. 2016-11-09 16:31:48 -08:00
Rishi Gupta 35481014a4 views.create_homepage_form: Ensure domain and subdomain match.
Ensure domain and subdomain correspond to the same realm when being passed
to forms.HomepageForm. Previously this was not the case when e.g. we got
here via the /register/<domain> endpoint.

This also effectively disables the register/<domain> endpoint when
REALMS_HAVE_SUBDOMAINS, or rather, foo.server.org/register/bar.com will try
to register you for the realm with string_id foo rather than realm with
domain bar.com.
2016-11-09 16:31:48 -08:00
Rishi Gupta 60b5b82b57 forms.py: Merge get_valid_realm into caller.
No change to behavior.
2016-11-09 16:31:48 -08:00
Tim Abbott f6d3c63a01 realm_alias: Access realm alias objects in lower-case. 2016-11-09 16:31:48 -08:00
Umair Khan cfded8b5af Django 1.10: Resolve QuerySet returned by model_to_dict.
Django 1.10 resolves ManyToManyField into a QuerySet in model_to_dict.
This commit further resolves the QuerySet to the primary keys.
2016-11-09 15:29:58 -08:00
Umair Khan f78752466c Django 1.10: URLs in response contain just the path.
The response object of the Django test client only contains the
path of the url.
2016-11-09 15:26:07 -08:00
Umair Khan dd174b537f Django 1.10: Update code according to new get_user function.
`django.contrib.auth.get_user` function is updated in Django 1.10, due to
which everytime we update the password of the user the password hash changes.
This causes authentication failure. Previously, our code worked correctly
because we use our own session middleware and the `get_user` code had a
conditional statement which allowed our code to bypass the authentication
code.
2016-11-09 15:26:07 -08:00
paxapy 6d93b3b60c lint: Fix E703 pep8 violations. 2016-11-09 15:18:35 -08:00
Steve Howell b5ef679360 Add get_topic_history_for_stream(). 2016-11-09 13:56:27 -08:00
Steve Howell 89b2e51129 Extract is_active_subscriber() helper for streams views. 2016-11-09 13:56:27 -08:00
Steve Howell a03a7d4c95 peer_remove: Send user_id, not email, for unsubscribe events. 2016-11-08 15:36:29 -08:00
Tim Abbott 3caee213e1 test_render_mention_stream_api: Fix nondeterministic failures.
This previously dependended on the stream ID for Denmark, which is not
a constant across all installations.
2016-11-08 14:13:48 -08:00