Commit Graph

3113 Commits

Author SHA1 Message Date
Tim Abbott 56d0cc69e9 socket: Remove unused djsession_engine. 2016-10-17 23:19:28 -07:00
Mohsen Ibrahim bfe5787197 Fix Django.utils.importlib deprecation warnings.
This changes `from django.utils.importlib import import_module` to
`from importlib import import_module`, as `django.utils.importlib` will
be removed in django version 1.9.
2016-10-17 23:19:28 -07:00
Mohsen Ibrahim e9899e4dc9 bugdown: Fix some Python-Markdown deprecation warnings.
Python-Markdown changed how one is supposed to declare extensions
being used.
2016-10-17 23:13:41 -07:00
Tim Abbott 63a49be09e event_queue: Import tornado.ioloop correctly.
We actually use tornado.ioloop, not the root tornado module, and while
the current code does work, it causes mypy errors with the incremental
cache.
2016-10-17 17:57:46 -07:00
Tim Abbott 4a4664d268 mypy: Remove a bunch of now-unnecessary type: ignore annotations.
Since mypy and typeshed have advanced a lot over the last several
months, we no longer need these `type: ignore` annotations.
2016-10-17 11:48:34 -07:00
Tim Abbott 3b0b65c2d0 management: Fix recent argument parsing regressions.
Fixes a regression introduced in
fbdf539bf2.
2016-10-16 20:36:33 -07:00
Laura Hampton fbdf539bf2 Replace optparse with argparse in the several management commands.
Specifically:
* fill_memcached_caches.py
* initialize_voyager_db.py
* logout_all_users.py
2016-10-16 17:08:35 -07:00
Tim Abbott 2bea0c46fe HomeTest: Add a basic test for desktop_home. 2016-10-16 11:39:08 -07:00
Tim Abbott 3ee9e2c45f tests: Add some basic i18n public URL tests. 2016-10-16 11:26:01 -07:00
Tim Abbott 122164fffb DocPageTest: add test for /about/ page. 2016-10-16 11:19:27 -07:00
Tim Abbott e37b6488b8 Bugdown: Remove checked-in CodeHilite extension.
The changes that required us to fork this extension had been merged
into upstream CodeHilite, so we can remove it and switch to using the
version that comes with python-markdown.
2016-10-16 00:40:08 -07:00
umkay c028815385 bugdown: Update python-markdown to version 2.6.7.
This updates Bugdown to reflect the changes in the updated
markdown. In particular, we now pass a default config object in the
__init__ for the Bugdown extension, update the make_md_engine function
to take kwargs as opposed to a config list, and have UListProcessor
inherit from ulist as opposed to olist (which no longer works).

We update the (forked from upstream) fenced_code extension's
makeExtension to take args and kwargs, and update
FencedBlockPreprocessor __init__ method with updated Codehilite
arguments.

We update the (forked from upstream) Codehilite extension to
mirror the logic with the latest upstream Codehilite:
	Add parse_hl_lines function
	update makeExtension to take args and kwarfs instead of config
	list
	Add regex for highlight lines
	use linenums instead of linenos
	use get_formatter_by_name instead of HtmlFormatter
	user get_lexer_by_name instead of TextLexer
	add hl_lines and use_pygments arguments to the codehlite
	constructor
2016-10-15 23:38:32 -07:00
Tomasz Kolek 92bfadcca4 Normalize github pull requests event message.
Add detailed info (description, source and target branch, assignee) to message.
Change subject to 'repo_name / PR #id title'.
Modify some test fixtures for better coverage.
2016-10-14 16:54:13 -07:00
Tomasz Kolek ce9ac30fac Normalize bitbucket pull requests event message.
Add detailed info (description, source and target branch, assignee) to message.
Change subject to 'repo_name / PR #id title'.
Modify some test fixtures for better coverage.
2016-10-14 16:54:13 -07:00
Tomasz Kolek 421c51cc14 Normalize gitlab pull requests event message.
Add detailed info (description, source and target branch, assignee) to message.
Change subject to 'repo_name / MR #id title'.
Modify some test fixtures for better coverage.

Fixes: #1883.
2016-10-14 16:54:13 -07:00
Tomasz Kolek 2941c1f517 Move converting commits data to text to separate function.
Create get_commits_content function for converting
commits list to text.
2016-10-14 16:54:13 -07:00
Tomasz Kolek d2cedd3667 Rename consts in lib/webhooks/git.py to make it more general.
Rename:
   PUSH_COMMITS_LIMIT to COMMITS_LIMIT
   PUSH_COMMIT_ROW_TEMPLATE to COMMIT_ROW_TEMPLATE
   PUSH_COMMITS_MORE_THAN_LIMIT_TEMPLATE to COMMITS_MORE_THAN_LIMIT_TEMPLATE
2016-10-14 16:54:13 -07:00
Tomasz Kolek ee70cc5141 Add git integration method to build message for pull requests events. 2016-10-14 16:54:13 -07:00
Tomasz Kolek 0f01150449 Add bitbucket tests for push, push a lot of commits and force push events. 2016-10-14 16:54:13 -07:00
Tomasz Kolek 165cdb79d9 Rename bitbucket2 fixtures directory to bitbucket.
Rename all bitbucket fixtrues from bitbucket2.* to bitbucket_v2.*.
2016-10-14 16:54:13 -07:00
umkay 278574a6ec Fix cursor() arguments for connection classes.
In order to use the latest version of psycopg2 with the latest version
of sqlalchemy we must integrate the changes made in the 2.4.6 release of
psycopg2. See
c86ca7687f
and the release notes for psycopg2 2.4.6.
2016-10-14 13:35:06 -07:00
Rishi Gupta 82b814a1cd analytics: Simplify frequency and measurement interval options.
Change the CountStat object to take an is_gauge variable instead of a
smallest_interval variable. Previously, (smallest_interval, frequency)
could be any of (hour, hour), (hour, day), (hour, gauge), (day, hour),
(day, day), or (day, gauge).
The current change is equivalent to excluding (hour, day) and (day, hour)
from the list above.

This change, along with other recent changes, allows us to simplify how we
handle time intervals. This commit also removes the TimeInterval object.
2016-10-14 10:18:37 -07:00
Tim Abbott 8f145e03cf views: split events_register.py from main views file. 2016-10-11 21:27:06 -07:00
Tim Abbott 0c02015541 views: Move one-click unsubscribe to its own file. 2016-10-11 21:27:06 -07:00
Tim Abbott 5b7dfafcac views: Move json_refer_friend to invite.py. 2016-10-11 21:27:06 -07:00
Tim Abbott dd352dd456 views: Move invitation views to their own file. 2016-10-11 21:27:06 -07:00
Tim Abbott 67d9e19ccf views: Split views/auth.py out of core views file. 2016-10-11 21:27:06 -07:00
Tim Abbott c886bd9065 do_update_message_flags: test starring historical messages. 2016-10-11 17:38:39 -07:00
Tim Abbott 0f1468e64f do_update_message_flags: Use access_message for check. 2016-10-11 17:20:15 -07:00
Tim Abbott f1a399a4e1 message: Create new access_message library function.
With reactions and other upcoming features, we'll be adding several
places where we need to check whether a particular user can access a
particular message.  It's best to just have a single helper function
for this purpose that we can use everywhere.
2016-10-11 17:17:19 -07:00
Tim Abbott d2e41ff48e json_fetch_raw_message: Allow fetching messages you received.
This also modifies the tests to verify that the user in question has
access to the relevant message.
2016-10-11 17:17:19 -07:00
Umair Khan 36c6a5d758 Add support for changing user's name by admin.
Fixes: #1553
2016-10-11 16:36:48 -07:00
Umair Khan b886e38a58 Increase test quality for GitHubAuthBackendTest. 2016-10-11 16:19:55 -07:00
Umair Khan 4fcd36b124 Add negative tests for Social auth backend.
Fixes: #1948
2016-10-11 16:19:55 -07:00
Rishi Gupta d75731f988 Realm creation flow: Remove invite page.
Previously, we sent users to an "invite your friends" page after they
created an organization. This commit removes that step in the flow and sends
users directly to the home page. We also remove the now-unused
initial_invite_page.html template, initial_invite.js (which pre-filled the
invite emails with characters from literature), and the /invite URL route.
2016-10-11 15:54:05 -07:00
Eklavya Sharma 41587edd2e bugdown: Change parameter name.
Change the parameter name of some functions from 'md' to 'content',
since the name 'md' seems to be the reason why this parameter was
wrongly annotated.
2016-10-11 11:31:01 -07:00
Eklavya Sharma f35d0f1c75 bugdown: Fix type annotations.
In some functions the first parameter was wrongly annotated as
`markdown.Markdown`. Change that to `text_type`.
2016-10-11 11:31:01 -07:00
Eklavya Sharma 81617fcdb0 Annotate zerver/tests/test_upload.py. 2016-10-11 11:29:14 -07:00
Eklavya Sharma f7092b1a90 Annotate zerver/tests/test_email_mirror.py. 2016-10-11 11:28:52 -07:00
Eklavya Sharma b741c7351d email_mirror: Use correct string types. 2016-10-11 11:27:35 -07:00
Eklavya Sharma ccce3ec0be zerver/lib/digest.py: Fix a type annotation. 2016-10-11 11:27:35 -07:00
Eklavya Sharma d92368e0f5 test_email_mirror: Add trivial annotations.
Add annotations of the form `type: () -> None` to test methods.
2016-10-11 11:27:35 -07:00
Eklavya Sharma 1dbabfe1ab Annotate zerver/tests/test_bugdown.py. 2016-10-11 11:27:35 -07:00
Eklavya Sharma 8e2f659f3d test_bugdown: Add trivial annotations.
Add annotations of the form `type: () -> None` to test methods.
2016-10-11 11:27:35 -07:00
Tomasz Kolek 8fa9ee1473 Add github tests for commits more than limit. 2016-10-10 17:39:05 -07:00
Tomasz Kolek ad5d5f2c99 Add beanstalk tests for commits more than limit. 2016-10-10 17:39:05 -07:00
Tomasz Kolek 07003ca52d Add gitlab tests for commits more than limit. 2016-10-10 17:39:05 -07:00
Tomasz Kolek a3e4415614 Add bitbucket2 tests for commits more than limit. 2016-10-10 17:39:05 -07:00
Tomasz Kolek 6fc1193743 Add to gitlab remove branch message handler. 2016-10-10 17:39:05 -07:00
Tomasz Kolek 312905007e Add EMPTY_SHA const to zerver/lib/webhooks/git.py. 2016-10-10 17:39:05 -07:00
Tomasz Kolek 89ef6c73d1 Normalize github and beanstalk remove branch event message. 2016-10-10 17:39:05 -07:00
Tomasz Kolek 56208d7ea1 Add bitbucket2 remove branch message handler. 2016-10-10 17:39:05 -07:00
Tomasz Kolek b031d6b029 Add bitbucket2 force push event handler. 2016-10-10 17:39:05 -07:00
Tomasz Kolek b69b5e55ce Add git integration method to build message for remove branch events.
Add get_remove_branch_event_message method to zerver/lib/webhooks/git.py
for normalizing remove branch message in all git integrations.
2016-10-10 17:39:05 -07:00
Tomasz Kolek f084dd05a8 Normalize github and beanstalk force push event message. 2016-10-10 17:39:05 -07:00
Tomasz Kolek 5f7c1519d1 Add git integration method to build message for force push events.
Add get_force_push_commits_event_message method to zerver/lib/webhooks/git.py
for normalizing force push message in all git integrations.
2016-10-10 17:39:05 -07:00
Tomasz Kolek e56e3f4aee Normalize bitbucket2 push event subject. 2016-10-10 17:39:05 -07:00
Tomasz Kolek 741a91d249 Normalize bitbucket push event subject. 2016-10-10 17:39:05 -07:00
Tomasz Kolek 1f0f197479 Normalize github and beanstalk push event subjects. 2016-10-10 17:39:05 -07:00
Tomasz Kolek 32934a02e9 Add template for subject that contains repo name and branch name. 2016-10-10 17:39:05 -07:00
Tomasz Kolek bc4ce7e781 Normalize gitlab push event message.
Fixes: #1884.
2016-10-10 17:39:05 -07:00
Tomasz Kolek 6a727bc476 Normalize bitbucket2 push event.
Normalize bitbukcet2 push event by creating it by
get_push_commits_event_message common function.
2016-10-10 17:39:05 -07:00
Tomasz Kolek 06ad0102a2 Remove now-unused build_commit_list_content. 2016-10-10 17:38:41 -07:00
Tomasz Kolek 809bc5f29b Normalize bitbucket push event.
Normalize bitbukcet push event by creating it by
get_push_commits_event_message common function.
2016-10-10 17:33:02 -07:00
Tomasz Kolek 1c1a003ba1 Normalize github push event.
Normalize github push event by creating it by
get_push_commits_event_message common function.
2016-10-10 17:33:02 -07:00
Tomasz Kolek 8697a1e5a2 Add common git integration method to build message for push events.
This adds get_push_commits_event_message method to
zerver/lib/webhooks/git.py for normalizing push messages in all git
integrations.
2016-10-10 17:32:41 -07:00
Kartik Maji e7f6f40100 Add test to check add peer event recipients. 2016-10-10 10:54:14 -07:00
K.Kanakhin e104943e12 storage: Fix build-release-tarball py3 compatibility.
- Open files in storage.py with 'rb' mode
  which allow to use decode command in PY3.
2016-10-10 09:54:51 -07:00
Umair Khan eb7ce73628 Move name_changes_disabled to zerver.models 2016-10-10 09:27:52 -07:00
Umair Khan 4fa427bc74 auth_backends: Add backend tests for subdomains logic.
Fixes: #1870
2016-10-10 08:55:39 -07:00
Umair Khan a32cee4b10 Add test to show error msg for invalid subdomain. 2016-10-10 08:42:34 -07:00
Umair Khan d0acda4635 Google: Show error on login page for wrong subdomain.
Takes care of Google OAuth.

Fixes: #1871
2016-10-10 08:42:34 -07:00
Umair Khan c23aaa1785 GitHub: Show error on login page for wrong subdomain.
While logging in through GitHub, if the user tries to login
to the wrong subdomain then show an appropriate message.
2016-10-10 08:42:34 -07:00
Umair Khan d7efdf050c Ignore docs directory while creating translation strings.
Fixes: #1941.
2016-10-10 08:23:56 -07:00
Umair Khan a9343a65d5 Don't localize javascript comments.
Fixes: #1934.
2016-10-10 08:21:02 -07:00
sonali0901 5623fa9994 request docs: Fix typo in spelling of "parameter". 2016-10-09 11:13:49 -07:00
Rishi Gupta ea050d5fd8 test_runner: Fix error message for failed import.
Previously, we suggested running
`python -c import zerver.tests.test_mytest`
when importing test_mytest failed, which doesn't work.

This commit adds the missing quotes, making it
`python -c 'import zerver.tests.test_mytest'`
2016-10-08 11:51:53 -07:00
Brock Whittaker 3efe3601b8 Allow for templates/settings to be read and compiled.
This allows for the handlebars templates in the template/settings
subdirectory to be found and compiled.
2016-10-05 22:24:22 -07:00
Rishi Gupta d387012bc6 Fix EXTERNAL_HOST computations in test_settings.py.
test_settings.py was setting EXTERNAL_HOST after importing settings.py,
which has several variables (like SERVER_URI) that are computed from
EXTERNAL_HOST.

[tweaked by tabbott to add comments explaining the story here].
2016-10-05 19:04:55 -07:00
Rishi Gupta 777fcaa6a0 Add new organization type field to Realm objects.
Adds a new field org_type to Realm.  Defaults for restricted_to_domain
and invite_required are now controlled by org_type at time of realm
creation (see zerver.lib.actions.do_create_realm), rather than at the
database level.  Note that the backend defaults are all
org_type=corporate, since that matches the current assumptions in the
codebase, whereas the frontend default is org_type=community, since if
a user isn't sure they probably want community.

Since we will likely in the future enable/disable various
administrative features based on whether an organization is corporate
or community, we discuss those issues in the realm creation form.
Before we actually implement any such features, we'll want to make
sure users understand what type of organization they are a member of.

Choice of org_type (via radio button) has been added to the realm
creation flow and the realm creation management command, and the
open-realm option removed.

The database defaults have not been changed, which allows our testing code
to work unchanged.

[includes some HTML/CSS work by Brock Whittaker to make it look nice]
2016-10-05 17:01:46 -07:00
Tomasz Kolek dbeab6aa6f Optimize checks of test database state by moving into Python.
Previously, the generate-fixtures shell script by called into Django
multiple times in order to check whether the database was in a
reasonable state.  Since there's a lot of overhead to starting up
Django, this resulted in `test-backend` and `test-js-with-casper`
being quite slow to run a single small test (2.8s or so) even on my
very fast laptop.

We fix this is by moving the checks into a new Python library, so that
we can avoid paying the Django startup overhead 3 times unnecessarily.
The result saves about 1.2s (~40%) from the time required to run a
single backend test.

Fixes #1221.
2016-10-05 10:40:19 -07:00
umkay d260a22637 Add a new statistics/analytics framework.
This is a first pass at building a framework for collecting various
stats about realms, users, streams, etc. Includes:
* New analytics tables for storing counts data
* Raw SQL queries for pulling data from zerver/models.py tables
* Aggregation functions for aggregating hourly stats into daily stats, and
  aggregating user/stream level stats into realm level stats
* A management command for pulling the data

Note that counts.py was added to the linter exclude list due to errors
around %%s.
2016-10-04 17:18:54 -07:00
Steve Howell 43e21f6fb4 Remove unused bugdown global from models.py.
Fixes: #1379

(It was really a series of fixes leading up to this.)
2016-10-04 11:35:03 -07:00
Steve Howell b2f84f0fa4 Move render_markdown() to lib/message.py.
This removes a bugdown circular dependency.
2016-10-04 11:34:53 -07:00
Steve Howell b81a961fdd Remove Message.set_rendered_content(). 2016-10-04 11:31:21 -07:00
Steve Howell 7fb992dba3 Simplify and "fix" render_old_messages management command.
The command to render old messages now looks for all messages
not matching the bugdown version, and it no longer directly calls
into model code.  We should still be extremely cautious about
using this code.
2016-10-04 11:31:20 -07:00
Steve Howell 6b71f5bd5f Inline most calls to set_rendered_content().
This is part of breaking the circular dependency on
bugdown in models.py.  A subsequent commit will fully
kill off set_rendered_content().
2016-10-04 11:31:20 -07:00
Steve Howell 583a6bbadd Extract zerver/lib/message.py.
This pulls message-related code from models.py into a new
module called message.py, and it starts to break some bugdown
dependencies.  All the methods here are basically related to
serializing Message objects as dictionaries for caches and
events.

    extract_message_dict
    stringify_message_dict
    message_to_dict
    message_to_dict_json
    MessageDict.to_dict_uncached
    MessageDict.to_dict_uncached_helper
    MessageDict.build_dict_from_raw_db_row
    MessageDict.build_message_dict

This fix also removes a circular dependency related
to get_avatar_url.

Also, there was kind of a latent bug in Message.need_to_render_content
where it was depending on other calls to Message to import bugdown
and set it globally in the namespace.  We really need to just
eliminate the function, since it's so small and used by code that
may be doing very sketchy things, but for now I just fix it.  (The
bug would possibly be exposed by moving build_message_dict out to the
library.)
2016-10-04 11:31:20 -07:00
Steve Howell ac994fdd51 Move three functions from models.py to lib/cache.py.
I move these three functions to lib/cache.py:

    to_dict_cache_key_id
    to_dict_cache_key
    flush_message

This will prepare us for a more significant refactoring that
eventually breaks down some circular dependencies with
Message and bugdown.
2016-10-04 11:31:20 -07:00
Steve Howell 69fa60080c tests: Test get_prereg_user_by_email(). 2016-10-04 10:35:43 -07:00
Steve Howell ad149141f3 tests: Test Subscription.__unicode__(). 2016-10-04 10:35:43 -07:00
Steve Howell 5ae8cff289 code cleanup: Remove unused Attachment methods.
This removes some unused code on Attachment.  Some of these
methods might be useful in a manage.py shell, but without tests,
they are not very trustworthy, and the Attachment model isn't that
complicated to write raw Django queries against.
2016-10-04 10:35:43 -07:00
Steve Howell 732bf5e6d0 tests: Test Attachment.__unicode__(). 2016-10-04 10:35:43 -07:00
Steve Howell cfc4010667 Add test code for UserMesssage.__unicode__(). 2016-10-04 10:35:43 -07:00
Steve Howell fc2329ccb7 tests: Add test_is_status_message(). 2016-10-04 10:35:43 -07:00
Tim Abbott ac2007dd9b resize_avatar: Add a size option. 2016-10-02 21:29:04 -07:00
Tim Abbott 87b9017845 upload: Extract get_avatar_url and add to UploadBackend. 2016-10-02 21:20:43 -07:00
Tim Abbott 22fd7ba02a avatar: Move avatar hash computations to their own file. 2016-10-02 21:19:10 -07:00
baali 142dce2cd4 Replace legacy endpoint for renaming a stream and add tests.
This moves the logic for renaming a stream to the REST API
update_stream_backend method, eliminating the legacy API endpoint for
doing so.

It also adds a nice test suite covering international stream names.
2016-10-02 20:33:49 -07:00
Rishi Gupta 48bec80c61 Fix test_signup.LoginTest.test_register.
The test currently expects 67 database queries during registration, but we
added two more (likely calls to resolve_subdomain_to_realm) in ea39fb2.
2016-10-02 19:16:24 -07:00