Commit Graph

1196 Commits

Author SHA1 Message Date
Tomasz Kolek 89d53a0640 Change name of PR subject template to be applicable for issues. 2016-10-20 23:15:30 -07:00
Tomasz Kolek 289f400c8c Add git integrations method to build message for issues events. 2016-10-20 23:15:30 -07:00
Arpith Siromoney ee97ba04fe Add an API endpoint to send typing notification events.
POST to /typing creates a typing event
Required parameters are 'op' ('start' or 'stop') and 'to' (recipient
emails). If there are multiple recipients, the 'to' parameter
should be a JSON string of the list of recipient emails.
The event created looks like:
{
  'type': 'typing',
  'op': 'start',
  'sender': 'hamlet@zulip.com',
  'recipients': [{
    'id': 1,
    'email': 'othello@zulip.com'
  }]
}
2016-10-20 23:05:18 -07:00
Tim Abbott 3d183ffe23 text_fixtures: Fix case where source_hash_file doesn't exist.
We unfortunately just broke provisioning because we didn't correctly
handle the case where the source_hash_file didn't exist.
2016-10-20 22:21:45 -07:00
Steve Howell 409a84698c Handle peer_remove events correctly for subscriptions.
We now send peer_remove events to folks who have never subscribed
to the streams (except for private streams and zephyr).

We also use logic that is more similar to how
bulk_add_subscriptions() works.
2016-10-20 22:05:19 -07:00
Steve Howell 68ae4da626 Extract query_all_subs_by_stream().
This code was in bulk_add_subscriptions, but we want the same logic
to eventually happen in bulk_remove_subscriptions.
2016-10-20 22:05:19 -07:00
Steve Howell d13b0554b3 Remove emails_by_stream in bulk_add_subscription().
There are so many collections in this function, it's hard to keep
them all straight.  This one is fairly redundant, so it was easy
to remove.
2016-10-20 22:05:19 -07:00
Steve Howell d706bfa69f Move some code from notify_subscriptions_removed().
There are two reasons for this change.  First, we want to be
consistent with notify_subscriptions_added(), which doesn't
handle "peer" events.  Second, we want to fix this code in a
subsequent commit not to do one user at a time, which is
inefficient.
2016-10-20 22:05:19 -07:00
Steve Howell dda4c61f2a Remove do_remove_subscription(). 2016-10-20 22:05:19 -07:00
Steve Howell 4ae117ed14 Use bulk_remove_subscriptions() in do_deactivate_stream().
We are deprecating do_remove_subscription().
2016-10-20 22:05:19 -07:00
Steve Howell 44bbece5a6 Remove do_remove_subscription() calls in tests.py. 2016-10-20 22:05:19 -07:00
Steve Howell b2d06fad42 Extract get_peer_user_ids_for_stream_change(). 2016-10-20 22:05:19 -07:00
Steve Howell 5eee77fe81 Remove do_add_subcription(). 2016-10-20 22:05:19 -07:00
Steve Howell 78b73a0c91 Replace do_add_subscription() in test_helpers. 2016-10-20 22:05:19 -07:00
Umair Khan 45166cb39c is_template_database_current: Check populate_db and postgres-init-test-db.
Compare the hash of 'zilencer/management/commands/populate_db' with
'var/populate_db_hash' and 'tools/setup/postgres-init-test-db' with
'var/postgres_init_test_db_hash' and if any comparison fails rebuild
the database.

With fixes from tabbott.
2016-10-20 22:04:47 -07:00
K.Kanakhin 7bc0c7f0d5 bugdown: Fix patterns ordering in Markdown extension.
- Put 'modal_link' and 'link' inline patterns after avatar inline pattern
  to correct applying them to parsed text.

Fixes:  #1654
2016-10-20 11:18:41 -07:00
Tim Abbott 50356cc325 event_queue: Eliminate buggy need_timeout heartbeat code.
As best I can tell, this option was completely confused in two ways:
* The name is confusing; it actually controls whether we _clear_ the
  timeout associated with the current handler
* It's not clear why one would not want it to be unconditionally true.

From reading the history, I'm pretty sure I had just misread the code
when I created this.

And this caused a real bug; a later refactoring caused us to basically
never cancel the timeouts, which in turn resulted in 90% of all events
traffic being hearbeats with a much lower frequency (~5s) than the
intended 45s.  Removing this code fixes that nasty bug.
2016-10-19 14:32:49 -07:00
Tim Abbott a6e779a56c events: Add event types to logging.
This should make it a lot easier to analyze our events traffic and
debug potential issues with it.
2016-10-19 14:32:16 -07:00
Rishi Gupta 83f902953a newsletter signup: Update merge_vars.
merge_vars is the user meta data we store in mailchimp. This commit changes
what we store from realm.domain to realm.id, since realm.domain is being
deprecated, and changes the OPTIN_TIME to a nicer format.
2016-10-18 19:29:12 -07:00
Brock Whittaker 77233c7e3b bugdown: Distinguish between YouTube Videos and Image Previews.
This distinguishes between YouTube Videos and Image Previews by adding
a particular “youtube-video” class to the preview along with changing
the title to the video ID rather than the link. This serves to allow
the lightbox to ID when a lightbox preview should be treated like a
YouTube video rather than an image preview.

This also modifies the tests in bug down to expect a youtube-video class
along with the title to just be the video ID on YouTube rather than the
entire URL link.
2016-10-18 14:45:47 -07:00
Tim Abbott 56d0cc69e9 socket: Remove unused djsession_engine. 2016-10-17 23:19:28 -07:00
Mohsen Ibrahim bfe5787197 Fix Django.utils.importlib deprecation warnings.
This changes `from django.utils.importlib import import_module` to
`from importlib import import_module`, as `django.utils.importlib` will
be removed in django version 1.9.
2016-10-17 23:19:28 -07:00
Mohsen Ibrahim e9899e4dc9 bugdown: Fix some Python-Markdown deprecation warnings.
Python-Markdown changed how one is supposed to declare extensions
being used.
2016-10-17 23:13:41 -07:00
Tim Abbott 63a49be09e event_queue: Import tornado.ioloop correctly.
We actually use tornado.ioloop, not the root tornado module, and while
the current code does work, it causes mypy errors with the incremental
cache.
2016-10-17 17:57:46 -07:00
Tim Abbott 4a4664d268 mypy: Remove a bunch of now-unnecessary type: ignore annotations.
Since mypy and typeshed have advanced a lot over the last several
months, we no longer need these `type: ignore` annotations.
2016-10-17 11:48:34 -07:00
Tim Abbott e37b6488b8 Bugdown: Remove checked-in CodeHilite extension.
The changes that required us to fork this extension had been merged
into upstream CodeHilite, so we can remove it and switch to using the
version that comes with python-markdown.
2016-10-16 00:40:08 -07:00
umkay c028815385 bugdown: Update python-markdown to version 2.6.7.
This updates Bugdown to reflect the changes in the updated
markdown. In particular, we now pass a default config object in the
__init__ for the Bugdown extension, update the make_md_engine function
to take kwargs as opposed to a config list, and have UListProcessor
inherit from ulist as opposed to olist (which no longer works).

We update the (forked from upstream) fenced_code extension's
makeExtension to take args and kwargs, and update
FencedBlockPreprocessor __init__ method with updated Codehilite
arguments.

We update the (forked from upstream) Codehilite extension to
mirror the logic with the latest upstream Codehilite:
	Add parse_hl_lines function
	update makeExtension to take args and kwarfs instead of config
	list
	Add regex for highlight lines
	use linenums instead of linenos
	use get_formatter_by_name instead of HtmlFormatter
	user get_lexer_by_name instead of TextLexer
	add hl_lines and use_pygments arguments to the codehlite
	constructor
2016-10-15 23:38:32 -07:00
Tomasz Kolek 2941c1f517 Move converting commits data to text to separate function.
Create get_commits_content function for converting
commits list to text.
2016-10-14 16:54:13 -07:00
Tomasz Kolek d2cedd3667 Rename consts in lib/webhooks/git.py to make it more general.
Rename:
   PUSH_COMMITS_LIMIT to COMMITS_LIMIT
   PUSH_COMMIT_ROW_TEMPLATE to COMMIT_ROW_TEMPLATE
   PUSH_COMMITS_MORE_THAN_LIMIT_TEMPLATE to COMMITS_MORE_THAN_LIMIT_TEMPLATE
2016-10-14 16:54:13 -07:00
Tomasz Kolek ee70cc5141 Add git integration method to build message for pull requests events. 2016-10-14 16:54:13 -07:00
umkay 278574a6ec Fix cursor() arguments for connection classes.
In order to use the latest version of psycopg2 with the latest version
of sqlalchemy we must integrate the changes made in the 2.4.6 release of
psycopg2. See
c86ca7687f
and the release notes for psycopg2 2.4.6.
2016-10-14 13:35:06 -07:00
Rishi Gupta 82b814a1cd analytics: Simplify frequency and measurement interval options.
Change the CountStat object to take an is_gauge variable instead of a
smallest_interval variable. Previously, (smallest_interval, frequency)
could be any of (hour, hour), (hour, day), (hour, gauge), (day, hour),
(day, day), or (day, gauge).
The current change is equivalent to excluding (hour, day) and (day, hour)
from the list above.

This change, along with other recent changes, allows us to simplify how we
handle time intervals. This commit also removes the TimeInterval object.
2016-10-14 10:18:37 -07:00
Tim Abbott 0f1468e64f do_update_message_flags: Use access_message for check. 2016-10-11 17:20:15 -07:00
Tim Abbott f1a399a4e1 message: Create new access_message library function.
With reactions and other upcoming features, we'll be adding several
places where we need to check whether a particular user can access a
particular message.  It's best to just have a single helper function
for this purpose that we can use everywhere.
2016-10-11 17:17:19 -07:00
Eklavya Sharma 41587edd2e bugdown: Change parameter name.
Change the parameter name of some functions from 'md' to 'content',
since the name 'md' seems to be the reason why this parameter was
wrongly annotated.
2016-10-11 11:31:01 -07:00
Eklavya Sharma f35d0f1c75 bugdown: Fix type annotations.
In some functions the first parameter was wrongly annotated as
`markdown.Markdown`. Change that to `text_type`.
2016-10-11 11:31:01 -07:00
Eklavya Sharma ccce3ec0be zerver/lib/digest.py: Fix a type annotation. 2016-10-11 11:27:35 -07:00
Tomasz Kolek 312905007e Add EMPTY_SHA const to zerver/lib/webhooks/git.py. 2016-10-10 17:39:05 -07:00
Tomasz Kolek b69b5e55ce Add git integration method to build message for remove branch events.
Add get_remove_branch_event_message method to zerver/lib/webhooks/git.py
for normalizing remove branch message in all git integrations.
2016-10-10 17:39:05 -07:00
Tomasz Kolek 5f7c1519d1 Add git integration method to build message for force push events.
Add get_force_push_commits_event_message method to zerver/lib/webhooks/git.py
for normalizing force push message in all git integrations.
2016-10-10 17:39:05 -07:00
Tomasz Kolek 32934a02e9 Add template for subject that contains repo name and branch name. 2016-10-10 17:39:05 -07:00
Tomasz Kolek 8697a1e5a2 Add common git integration method to build message for push events.
This adds get_push_commits_event_message method to
zerver/lib/webhooks/git.py for normalizing push messages in all git
integrations.
2016-10-10 17:32:41 -07:00
sonali0901 5623fa9994 request docs: Fix typo in spelling of "parameter". 2016-10-09 11:13:49 -07:00
Rishi Gupta ea050d5fd8 test_runner: Fix error message for failed import.
Previously, we suggested running
`python -c import zerver.tests.test_mytest`
when importing test_mytest failed, which doesn't work.

This commit adds the missing quotes, making it
`python -c 'import zerver.tests.test_mytest'`
2016-10-08 11:51:53 -07:00
Rishi Gupta 777fcaa6a0 Add new organization type field to Realm objects.
Adds a new field org_type to Realm.  Defaults for restricted_to_domain
and invite_required are now controlled by org_type at time of realm
creation (see zerver.lib.actions.do_create_realm), rather than at the
database level.  Note that the backend defaults are all
org_type=corporate, since that matches the current assumptions in the
codebase, whereas the frontend default is org_type=community, since if
a user isn't sure they probably want community.

Since we will likely in the future enable/disable various
administrative features based on whether an organization is corporate
or community, we discuss those issues in the realm creation form.
Before we actually implement any such features, we'll want to make
sure users understand what type of organization they are a member of.

Choice of org_type (via radio button) has been added to the realm
creation flow and the realm creation management command, and the
open-realm option removed.

The database defaults have not been changed, which allows our testing code
to work unchanged.

[includes some HTML/CSS work by Brock Whittaker to make it look nice]
2016-10-05 17:01:46 -07:00
Tomasz Kolek dbeab6aa6f Optimize checks of test database state by moving into Python.
Previously, the generate-fixtures shell script by called into Django
multiple times in order to check whether the database was in a
reasonable state.  Since there's a lot of overhead to starting up
Django, this resulted in `test-backend` and `test-js-with-casper`
being quite slow to run a single small test (2.8s or so) even on my
very fast laptop.

We fix this is by moving the checks into a new Python library, so that
we can avoid paying the Django startup overhead 3 times unnecessarily.
The result saves about 1.2s (~40%) from the time required to run a
single backend test.

Fixes #1221.
2016-10-05 10:40:19 -07:00
umkay d260a22637 Add a new statistics/analytics framework.
This is a first pass at building a framework for collecting various
stats about realms, users, streams, etc. Includes:
* New analytics tables for storing counts data
* Raw SQL queries for pulling data from zerver/models.py tables
* Aggregation functions for aggregating hourly stats into daily stats, and
  aggregating user/stream level stats into realm level stats
* A management command for pulling the data

Note that counts.py was added to the linter exclude list due to errors
around %%s.
2016-10-04 17:18:54 -07:00
Steve Howell b2f84f0fa4 Move render_markdown() to lib/message.py.
This removes a bugdown circular dependency.
2016-10-04 11:34:53 -07:00
Steve Howell 7fb992dba3 Simplify and "fix" render_old_messages management command.
The command to render old messages now looks for all messages
not matching the bugdown version, and it no longer directly calls
into model code.  We should still be extremely cautious about
using this code.
2016-10-04 11:31:20 -07:00
Steve Howell 6b71f5bd5f Inline most calls to set_rendered_content().
This is part of breaking the circular dependency on
bugdown in models.py.  A subsequent commit will fully
kill off set_rendered_content().
2016-10-04 11:31:20 -07:00